Deliverable D2.4
Guidelines for Pilot Training for a Personal Aerial Vehicle
Contractual delivery date:
November 2014
Actual delivery date:
Partner responsible for the Deliverable: University of Liverpool, UoL
Author(s):
Drs. M. Jump; M.D. White; L. Lu
Dissemination level1
PU Public X
PP Restricted to other programme participants (including the Commission Services)
RE Restricted to a group specified by the consortium (including the Commission
Services)
CO Confidential, only for members of the consortium (including the Commission
Services)
1 Dissemination level using one of the following codes: PU = Public, PP = Restricted to other programme
participants (including the Commission Services), RE = Restricted to a group specified by the consortium
(including the Commission Services), CO = Confidential, only for members of the consortium (including the
Commission Services)
Project No. 266470 Deliverable D2.4
i
Document Information Table
Grant agreement no. ACPO-GA-2010-266470
Project full title myCopter – Enabling Technologies for Personal Air
Transport Systems
Deliverable number D2.4
Deliverable title Guidelines for Improved Training Effectiveness through
Use of New Control and Information Systems
Nature2 P
Dissemination Level PU
Version 1.0
Work package number WP2
Work package leader UoL
Partner responsible for Deliverable UoL
Reviewer(s) Dr. M. Jump (UoL, December 2014)
Dr. M.D. White (UoL, December 2014)
The research leading to these results has received funding from the European Community's Seventh
Framework Programme (FP7/2007-2013) under grant agreement no 266470.
The author is solely responsible for its content, it does not represent the opinion of the European
Community and the Community is not responsible for any use that might be made of data appearing
therein.
2 Nature of the deliverable using one of the following codes: R = Report, P = Prototype, D = Demonstrator, O = Other
Project No. 266470 Deliverable D2.4
ii
Revision Table
Version Date Author Comments
1.0 24/12/2014 As front
sheet
Initial Issue
Project No. 266470 Deliverable D2.4
iii
Executive Summary
This report describes research activities conducted at the University of Liverpool as part of the
myCopter project into the development of training requirements for pilots of Personal Aerial Vehicles
(PAVs). The work has included a Training Needs Analysis (TNA) to determine the skills that need to
be developed by a PAV pilot and the development of a training programme that covers the
development of the skills identified by the TNA. The effectiveness of the training programme has
been evaluated using the first three Levels of Kirkpatrick’s method. The evaluation showed that the
developed training programme was effective, in terms of engaging the trainees with the subject, and
in terms of developing the skills required to fly a series of PAV-mission related tasks in a flight
simulator.
Project No. 266470 Deliverable D2.4
iv
Table of Contents
Document Information Table ................................................................................................................... i
Executive Summary ................................................................................................................................. iii
Table of Contents .................................................................................................................................... iv
Notation and Glossary .............................................................................................................................. v
1. Introduction ..................................................................................................................................... 1
2. TRAINING for Drivers and Pilots – Existing Requirements and Practice ......................................... 3
2.1. Car Drivers in the UK ............................................................................................................... 3
2.2. Pilot Training in the UK ............................................................................................................ 4
2.3. Discussion of Existing Training Paradigms ............................................................................... 4
3. Proposed PAV Training Syllabus ...................................................................................................... 5
3.1. Key Skills for PAV Pilots ........................................................................................................... 5
3.2. Construction of PAV Training Programme .............................................................................. 6
4. Results ............................................................................................................................................. 7
4.1. Training Duration..................................................................................................................... 7
4.2. Level 1 Evaluation – Participant Satisfaction ........................................................................... 8
4.3. Level 2 Evaluation – Skills Test ................................................................................................ 9
4.4. Level 3 Evaluation – Real-World Commute ........................................................................... 11
5. Discussion ...................................................................................................................................... 14
6. Concluding Remarks ...................................................................................................................... 15
7. References ..................................................................................................................................... 16
Appendices ............................................................................................................................................ 17
A. Training Exercises ...................................................................................................................... 17
B. Progress Record Sheets ............................................................................................................... 1
Project No. 266470 Deliverable D2.4
v
Notation and Glossary ACAH Attitude Command, Attitude Hold
ACSH Acceleration Command, Speed Hold
CAA Civil Aviation Authority
CBD Central Business District
DSA Driving Standards Agency
GA General Aviation
HITS Highway-in-the-Sky
HMI Human-Machine Interface
HQs Handling Qualities
HUD Head Up Display
IMC Instrument Meteorological Conditions
MTE Mission Task Element
PATS Personal Aerial Transportation System
PAV Personal Aerial Vehicle
PPL(A) Private Pilot’s License (Aeroplane)
PPL(H) Private Pilot’s License (Helicopter)
RC Rate Command
SEP Single Engine Piston
TLX Task Load Index
TNA Training Needs Analysis
TRC Translational Rate Command
TS Test Subject
VFR Visual Flight Rules
VRC Vertical Rate Command
C Sideslip Angle Command
C Flight Path Angle Command
Project No. 266470 Deliverable D2.4
1
1. Introduction Research is underway in the European Union Framework Programme 7-funded project myCopter to
enable the technologies required to realise the concept of the Personal Aerial Vehicle (PAV) and
hence make their mass adoption possible [1]. The research activities of the myCopter project can be
categorised into three main themes:
1) Human-Machine Interaction (HMI), including cockpit technologies for inceptors and displays,
and vehicle handling characteristics;
2) Autonomous flight capabilities, including vision-based localisation and landing point
detection, swarming and collision detection and avoidance;
3) Socio-economic aspects of a Personal Aerial Transportation System (PATS) – the
requirements for such a system to become accepted and widely adopted by the general
public.
Within this framework, two approaches to the operation of the PAV have been considered. The first
of these is conceived as a fully automatic or even autonomous vehicle that is capable of completing
an entire flight by itself, with input from the occupant only in terms of routing and (in the case of the
automatic vehicle) observation and monitoring of the vehicle’s systems [2;3]. The second approach,
perhaps for earlier versions of a PAV, would require the human occupant to control some, or all, of
the piloting functions of the vehicle. For mass adoption to be feasible, however, it is considered
necessary that the PAV be much less costly to acquire and operate than existing General Aviation (GA)
aircraft – either fixed- or rotary-wing. One element of these costs is training, both initial and that
required to remain current. It was hypothesised that savings could be achieved here by creating PAV
responses that are highly intuitive and that can be learned and understood quickly. In essence, the
PAV would have to have excellent Handling Qualities (HQ) designed into it from the very beginning.
Within the first of the themes identified above therefore, HQ requirements for the PAV have been
examined. The work has included the identification of response types (i.e. the manner in which the
vehicle responds following a cockpit control input) that permit ‘flight-naïve’ pilots (those with little or
no previous flight experience) with a broad range of aptitudes for flight tasks to rapidly develop the
skills required to operate a PAV simulation safely and repeatedly with a high degree of precision [4-6].
This work showed that a vehicle that offered a Translational Rate Command (TRC) response type (i.e.
the vehicle moves at a constant velocity over the ground for a constant stick deflection) in hover and
at low speeds could be operated by a wide range of test subjects, with minimal instruction. This was
found to be the case in both good environmental conditions, and in the presence of atmospheric
disturbances and a degraded visual environment.
The present report extends the previous research to consider the quantity and type of training that
would be required by prospective PAV pilots in order to be qualified to operate a manually-piloted
aircraft. A PAV training syllabus has been developed, and used to train a group of volunteers who
had no previous flying experience.
The report describes the development of the syllabus, based on a Training Needs Analysis (TNA) [7]
for PAV flight, and current ‘best practice’ for the training of both private pilots (both helicopter
(PPL(H)) and aeroplane (PPL(A))), and car drivers. Whilst current PPL training may be thought of as
being more directly applicable to the PAV, in the scenario of mass adoption of the PAV, many trainee
PAV pilots would already have some knowledge and experience of car driving, and so commonality
(where feasible) would permit more effective transfer of this knowledge to the PAV training.
Project No. 266470 Deliverable D2.4
2
Further, the report presents the results of trials conducted using the University of Liverpool
HELIFLIGHT-R flight simulator [8] in which the volunteers were trained using the syllabus developed
for that purpose. The aims of the trials were to study the effectiveness of the training syllabus and to
explore the likely length of time required to complete the training for a range of test subjects.
Many methods have been developed for the assessment of training programmes, but perhaps the
most widely-used is Kirkpatrick’s Four Level model [9;10]. The four levels of evaluation allow the
effectiveness of the training to be evaluated in terms of the trainee’s engagement and satisfaction
(Level 1), immediate demonstration of the learning that has been achieved (Level 2), longer-term
application of the learning to the trainee’s job (Level 3) and finally the benefit to the organisation
from the trainee’s new skills (Level 4).
In the context of the evaluation of the PAV training syllabus, the first level was accomplished using
questionnaires that were completed by each participant at the end of their training. For the second
level evaluation, the participants undertook a final ‘skills test’, in which they flew a series of
manoeuvres related to the PAV’s role. The third level evaluation took the form of a ‘real-world’ PAV
flight that the participants were asked to fly. For both the second and third level evaluations, the
measurement of the precision achieved and level of control activity allowed the degree of success to
be measured. A fourth level evaluation could take the form of long-term assessment of the PAV pilot
while flying the real aircraft. As the scope of current PAV research is limited to simulation only, it will
not be feasible to conduct the fourth level evaluation during this project.
The structure of the evaluation of the training can take several forms [10]. These generally involve a
period of training followed by a post-training test to measure final performance. A pre-training test
can also be included to measure initial performance prior to training. More complex evaluation
structures can involve the use of control groups who do not receive training, in order to evaluate the
impact of external factors on the evaluation.
For the PAV training evaluation, time restrictions in terms of the availability of the simulator
prevented the use of a control group. Pre-training testing of role-specific tasks (i.e. actual flying in
the simulator) would have significantly impacted on the outcomes of the evaluations due to the
(intended) highly intuitive nature of the system being used during the training – i.e. the participants
would have been able to self-learn to a considerable extent while completing the pre-training test,
which would affect the quantity of training required while following the syllabus. Hence, evaluation
of the efficacy of the PAV training syllabus has been performed on the basis of post-training
performance only. The ability to successfully complete a ‘skills test’ and a ‘real-world’ evaluation has
been taken as the means to show that the participant has acquired the necessary skills for to fly a
PAV. Whilst the enforced absence of a pre-training test evaluation does impinge upon the ability to
directly measure the skills gained during the training programme, the use of an aptitude test to
assess natural flying ability (e.g. hand-eye coordination) allowed the performance of each participant
to be placed in context [5]. Furthermore, as none of the participants in these tests possessed any
previous flying experience (all had some driving experience, this is discussed in further detail in the
Results Section), and hence none had pre-existing directly-relevant knowledge, it has been assumed
that all of the participants started the training programme from an equivalent level of relevant
knowledge and skill.
Project No. 266470 Deliverable D2.4
3
2. Training for Drivers and Pilots – Existing Requirements and Practice This Section describes the current requirements, and typical practice, associated with training car
drivers and private pilots in the UK today. The primary sources for the information discussed on
actual practice in this Section are interviews conducted with highly experienced driving and flying
instructors – each with more than 15 years of practical training experience.
2.1. Car Drivers in the UK
UK car drivers are expected to be able to meet certain standards in terms of their actions on the road
and their knowledge of the ‘Highway Code’ – the rules that govern their driving behaviour. These
standards are set out by the UK’s Driving Standards Agency (DSA) [11]. The DSA also publishes a
national driving syllabus [12] that covers all points of learning – including the development of skills
and abilities and the acquisition of knowledge and understanding, required to meet the published
standards. The national syllabus is not, however, compulsory, and many driving instructors have
developed their own methods by which to train their students in the required skills. This often
involves breaking down the learning process into separate, grouped, components – for instance basic
vehicle control, road skills, interacting with other road users and so on. Within each of these
groupings, there might be 10-20 individual skills or knowledge items to be covered. These might
include, changing gear, steering, braking and clutch control etc. in the basic vehicle skills category
and signalling, road markings and junctions in the road skills category.
For each item of learning, an instructor will typically introduce the concept using graphical aids
(typically report-based, but increasingly using electronic means such as videos), and will then ask the
student to attempt the task relating to a particular skill. Progress is monitored according to the
amount of guidance that the instructor needs to supply to the student. At the beginning, this would
consist of comprehensive guidance of every stage of a given task, with the instructor telling the
student exactly what they need to do. As the student develops their skills, the instructor will be able
to reduce their input to prompts only, and eventually the student should be able to complete the
task independently.
The judgement as to when a learner driver is performing to an acceptable standard is typically a
subjective decision made by an instructor. Anecdotally, this may be performed on the basis of
whether or not the instructor would be happy for the learner to drive with members of the
instructor’s family in the car.
The UK driving examination takes place in two stages. The first of these is a computer-based theory
test, which assesses the candidate’s knowledge of the Highway Code. The second, the practical
driving test, has a duration of 40 minutes. During this time, the examiner will ask the student to
conduct a set of ‘standard’ manoeuvres (such as reversing around a corner, hill starts and so on) in
addition to general driving, as directed by the examiner. Recently an ‘independent driving’ element
has been introduced to the test in order to check on a student’s driving ability whilst following traffic
signs and making their own driving decisions. The examiner will judge (again, relatively subjectively)
whether the candidate is performing to an acceptable standard. Minor driving faults do not directly
result in test failure, but an accumulation of a sufficient number (either overall or within a single
category) will result in a failure. More serious faults, or indeed dangerous manoeuvres, will result in
immediate failure of the test.
Project No. 266470 Deliverable D2.4
4
2.2. Pilot Training in the UK
Pilot training in the UK is standardised to a much greater extent than is the case for driver training.
For fixed-wing aircraft, nineteen standard ‘lessons’ (although they may take more or less than one
actual flying session) have been specified by the UK Civil Aviation Authority (CAA), and are taught by
all flying schools. For helicopters, there are 27 ‘lessons’, the additional sessions being focussed on
hover and low speed operations. Each lesson covers a particular subject (e.g. the effect of the
controls, straight and level flight, turning flight etc.). Each lesson begins with a pre-flight briefing in
which the subject will be introduced, and the appropriate terminology defined. In the air, the
instructor will generally demonstrate the correct procedure, and then hand control to the student to
allow them to make their own attempt. By subsequently coaching the student through the
procedure (i.e. providing detailed, step-by-step instructions), appropriate behaviours are instilled and
refined until an acceptable standard has been achieved.
Unlike driver training, where progress is largely judged subjectively, pilot training involves the use of
some objective measures with associated tolerances – in height, heading, airspeed etc. (e.g. ±150ft in
height, ±15kts in airspeed during cruising flight etc. [13]) – to judge whether a student pilot has
attained an acceptable level of performance. A subjective element remains however, with the
instructor making judgements regarding the appropriateness of the student’s actions in terms of
ensuring the safe operation of the aircraft (for example, having an appropriate mental approach (e.g.
keeping ‘ahead’ of the aircraft), the ability to multi-task etc.). In addition to these checks, during the
course of a lesson, three ‘Progress Tests’ are defined in the PPL syllabus. These are designed to verify
that the student pilot is able to demonstrate the techniques that have been learned during the
lessons.
As with learning to drive, becoming a licensed pilot involves the completion of both theory and
practical exams. A PPL student must pass nine theory exams, covering subjects such as Air Law,
Human Performance and Navigation. The practical flying skills test includes navigation, circuits and
dealing with a simulated engine failure, in addition to general handling. The examiner will use both
the quantitative tolerances of height, heading and airspeed, and subjective judgement to determine
whether or not a student has successfully passed the practical test.
2.3. Discussion of Existing Training Paradigms
It is evident from the commentary above that there are a number of similarities in terms of the
methods used to train pilots and car drivers – particularly, in terms of the way in which new
techniques are introduced to a student, and in which progress is assessed. In both scenarios,
learners are introduced to new concepts progressively, and are not expected to master control of all
aspects of their vehicle simultaneously. Similarities also exist in the methods used to examine
competency – with theory exams and practical tests in both cases.
While there are common elements to the methods described above for car driving and flying
instruction and examination, a number of additional limitations are imposed on a PPL student. Firstly,
it is a legal requirement that a trainee pilot must accumulate a minimum quantity of ‘hands-on’
learning prior to being able to acquire a license. This is a minimum of 45 hours, which must include
at least 25 hours of ‘instructed’ flight and 10 hours of ‘solo’ flight, and should also include at least 5
hours of ‘cross-country’ flying – which requires the student to exercise their navigation skills. PPL
Project No. 266470 Deliverable D2.4
5
students are also required to meet more stringent medical standards, although a discussion of these
is beyond the scope of the current report.
Secondly, a newly-qualified driver can drive any four-wheeled vehicle with a total mass of less than
3.5 tonnes, in any environmental conditions. A newly-qualified PPL(A)-holder is limited to basic
Single Engine Piston (SEP) aircraft. Any additional features that complicate the operation of the
aircraft (for example retractable undercarriage, multiple engines etc.), require separate ‘type ratings’
for that particular aircraft. With the PPL(H), aircraft types are even more restricted – every individual
helicopter type is covered by its own type rating. Further, basic PPL-holders are allowed to fly only
during daylight hours and in Visual Flight Rules (VFR) conditions. To fly in more adverse conditions,
pilots require additional training and further qualifications (the Night Qualification and IMC Rating,
respectively).
3. Proposed PAV Training Syllabus
3.1. Key Skills for PAV Pilots
At an early stage in the myCopter project, an outline ‘commuting’ scenario was developed to inform
the subsequent research [1]. This scenario requires the PAV to perform a vertical take-off from a
residential location, climb and accelerate to cruising flight. Upon reaching the destination in the
Central Business District (CBD) of a city, the PAV must descend and decelerate to a hover above the
landing point, following which the landing is performed vertically. Using this description as a basis, a
list of manoeuvres that would need to be performed by a PAV pilot was developed. These, in turn,
were used to identify the skills that the PAV pilot would need to demonstrate for manual flight,
based on the ideal PAV response characteristics identified in the earlier myCopter research [4-6].
In total, 24 key skills have been identified that relate to manual PAV handling. These are as follows:
1. Use of longitudinal inputs in hover to control forward speed (TRC response type);
2. Use of lateral inputs in hover to control lateral speed (TRC response type);
3. Combined use of longitudinal and lateral inputs to control horizontal flight path angle;
4. Use of pedals in hover to control heading and yaw rate (Rate Command (RC) response type);
5. Use of the collective lever in hover to control height and vertical rate (Vertical Rate Command
(VRC) response type);
6. Combined use of pedals and lateral inputs at low speed (<25kts) to improve turn coordination;
7. Use of longitudinal inputs in forward flight to control speed (Acceleration Command, Speed
Hold (ACSH) response type);
8. Use of lateral inputs in forward flight to control heading (Attitude Command, Attitude Hold
(ACAH) response type);
9. Use of the collective lever in forward flight to control vertical flight path angle (flight path
angle command (C) response type);
10. Function of the pedals in forward flight (sideslip angle command (C) response type);
11. Combined use of lateral inputs and collective in forward flight to perform climbing and
descending turns;
12. Combined use of lateral and longitudinal inputs in forward flight to perform accelerative and
decelerative turns;
Project No. 266470 Deliverable D2.4
6
13. Combined use of longitudinal inputs and collective in forward flight to perform accelerative
and decelerative climbs and descents;
14. Combined use of longitudinal and lateral inputs and collective in forward flight to perform
accelerative or decelerative climbing or descending turns;
15. Longitudinal transition from TRC to ACSH;
16. Lateral transition from TRC to ACAH;
17. Collective transition from VRC to C;
18. Pedals transition from RC to C;
19. Longitudinal transition from ACSH to TRC;
20. Lateral transition from ACAH to TRC;
21. Collective transition from C to VRC;
22. Pedals transition from C to RC;
23. Use of secondary ‘automation’ functions (such as height hold, direction hold etc.) and
24. Use of instrumentation – including HUD symbology for guidance and navigation
It is acknowledged that additional knowledge and skills would be required in terms of cockpit
procedures, navigation, communications etc., although it is anticipated that training requirements
here would be minimised by effective cockpit design optimisation [14] and by the provision of
automatic functionality for route-planning etc. Due to the uncertainty related to these issues, the
study of their training requirements was considered to be beyond the scope of the current work.
Another important element of both driving and flight training is preparation for failures and other
emergency scenarios. Again, it might be anticipated that automatic systems, such as collision
detection and avoidance, would mitigate the need for some of this training. Training requirements
for these emergency scenarios will be studied as the myCopter project progresses.
3.2. Construction of PAV Training Programme
The 24 skills identified above were grouped into four ‘lessons’, each focussed on a specific part of the
PAV flight envelope. The lessons were set out as follows:
Lesson 1: Hover and Low Speed Flight – this lesson covers skills (1)-(6), and introduces the student
PAV pilot to all that is required to operate the vehicle at air speeds below 15kts.
Lesson 2: Cruising Flight – this lesson covers skills (7)-(14), and introduces all of the requirements for
flight at speeds greater than 25kts
Lesson 3: Transition – this lesson covers skills (15)-(22), covering the changes in response
characteristics between hover and low speed flight (< 15kts) and cruising flight (> 25kts)
Lesson 4: Advanced Functions – this lesson covers skills (23)-(24), which focus on the ‘automation’
functions of height and direction hold, and the visual symbology provided by a Head-Up Display for
attitude and flight-path and navigation using a Highway-in-the-Sky.
In addition to these 4 lessons covering the basic skills required to fly the PAV, a fifth lesson was
created that focussed specifically on the conduct of typical PAV manoeuvres – such as precision
hovering, vertical landings and descending approaches to hover [5]. These manoeuvres might be
considered as being the equivalent of the ‘reverse around a corner’ or ‘parallel parking’ manoeuvres
associated with driver training, or standard flying manoeuvres such as performing ‘circuits’ around
the airfield.
Project No. 266470 Deliverable D2.4
7
For each skill within a lesson, a series of exercises designed to introduce and subsequently refine the
skill were taught. For example, from the first lesson, for the skill of forward speed control, the
exercises were:
1) Use longitudinal stick input to set a desired forward speed
2) Accelerate/decelerate from one forward speed to another forward speed
3) Decelerate to hover
4) Control deceleration to hover at a specific point above the ground
A complete listing of the training exercises for all skills is included as Appendix A at the end of this
report.
For each exercise, a ‘briefing’ was conducted, introducing the purpose of the exercise and what
would be attempted. A demonstration was provided by the instructor (a member of the myCopter
project team who was very familiar with the characteristics of the simulation), with the required
control inputs and visual observations (i.e. the outside world features that the trainee should be
monitoring) highlighted. The student then attempted the exercise, and through repeated practice
with coaching from the instructor in terms of how to modify their technique to ensure safe and
precise control of the PAV, improved until a good, repeatable standard was attained (as with driver
and flying training, this was judged subjectively based on correct use of the controls and the trainee’s
apparent confidence in the control inputs being made along with the subsequent responses of the
vehicle). This was tracked using record sheets (see Appendix B) that allowed improvements in
competency to be followed and for the length of time spent on each skill to be recorded.
Progression to the next exercise was not permitted until at least ‘acceptable’ performance had been
achieved – in other words, the student was able to operate the vehicle safely (without large
overshoots of position, for example), repeatably and to a reasonable level of precision.
4. Results Five Test Subjects (TSs) undertook the PAV training syllabus. Their ages ranged from 22 to 45. Four
of the TSs were male, one female. All were car drivers, with driving experience levels that
corresponded to their age (the least experienced had been driving for 5 years, the most experienced
25 years). None of the TSs had any previous flying experience.
4.1. Training Duration
Figure 1 shows the total amount of time required by each TS to progress through the syllabus,
broken down into the individual lessons. It can be seen that four of the five TSs were able to
complete the syllabus in less than 300 minutes/5 hours. TS5, however, progressed at a much slower
pace, and failed to complete all 5 lessons in the time available. It is interesting to note that the
aptitude test taken prior to the start of the training identified this TS as being more likely to struggle
with the demands of the training than the other TSs (aptitude score of 0.56 for TS5, compared to
scores in the range 0.74-0.82 for the other TSs; higher scores indicating greater aptitude). TS5 also
reported that they had always required a lot of time and practice to become proficient with new
‘manual’ skills – for example, when learning to drive a car.
Project No. 266470 Deliverable D2.4
8
Figure 1: Training Time for Individual Test Subjects
It can be seen in Figure 1 that the individual lessons required different amounts of time. There was,
however, a good level of consistency between the TSs in terms of which lessons required more or
less time (the percentages on Figure 1 show the proportion of time spent by each TS on each lesson).
The lesson that demanded the greatest amount of time was Lesson 2 – covering control of the
aircraft in forward flight. Whilst the characteristics of the individual control axes could be learned
quite quickly, all of the TSs found that more time was required to reach the ‘acceptable’ standard
when simultaneous, coordinated multiple control inputs had to be made (skills 11-14). As with the
single-axis tasks, the process of physically moving the controls to start the PAV moving in the correct
sense was not demanding for the TSs. The main complexity introduced by the exercises for these
skills was the requirement to regularly monitor two or more of the controlled vehicle states (e.g.
airspeed, heading, altitude). The requirement to share attention across a number of information
sources required all of the TSs to spend time developing their instrument scan patterns, and to build
sufficient confidence in their knowledge of the vehicle’s responses. Prior to reaching this point in the
syllabus, the TSs had generally only been asked to apply control inputs in a single axis, allowing them
to focus on the way in which the controlled parameter was changing. For the multi-axis exercises in
Lesson 1, more readily available outside visual cues allowed the TSs to assimilate flight information
without the requirement for the comprehensive scan that was demanded in Lesson 2.
Lesson 3, in contrast, was straightforward for all of the participants. The subjects for this lesson –
transitioning between the low speed regime and the high speed regime, did not require the
demonstration of large amounts of skill or significant practice by the TSs. Rather, the key outcomes
from this lesson were the acquisition of theoretical knowledge and understanding by the TSs of the
expected behaviour of the aircraft during the transition stage. A short period of practice to reinforce
the theoretical knowledge was then all that was required to complete the objectives of this lesson.
4.2. Level 1 Evaluation – Participant Satisfaction
Each of the participants who completed all five lessons was asked to complete a questionnaire that
explored their satisfaction with the training that they had received. The questionnaire contained five
Project No. 266470 Deliverable D2.4
9
questions with quantitative answers, plus a number of ‘open’ questions for the participant to explain
the reasons for the answers that they had given. The five quantitative questions were:
1) To what extent do you feel that you have learned the skills necessary to fly a PAV from the programme?
2) Was the programme stimulating?
3) Was the pace of the programme appropriate for you?
4) Was the programme sufficiently flexible to meet your needs?
5) Was the programme challenging?
In each case, the participant was asked to respond on a scale from 1 to 8. A score of 8 indicated
strong agreement with the statement, while a score of 1 indicated strong disagreement. In the case
of question 3, a score of 8 indicated a pace that was too rapid, while a score of 1 indicated a pace
that was too slow.
Figure 2 shows the average score given by the participants for each question, together with the
upper and lower bounds of the ratings awarded. It can be seen that the participants found the
training programme to be effective at teaching them the skills they felt they needed (based on the
requirements of the final evaluations conducted following the training phase), was stimulating and
flexible. The participants found the pace of the training to be neither too fast nor too slow. The
participants generally found the training to be moderately challenging, indicating that the
characteristics of the PAV were relatively straightforward to learn, but that there remained sufficient
challenge to engage and stimulate the participants.
Figure 2: Participant Responses to Satisfaction Questionnaire
4.3. Level 2 Evaluation – Skills Test
Following completion of the training programme, each of the TSs who reached this stage took part in
a skills test. The test consisted of five Mission Task Elements (MTEs), used in earlier stages of the
myCopter research [5]. The MTEs are representative of various elements of the myCopter
commuting scenario. The five MTEs are as follows:
1) Hover – aircraft is accelerated to a speed of 6-10kts along a track aligned at 45° to its heading. The aircraft is then decelerated in a single, smooth action to hover at a prescribed point. The positioning accuracy with which the hover can be maintained is monitored. Height and heading are maintained constant throughout.
Project No. 266470 Deliverable D2.4
10
2) Vertical Reposition – the aircraft performs a hovering climb of 30ft while maintaining plan position and heading. A time limit of 10s is imposed on the climb.
3) Landing – the aircraft must perform a vertical touch down within a tightly constrained area. A 10s time limit is imposed on the final stages of the landing (height above ground < 10ft).
4) Decelerating Descent – the aircraft begins in cruising flight at a height of 500ft above the ground, at 60kts. When a marked position is reached, the aircraft descends and should begin to decelerate. The manoeuvre is complete when the aircraft has been brought to a hover at a height of 20ft above the marked end point.
5) Aborted Departure – the aircraft accelerates from hover to 40kts, and then decelerates back to hover. Height, heading and lateral track are held constant during this manoeuvre. A time limit of 25s is imposed on this task, making the level of aggression significantly higher than the other tasks.
For each task, a set of ‘desired’ performance boundaries have been identified (for the Hover for
example, in height (±2ft) and heading (±5°) deviation, and in plan position (±3ft either laterally or
longitudinally) during the steady hover phase of the task). These are identified to the pilots using
reference objects placed in the outside world visual scene. The TSs were asked to attempt to stay
within these boundaries whilst flying the MTEs.
Figure 3 shows the average time spent within the desired performance boundaries for each MTE
across the TSs who completed the skills test. Also shown for comparison is data from earlier
myCopter testing [5] in which the TSs were asked to attempt the MTEs without having had any
formal training. The TSs for this data were different to those being studied in this report, and had a
mixture of previous experience – from no flying or driving experience at all to holders of PPL(A)s and
PPL(H)s. It can be seen that those TSs who received training in the characteristics of the PAV
simulation were consistently able to achieve an excellent level of precision (>98% time spent in the
desired performance region) in all five MTEs. Although the ‘untrained’ TSs were able to achieve good
precision (confirming the highly intuitive nature of the response characteristics of the PAV
simulation), the precision achieved by the ‘trained’ TSs was better than the average precision
achieved by the ‘untrained’ TSs in every task (between 1% and 5% improvement in time spent within
the desired performance boundaries). This was particularly true in the Landing and Decelerating
Descent tasks. These two tasks, perhaps more so than the others, demand the application of
developed technique by the pilot, particularly in terms of use of the ‘advanced’ functions (such as the
use of a ‘hat’ switch to command small velocity perturbations for fine positioning in the Landing
MTE) and Head-Up Display symbology (flight path vector indicator and deceleration rate indicator to
judge the approach to hover in the Decelerating Descent MTE). The training received by the TSs has
clearly been beneficial in terms of allowing the target level of accuracy to be achieved.
Project No. 266470 Deliverable D2.4
11
Figure 3: Improvement in Task Precision Following Training
4.4. Level 3 Evaluation – Real-World Commute
To judge whether the participants in the training programme had developed the skills required to fly
the ‘real-world’ task of the commute, a simulation scenario was developed whereby the PAV pilot
would fly from the village of Kingsley Green to the south-east of Liverpool into the city centre. The
course that the participants were asked to follow is shown in Figure 4. It can be seen that this was
not a direct route – as Liverpool’s international airport is located directly between Kingsley Green
and the city. Hence, a deviation inland from the direct route was incorporated, with the PAV
avoiding the airport’s GA circuit patterns. The en-route planned altitude was 800ft. It was assumed
for the virtual scenario that all required airspace clearances were in place. The route follows the
River Mersey as Liverpool city centre is approached. This was to simulate noise abatement
procedures for the more densely populated regions being over flown. These deviations from the
direct path also provided an opportunity to incorporate manoeuvring elements into the evaluation,
rather than having a long, straight flight track. The total flight duration for this task was
approximately 11 minutes (compared to an equivalent road journey time of 30 – 90 minutes). The
visibility was good, and there was no wind or other atmospheric disturbance introduced to the
simulated environment. Similarly, no other air traffic of any kind was introduced into the scenario.
Project No. 266470 Deliverable D2.4
12
Figure 4: Route of Complete Commute (Map Data Copyright © Google)
At the start of the route, in Kingsley Green (Figure 5), the PAV begins on the ground in the centre of a
grassy area. A vertical take-off is performed, with the PAV climbing to a height of 75ft above the
ground so as to be clear of the surrounding buildings and trees. The PAV is then accelerated towards
the cruise whilst simultaneously climbing to the cruising altitude of 800ft and turning onto the course
for the first leg of the route. When the PAV nears the city centre, this process is reversed,
descending and decelerating, and eventually coming to a hover above an open area close to the city’s
financial centre. The PAV was then repositioned to a marked parking position, onto which a vertical
landing is performed.
4.5.
Figure 5: Start of Commute in Village Location (Map Data Copyright © Google)
The participants in this study used a Highway-in-the-Sky (HITS) [15;16] display to navigate along the
planned route (Figure 6). The HITS is attractive for PAVs due to its intuitive (i.e. visually
straightforward to determine appropriate control inputs to follow the correct route) and conformal
(i.e. is directly related to real terrain features) nature. The size of the boxes that form the HITS
informed the pilot as to the allowable discrepancy between planned and actual routing. It is
anticipated that PAVs would operate at considerably higher traffic densities than existing commercial
or private aviation. This leads to a requirement for precise positioning, and rigour in the
maintenance of position in order to avoid conflicts with other PAV traffic.
Project No. 266470 Deliverable D2.4
13
Figure 6: Highway-n-the-Sky used for PAV Navigation
The HITS also provided airspeed limit indications to the pilots. These was presented in the form of
UK-style road speed limit boards, albeit displaying limits as knots rather than miles per hour
(airspeed readouts for the PAV were also displayed in knots).
All of the TSs were able to fly the PAV along the HITS without incident, remaining well within the
boundaries throughout. Figure 7 shows a typical example of deviation measured from the centre of
the HITS boxes (which have dimensions of ±100ft). The larger spikes in deviation correspond to
points at which the PAV was turning onto the next leg of the route. Additionally, the pilots were
always able to adhere to the airspeed limits. This is illustrated in Figure 8; the airspeed limits were
sequentially 120kts, 80kts, 50kts and 30kts.
Figure 7: Lateral Deviation from Centre of HITS during Commute
Project No. 266470 Deliverable D2.4
14
Figure 8: Airspeed during Commute
Following completion of the commute scenario, each TS was asked to rate their workload using the
NASA Task Load Index (TLX) rating scale [17]. This system asks a participant to evaluate workload
using 6 factors – mental demand, physical demand, temporal demand, performance, effort and
frustration. Each factor is then weighted by its relative contribution to the overall workload to create
a single workload score between 0 and 100. A TLX of 0 indicates no workload at all, while a TLX of
100 indicates that the participant is at their maximum tolerable level in each area assessed.
The TSs returned an average TLX rating of 24 for the commute scenario, with a maximum of 30. They
commented that the workload in general was very low, giving plenty of time for observation,
monitoring etc. There were, however, occasions during the scenario where the workload increased.
These were generally the points at which the route required the pilot to perform two or three
actions simultaneously – i.e. airspeed change, heading change and/or altitude change.
5. Discussion The results presented above indicate that the training syllabus developed as part of this research was
an effective method by which to transfer the required knowledge and skills to the participants to
allow them to operate a PAV safely (i.e. within tolerances) and reliably (i.e. repeatedly). The
precision achieved in the manoeuvres used for the ‘skills test’ was improved in comparison to a
dataset for a group of ‘untrained’ test subjects. While in absolute terms the magnitude of the
improvement was not large, it should be noted that the ‘untrained’ subjects were already able to fly
the PAV to a high level of precision, demonstrating the intuitive nature of the PAV’s responses. In
this context, the improvement in achieved precision with the ‘trained’ subjects is useful. In none of
the MTEs did the trained subjects average less than 98% of time spent inside the task’s desired
performance boundaries.
The ‘trained’ test subjects were also able to complete the ‘real-world’ test – the commute scenario –
with a good degree of accuracy and with low workload. To contrast with the results reported here,
TLX ratings in the region of 55-60 have previously been reported for undistracted, qualified drivers
operating a car in a simulated urban environment [18]. TLX ratings are, however, a subjective
measure, meaning that it is not always possible to have complete read-across between different sets
of results. Nevertheless, these results provide an indication that the PAV is not more difficult to fly in
a typical role than a car is to drive. Given that all of the TSs were able to keep the PAV well within
Project No. 266470 Deliverable D2.4
15
the boundaries indicated by the HITS, the low workload is perhaps the more important of these two
metrics. Given the potential duration of a typical PAV flight (10-30 minutes), it would be
unacceptable for the workload to be continuously high, as this would lead to pilot fatigue.
Based on the subjective questionnaire completed by the TSs, all found the training to be engaging
and stimulating. This is an important consideration in training programme development, as without
trainee engagement in the process, learning typically occurs at a much slower rate [19]. Given that
one of the objectives of the myCopter project has been to determine the most effective methods by
which to reduce the costs associated with a PAV, a training programme that delivers high levels of
participant engagement is an obvious requirement.
The participants generally reported that they felt that they had received a comprehensive level of
training for the tasks that they were asked to carry out in the final evaluations. Two main items were
identified where the participants felt that additional training could have been delivered. The first of
these was simply further time to practice the various skills that were taught during the training.
Although all of the participants achieved a good level of performance in all of the exercises during
the course of the programme, further practice and experience will always be of benefit in terms of
developing a thorough understanding of exactly how the vehicle will respond to any given control
input. This is a phenomenon that can also be found in driving and (current) flight training – with the
expectation that newly-qualified drivers or pilots will need considerable time at the controls of their
vehicle before they have fully matured into their role.
The second area where the participants would have liked additional training was in the procedures
that would need to be followed in the case of something going wrong – either with the vehicle itself,
or with external factors (such as encroachment by other aircraft). As noted above, training for these
‘emergency’ situations was deliberately excluded from this phase of the research.
Finally, it was reported above that four of the five TSs in this study were able to complete the
training programme in less than five hours, while the fifth was slightly behind, having completed
three of the five lessons in just under five hours. Although, as discussed above, certain aspects of the
required training have been excluded from this study, and testing was exclusively simulation based
(which might remove the ‘startle’ and ‘fear’ related to real-world operations), these numbers
compare favourably with those typically expected for car driving (generally 20-40 hours) and flying
(45-100 hours). For a ‘real’ PAV training programme, it would be desirable to conduct at least some
of the training in simulation in order to minimise costs. The training would then progress to the
actual aircraft. The impact of this multi-stage approach on total training time would need to be
evaluated.
6. Concluding Remarks This report has described the creation and evaluation of a training syllabus for PAV pilots. The work
has assumed that the PAV is to be flown manually, and that it responds according to the best
characteristics identified during earlier work in the myCopter project. The following conclusions can
be drawn from this work:
A PAV training syllabus should cover the key skills associated with being able to establish and
hold airspeed, heading and height in low speed and cruising flight modes. It should also cover
the methods required to transition between the two modes.
Project No. 266470 Deliverable D2.4
16
The syllabus would also need to cover use of ancillary functions and display symbology.
A typical training duration of less than five hours was required in a simulation environment to
develop the skills necessary for PAV flight in benign environmental conditions.
Less able students require longer periods of training. One test subject – who typically struggles
to learn new manual skills – completed approximately 60% of the training in 4 hours 45 minutes.
Short periods of effective training can improve performance, even when the ‘operator’ is
controlling a highly intuitive system.
This work described in this report does not present a complete picture of the training that would be
required by a prospective PAV pilot. In particular, further training would be required for handling of
emergency situations, and any other aspects of conventional private aviation that would not be
eliminated by the incorporation of automatic or autonomous functions within the PAV. This would
need to be the subject of ongoing research.
7. References 1 Jump, M. et al, myCopter: Enabling Technologies for Personal Air Transport Systems, Royal
Aeronautical Society conference “The Future Rotorcraft – Enabling Capability through the
application of technology”, London, UK, June 2011
2 Achtelik, M.W. et al, Vision-Based MAV Navigation: Implementation Challenges Towards a Usable
System in Real-Life Scenarios, Workshop on Integration of Perception with Control and Navigation
for Resource-Limited Highly Dynamic Autonomous Systems, Robotics: Science and Systems, 2012
3 Sun X. , Christoudias C. M. , Lepetit V. and Fua P., Real-time landing place assessment in man-
made environments Machine Vision and Applications 25(1) pp. 211-227, 2014
4 Perfect, P., Jump, M. and White, M.D., Development of Handling Qualities Requirements for a
Personal Aerial Vehicle, Proceedings of the 38th European Rotorcraft Forum, Amsterdam,
Netherlands, September 2012
5 Perfect, P., Jump, M. and White, M.D., Towards Handling Qualities Requirements for Future
Personal Aerial Vehicles, Proceedings of the 69th Annual Forum of the American Helicopter
Society, Phoenix, AZ, USA, May 2013
6 Perfect, P., Jump, M. and White, M.D., Investigation of Personal Aerial Vehicle Handling Qualities
Requirements for Harsh Environmental Conditions, Proceedings of the 70th Annual Forum of the
American Helicopter Society, Montreal, Quebec, Canada, May 2014
7 Moore, M.L. and Dutton, P., Training Needs Analysis: Review and Critique, The Academy of
Management Review, Vol. 3, No. 3, July 1978, pp. 532-545
8 White, M.D., Perfect, P., Padfield, G.D., Gubbels, A.W. and Berryman, A.C., “Acceptance testing
and commissioning of a flight simulator for rotorcraft simulation fidelity research”, Proceedings of
the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering, Volume 227
Issue 4 April 2013 pp. 655 – 678, 2013
9 Kirkpatrick, D.L., “Techniques for Evaluating Training Programs”, Training and Development
Journal, June 1979, pp. 178-192
10 Charlton, S.G. and O’Brien, T.G. (Eds.) Handbook of Human Factors Testing and Evaluation, 2nd
Edition, Lawrence Erlbaum Associates, Mahwah, NJ, 2002
11 DSA Driving Standard, Safe and Responsible Driving (Category B), HMSO, March 2010
12 DSA Syllabus, Safe and Responsible Driving (Cat B), HMSO, March 2010
Project No. 266470 Deliverable D2.4
17
13 Notes for the Guidance of Applicants Taking the LAPL and PPL Skill Test (Aeroplanes), CAA
Standards Document 19, Version 7, CAA, September 2012
14 Olivari M. , Nieuwenhuizen F.M. , Bülthoff H.H. and Pollini L., An Experimental Comparison of
Haptic and Automated Pilot Support Systems, AIAA Modeling and Simulation Technologies
Conference, AIAA, 2014
15 Mulder, M., “An Information-Centred Analysis of the Tunnel-in-the-Sky Display, Part One: Straight
Tunnel Trajectories,” The International Journal of Aviation Psychology, Vol. 13, 2003, pp 49 – 72
16 Mulder, M., “An Information-Centred Analysis of the Tunnel-in-the-Sky Display, Part Two: Curved
Tunnel Trajectories,” The International Journal of Aviation Psychology, Vol. 13, 2003, pp 131 – 151
17 Hart, S.G. and Staveland, L.E., Development of NASA-TLX (Task Load Index): Results of empirical
and theoretical research, In P.A. Hancock and N. Meshkati (Eds.) Human Mental Workload.
Amsterdam: North Holland Press, 1988
18 Slick, R.F., Cady, E.T. and Tran, T.Q., Workload Changes in Teenaged Drivers Driving with
Distraction, Proceedings of the Third International Driving Symposium on Human Factors in Driver
Assessment, Training and Vehicle Design, Rockport, Maine, June 2005
19 Kuh, G.D. et al, “Unmasking the Effects of Student Engagement on First-Year College Grades and
Persistence”, The Journal of Higher Education, Vol. 79, No. 5, September 2008
Appendices
A. Training Exercises
This appendix lists each of the skills identified earlier in the report. For each skill, the exercises used
to develop that skill are listed.
1) Use of longitudinal inputs in hover to control forward speed (TRC response type)
a. Use longitudinal stick input to set a desired forward speed
b. Accelerate/decelerate from one forward speed to another forward speed
c. Decelerate to hover
d. Control deceleration to hover at a specific point above the ground
2) Use of lateral inputs in hover to control lateral speed (TRC response type)
a. Use lateral stick input to set a desired forward speed
b. Accelerate/decelerate from one lateral speed to another lateral speed
c. Decelerate to hover
d. Control deceleration to hover at a specific point above the ground
3) Combined use of longitudinal and lateral inputs to control horizontal flight path angle
a. Use of simultaneous longitudinal and lateral stick inputs to generate 45° trajectory
b. Use of longitudinal and lateral stick inputs to modify trajectory
c. Slalom using lateral stick inputs
d. Decelerate to hover
e. Control deceleration to hover at a specific point above the ground
4) Use of pedals in hover to control heading and yaw rate (Rate Command (RC) response type)
a. Use of pedal input to set desired yaw rate
b. Use of pedals to modify yaw rate
c. Decelerate yaw to stop at specific heading
d. Slalom using pedal inputs
Project No. 266470 Deliverable D2.4
18
5) Use of the collective lever in hover to control height and vertical rate (Vertical Rate
Command (VRC) response type)
a. Use of collective input to set desired vertical rate
b. Use of collective input to modify vertical rate
c. Decelerate to stop at specific height
6) Combined use of pedals and lateral inputs at low speed (<25kts) to improve turn
coordination
a. Demonstration exercise of effect of flight path lead/lag when using either pedals or lateral
stick individually
7) Use of longitudinal inputs in forward flight to control speed (Acceleration Command, Speed
Hold (ACSH) response type)
a. Use of longitudinal stick input to set acceleration/deceleration rate
b. Capture of new forward speed
8) Use of lateral inputs in forward flight to control heading (Attitude Command, Attitude Hold
(ACAH) response type)
a. Use of lateral stick input to set bank angle
b. Changing from one bank angle to another
c. Capture of a new heading
d. Capture of defined track over ground (e.g. along runway centreline)
e. Effect of speed on turning dynamics
9) Use of the collective lever in forward flight to control vertical flight path angle (flight path
a. Use of collective lever to set climb or descent angle
b. Capture of new height
c. Effect of speed on climbing dynamics
10)
a. Demonstration of sideslip angle response type
11) Combined use of lateral inputs and collective in forward flight to perform climbing and
descending turns
a. Commencing lateral and collective inputs simultaneously
b. Turning to new heading while climbing or descending to new height
c. Capture of defined ground track while climbing or descending to new height
d. Pacing turn and climb/descent to complete both simultaneously
12) Combined use of lateral and longitudinal inputs in forward flight to perform accelerative and
decelerative turns
a. Commencing lateral and longitudinal inputs simultaneously
b. Turning to new heading while accelerating or decelerating to new speed
c. Capture of defined ground track while accelerating or decelerating to new speed
d. Pacing turn and acceleration/deceleration to complete both simultaneously
13) Combined use of longitudinal inputs and collective in forward flight to perform accelerative
and decelerative climbs and descents
a. Commencing longitudinal and collective inputs simultaneously
b. Accelerating/decelerating to new speed while climbing/descending to new height
c. Pacing acceleration/deceleration and climb/descent to complete both simultaneously
14) Combined use of longitudinal and lateral inputs and collective in forward flight to perform
accelerative or decelerative climbing or descending turns
Project No. 266470 Deliverable D2.4
19
a. Commencing inputs on all three controls simultaneously
b. Turning, climbing/descending and accelerating/decelerating to new heading, height and
speed
c. Capture of defined ground track while climbing/descending and accelerating/decelerating
d. Pacing manoeuvres to complete all three simultaneously
15) Longitudinal transition from TRC to ACSH
a. Discuss theory of mode change
b. Accelerate from hover to forward flight – slowly
c. Accelerate from hover to forward flight - rapidly
16) Lateral transition from TRC to ACAH
a. Discuss theory of mode change
b. Demonstration of why lateral inputs during transition should be avoided where possible
17)
a. Discuss theory of mode change
b. Use collective control to perform height change while accelerating from hover to forward
flight
18)
a. Discuss theory of mode change
b. Demonstration of why pedal inputs during transition should be avoided where possible
19) Longitudinal transition from ACSH to TRC
a. Discuss theory of mode change
b. Decelerate from forward flight to hover
20) Lateral transition from ACAH to TRC
a. Discuss theory of mode change
b. Demonstration of why lateral inputs during transition should be avoided where possible
21)
a. Discuss theory of mode change
b. Use collective control to perform height change while decelerating from forward flight to
hover
c. Use collective control to track ground object while decelerating from forward flight to hover
22)
a. Discuss theory of mode change
b. Demonstration of why pedal inputs during transition should be avoided where possible
23) Use of secondary ‘automation’ functions (such as height hold, direction hold etc.)
a. Use of height hold function – when to use, how to engage
b. Use of direction hold function – when to use, how to engage
c. Use of speed beep function – when to use, how to operate
24) Use of instrumentation
a. General use of head down and head up symbology
b. Use of HUD flight path marker
c. Use of HUD deceleration rate indicator
d. Use of HUD highway-in-the-sky display
Project No. 266470 Deliverable D2.4
1
B. Progress Record Sheets
Figure B1: Training Session Record
Project No. 266470 Deliverable D2.4
2
Figure B2: Training Progress Record
Top Related