Project Management Training Case Study:

How PYRAMID is Bringing an End to Fare Woes  

What is PYRAMID?

“Project PYRAMID” is the name given to a Transport for London (TfL) initiative for project and programme management (PPM) training and professional development.  TfL are currently managing a £13bn improvement programme for public transport in the capital and 'PYRAMID' looks to support and enhance the capability of their entire PPM community, with appropriate interventions.
 
The design and integration of these interventions, across such a range of management tiers, represents a significant technical challenge as the nature of the PPM roles, like the individuals engaged, changes whilst passing through the various levels.
 
Full-career competence development for PPM professionals is complex.  PMProfessional Learning (PMPL) integrates the three aspects of competence, namely 'Technical', 'Behaviour' and 'Context', within their 'Stepping Stones' model.  The required blend of these varies in accordance with the seniority and career stage of the individual.  The model guides the design of the different training interventions needed to reflect this.

For TfL, the final portfolio is both varied and extensive, consisting of eleven different types of intervention.  PMPL have delivered seven of these, accounting for the vast majority of the thousand-plus delegate places already delivered, some 27% more than originally anticipated.

Measuring Benefit

“Was this project worthwhile?” may be the most important question for any project but alas, it is rarely the easiest to answer.  Ultimately the answer relies on comparison of cost and benefit and whereas cost is a relatively easy commodity to quantify, benefit is not.  This is especially so for projects delivering training since many of the benefits are subjective and abstract.  Because of this many of us simply abandon any concept of benefit measurement but the PYRAMID team was determined to meet this challenge head-on and underpin their work with objective assessment of the collective benefit.

Lara Taylorson, PYRAMID Business Manager at TfL explains.
"Ultimately the 'acid test' of any project’s success is whether the intended benefit was actually delivered.  Alas this is usually some years after handover of the project.  Many of our capital projects have lifecycle durations of up to 5-10 years and so basing our evaluation for PYRAMID on the final outcome of these projects was not realistic . To allow us to guide, control and evaluate PYRAMID we needed a method that measured effectiveness in the short and medium term."

In reality a number of measures were adopted.

Firstly, each delegate attending an intervention was obliged to complete a 'Feedback Form' detailing their impressions of the relevant material, presenter and accommodation.  This widely adopted technique is excellent for swiftly providing information and for PYRAMID the responses have been reassuringly positive.  Whilst this builds confidence, care must be taken; delegate’s perceptions of a course are not objective assessments of how well they have learnt.

Given the nature of the interventions there is an opportunity for a second, more objective measurement.  A large part of PMPL’s involvement in PYRAMID was the provision of certification courses (AMP Introductory Certificate and APMP).  The concepts of exam “pass” or “fail” provide for quantitative and objective assessment of retained knowledge but again, care needs to be taken.  Whereas they focus attention and stimulate delegates to perform, the result of the exam can detract from the amount of additional knowledge delegates secure and subsequently use, which is the essential purpose.  Some delegates acquire enormous additional capability even though the exam was not passed.  For the record, TfL’s results are good with pass rates above the national average, but as discussed, it offers only a limited perspective.  How then, to get a better picture of whether PYRAMID was delivering the benefit?  There was a third way.

Alison Maher, Head of Capability Development at TfL explains'

"We wanted quantifiable data relating to post-intervention changes in the behaviour of delegates, so we undertook an “Intermediate Evaluation Study”.  Approximately 12 months after the interventions we contacted the delegates and critically, also their line managers. This comprehensive and detailed assessment used questionnaires and interviews to compare current and pre-intervention behaviour.  As well as helping us to develop further the PYRAMID interventions, it offered an assessment as to whether we had made a change.  Were we delivering the promised benefit?"

The ability of the “Intermediate Evaluation Study” to answer these questions was further enhanced by careful planning on behalf of the PYRAMID team who having anticipated this scenario, had chosen to conduct an extensive analysis of the capability of the 1100 individuals in the PPM community at TfL before any intervention was delivered.  This established a clear baseline condition that allowed for a true measure of PYRAMID’s influence.

The Outcome

To the delight of everyone concerned all efforts at measuring the impact of PYRAMID were positive, as Alison recalls.

"Given the amount of time and effort we had put in, we were confident, but to see these results coming through indicating such strong improvements in capability across such a range of topics was fantastic.  Even more encouraging was the attitude of the delegates with so many of them expressing genuine gratitude and praise for something they felt had helped them both professionally and personally.  It was a great result for the whole team."

It has become obvious that others agree with this sentiment.  Delegates’ perceptions but also, critically, objective assessment have clearly demonstrated the true value of the project.  So much so that the services of PYRAMID are in great demand and it is being further extended into the Metronet organisation which has recently been embraced by TfL.

Whilst PYRAMID may not have the eventual staying power of its namesake, it does seem that PMPL and TfL are to be working together for many years to come.