For the last two years P3M3® (Portfolio, Programme and Project Management Maturity Model) has been mandated by the Federal and Queensland State Governments for assessing the organisational capability of agencies to effectively manage ICT-enabled investments. The results have been mixed, not just in terms of how mature agencies are, but also in terms of how the assessments have been conducted. Federal Government agencies will undergo another round of P3M3 assessments before the end of September with results to be reported to the Secretaries’ ICT Governance Board. There are a number of traps that agencies should be aware of when considering a self-assessment.
Trap 1 - Misunderstanding what 'self-assessment' really means
There was considerable confusion when the term ‘self-assessment’ was first introduced, not least because there were two distinct meanings being used. In the Federal Government context, self-assessment originally meant that assessments were to be run by agencies themselves and that it would not involve a central agency arranging each P3M3 assessment or ‘judging’ individual agency capability. The P3M3 model itself, however, offered self-assessment questionnaires as one of a number of ways for an organisation to begin to explore the P3M3 model.
Most agencies have now recognised that a P3M3 assessment is something they must own As such, the question of self-assessment then becomes “To what degree can we do this ourselves and to what degree do we need external assistance?”
Trap 2 - Not ensuring the right level of experience is involved
The P3M3 model is relatively simple to understand in structure; however a deeper level of understanding is required to grasp which elements of the model are the cornerstones of maturity measurement. There is a significant level of detail in the model and not all of it carries equal weight when undertaking an assessment.
In order to assess portfolio, programme and project management maturity, it is also important that the assessors understand what the disciplines of portfolio management, programme management and project management look like when they are well defined and applied in practice. Agencies must consider whether internal assessors have the skills and experience to make such a judgement. Many agencies have the requisite experience in relation to project management, but it is less common for portfolio management and programme management.
Trap 3 - Misinterpretation
The assessments undertaken to date show that consistent interpretation is a critical ingredient in a valid assessment. It is why assessments undertaken using online surveys are of questionable value – those responding to the survey must place their own interpretation on the meaning of the model without the benefit of an experienced assessor to guide them.
A simple example of this issue is interpretation of the word ‘programme’ – if people interpret it to mean ‘Government program’ rather than ’change programme’ when undertaking a survey then the P3M3 assessment results will be invalid.
Trap 4 - Using unskilled interviewers
The accepted method for undertaking a P3M3 assessment relies primarily on interviews with people in key management roles – programme managers, project sponsors, investment committee members etc. It is important that the correct people are selected for interview, and that the assessors are skilled interviewers. The APMG Group provides third-party accreditation of Registered Consultants who are qualified to undertake P3M3 assessments.
P3M3 is here to stay – What are the next steps?
Agencies are implementing their capability improvement plans. They will complete their next P3M3 assessment, to compare their actual capability to their target capability, and report the results to Secretaries’ ICT Governance Board, by 30 September 2012.
From September 2012, agencies are required to complete regular P3M3 assessments of their portfolio, program and project management capability and report to Secretaries’ ICT Governance Board on progress against their capability improvement plans.
Agencies must obtain independent validation of their current capability before undertaking major ICT projects. The Government considers agencies’ capability as part of the ICT Two Pass Review process to strengthen the link between policy formulation and implementation.
P3M3 is here to stay. While I hate to use the blog as a ‘sales tool’, it is important to note that Tanner James can help agencies undertake a self-assessment in the optimum way – using APS resources to build internal skills and reduce cost while benefiting from external expertise arising from having conducted many assessments. Conducting an assessment this way can produce a valid assessment for less than half the cost of having a full external assessment undertaken.
If you have undertaken a self-assessment before, or started to explore the self-assessment questionnaires, then I would welcome your questions, observations or lessons learned.