Assurance: Helping the CEO to sleep at night

Assurance: Helping the CEO to sleep at night

Do you have confidence in project delivery?
Will the benefits you signed up to be delivered?
Is concern over these two issues becoming a distraction from your broader business?
These are typical questions faced by senior project and business professionals on a daily basis. The continual niggle of doubt undermines confidence, particularly when the consequences of project failure can be severe.

Lets face it, the evidence tends to support your concerns. The statistics on project failure are varied across industry sectors, but consistent in their message. Within the energy sector ~40% of projects under $500m fail, increasing to >60% failure rate for projects >$1bn[1]. 72% of transformation projects[2] fail to deliver the projected benefits outlined in the business case. Its not all bad, but 10 mins on google helps to underline that the concern is legitimate.

The bigger they are, the harder they fall. Furthermore, billion dollar projects tend to ‘create their own weather’ and when they fail, they tend to do so in a spectacular way.

[1] Industrial Megaprojects. Ed Merrow
[2] McKinsy & Company study “Beyond Performance” and Deloitte Global Survey 2008.
But what can I do?
Its not an easy question to answer. The CEO should have absolute trust in his team and that he is being told exactly what he needs to be told to ensure he has appropriate control of the project – afterall, the project may be fundamental to the ongoing viability of the business. However, in practice how often does this happen? Dashboards can provide insight but they lack depth in the areas of specific concern; the instrument panel may highlight that everything is operating within parameters but is it configured to pick up a fire in the exhaust? Vested interests influence reporting bias. Bad news may be deferred until it has been validated – or until there is an opportunity to manipulate the figures and spin up the reality. There are a range of other issues which help to cloud real project transparency, which I will explore further below.

In view of this, its typical for the CEO to secure a second opinion. This may be through line management, a team from within the organisation unconnected to the project or an independent third party. There are plenty of third parties to choose from who all profess to being able to provide project look through and get to the true heart of project. But in practice, they do differ tremendously.

Contracting independent assurance can help you to sleep at night but its not as simple as letting someone loose on the project and waiting for the report at the end of the process. It needs to be done intelligently otherwise it can become a mechanistic and ‘box ticking’ process. If you are paying for a service to give you a step change in confidence, then you have a vested interest in ensuring that you understand the barriers to conducting an effective review.
The levels of assurance

Many of the large government and private sector organisations have a nested approach to managing assurance

A team of professionals ensure that process is followed and outputs are accurately measured. They ensure that reporting is accurate and measured against the project baseline.

Ensuring that the baseline reflects reality, particularly when lags inherent in the process (earned value management, accruals, change management authorisation) can impede the approval of project changes.

Senior management are accountable for delivery and assure themselves at that appropriate management controls are in place, performance reporting is robust and supported by accurate data and to assure themselves of the probability of a successful business outcome. They provide challenge and support.

The effectiveness of this tends to be driven by:

Personalities. How much challenge do they bring when they are immersed in the project?
Vested interest. How embedded are senior staff in managing the impact of their decisions which subsequently may have turned sour?
Filtering. The tendency of senior management to soften the message to senior management until they have all the facts or until they have had an opportunity to ‘fix it’.

Ensuring that the baseline reflects reality, particularly when lags inherent in the process (earned value management, accruals, change management authorisation) can impede the approval of project changes.

Scrutiny from key senior functional staff within the organisation. They understand the organisational and process constraints and have first hand experience of the project, its history and key decisions. They provide functional support and deep levels of expertise. In addition they:
Review established governance processes
Review organisational effectiveness
Provide assurance of business cases
Oversee performance monitoring/reporting and assure data integrity.
Conduct focused, risk based reviews.
Provide challenge and support.

Vested interests, particularly when functional staff become immersed in project decision making. This is particularly prevalent when the project is in crisis with ‘all hands to the pump’. The team effectively becomes an extension of the project team and can lose their independence and a degree of objectivity.

If the team is under the control of the overall group Programme Director they can also influence their approach and objectivity. Everyone sits on a spectrum between doing the best for the project and managing their self interests: where does your Programme Director sit? You can’t necessarily influence it, but you can manage it.

The SRO or Project Sponsor would appoint an external organisation experienced in programme level assurance for major projects, with the prerequisite level of independence. Their role is to review and assure the effectiveness of the overall control framework. They would also conduct risk based reviews into areas of particular concern.

Ensuring the rigour in the review and particularly in the follow up actions. Is the review perceived as a hurdle to be overcome or as a value added exercise? Are actions completed to tick boxes or to secure tangible project improvement?
This sounds like an awful lot of overhead. Do I need it all?
This blog provides a summary of best practice but it is impossible to prescribe a one size fits all solution. The assurance methodology needs to be right for your organisation and to address the specific issues that keep you awake.
It is worth exploring some of the issues which may be lurking under the surface when you commission such as review:
Senior project staff interests.
“I am busy delivering my project. A team of people crawling all over their work is an unwelcome distraction”.
“They may find out where I have buried the bad news, which may have a significant impact on my personal circumstances. I need to get through this with minimal personal impact”.

This will impact the degree to which the senior project staff engage in the assurance process. To be effective they need to have a vested interest in supporting an effective review. To achieve this it needs to be supportive: a long list of burdensome actions is the last thing that the project need. They require:
Support. Helping them to work through the issues and identify solutions.
Confidence that someone is watching their back.
A last line of defence: picking things up that they otherwise struggle to identify.
When commissioning an assurance service its important to understand the culture and drivers in the project team. If the project is under stress to deliver a milestone the review is likely to be thinly supported and perceived as a hurdle to be overcome rather than an opportunity for improvement. Senior project staff may also have vested interests to protect their position and career, so act defensively under challenge.

The review must have project buy in from the outset, otherwise it may become mechanistic and superficial. Something to be completed so that the team can get back to their day job. This requires a collegiate approach; no blame, no scapegoats but an objective assessment of project performance.
Its very easy to apply a set of frameworks and assess project and assurance maturity. The skill is in using these to characterise project performance, drill down into underlying issues and utilise this output to secure buy in for improvement.

The next stage is to drill down into specifics. Its often easiest to target low hanging fruit such as the level of process adherence. Whilst this is important, it should only form a small component of the review. The key is to sample and resample across the project utilising a risk based approach; working in partnership with the SRO/Project Sponsor and Project Team to identify a prioritised list of review themes. This is where the experience of the review team comes into its own.

Its also important for the assurance team to undertake the role of the ‘10th man’. To think the unthinkable. To identify the low probability, high impact events that could have catastrophic consequences for the project and the business.
It can be useful to bring in someone brand new to the project or the industry; it helps to inject a fresh perspective and apply best practice. It also helps in identifying trends through the application of parametric techniques.

Care should be taken in creating a parallel project estimate, either schedule or cost. Although it helps to create another perspective, it can lead to endless discussions about which estimate is right. The answer is… none of the estimates are right because the project will evolve. The important point is to use the analysis to inform action to drive out estimate uncertainty and identify milestones when you anticipate error bars to narrow.
So how should I go about implementing an assurance regime?
It depends on your approach. Many organisations have level 0 & 1 assurance and if applied effectively this can provide a significant mitigation to project delay. But the challenge is in the effectiveness of implementation and the inherent culture within the project team and associated reporting lines. Projecting Success Ltd have extensive exposure to ad hoc assurance reviews and we believe that such reviews are inefficient and can often mislead, creating a false sense of security. They help to measure a project at a point in time but often fail to get under the skin of the underlying performance and projected out-turn. More importantly, they tend to end in a report rather than in facilitating improvement to reverse a trend.

An emerging approach is to develop an assurance regime that is independent from the business, but sufficiently close to be able to understand the business drivers and project context. Rather than being an auditor, the assurance team need to operate collegiately to provide the SRO/Project Sponsor and project team with insight into performance issues combined with supporting the implementation of solutions. It’s a fine line between maintaining independence and getting embroiled in the details of the project, “going native”, but it’s an approach that needs to be taken to secure the required level of improvement in project out-turn. Assurance as a service working in partnership with your business, rather than a short term ad-hoc pulling together a ‘band of brothers’ approach.
How much does it cost?
The key to success is tailoring. The assurance regime needs to integrate with your organisation and also recognise the nature of your portfolio. >$1bn megaprojects tend to be fragile and have a higher rate of failure; when they fail they fail big. Integrated portfolios of projects require tight alignment. Complex projects in a dynamic environment require agility. A one size fits all approach is the wrong approach.

In terms of cost, benchmarks provided by the National Audit Office suggest that programme assurance costs run to between 0.2 per cent and one per cent depending on the risk profile and complexity of the programme. At Projected Success Ltd we confident that we can add value and are happy to link our fee with assurance success.