All-of-Government Portfolio, Programme and Project Assurance Framework

1. Introduction

1.1 New Zealand’s digital environment

We need to keep up with the current pace of digital evolution to maintain New Zealand’s standing as a digital leader.

Read about New Zealand’s place among the world’s digital elites and the benefits of government digital transformation:

1.2 Role of the System Assurance team

The role of the System Assurance team is to provide Ministers, the Government Chief Digital Officer (GCDO) and other key stakeholders with confidence that the system of assurance supporting digital government outcomes is effective.

Role of the System Assurance team

1.3 The system of assurance

Read about the system of assurance and the outcomes that deliver high levels of trust and confidence in digital public services:

The system of assurance

1.4 GCDO’s role in the investment system

The Treasury’s Investment Management System (IMS) team oversees New Zealand’s government investment system with a specific focus on improving the effectiveness of investment management and asset performance in the State sector supported by central agencies and functional leads, including the GCDO.

Working together, they operate as a multi-functional group, using their shared expertise to advise Ministers and government organisations to inform investment decisions.

The GCDO provides a system view of government’s investment in digital, data and technology. As part of the GCDO’s partnership with central agencies and other functional leads, they bring a digital perspective and expertise to key investment activities, including:

  • long-term investment planning
  • four-year planning
  • business case development
  • assurance planning and oversight.

The System Assurance team operates as part of the wider GCDO team that has a broad and integrated role in the investment system. The figure below illustrates a typical GCDO engagement during the investment lifecycle.

Figure 1: GCDO engagement across the investment lifecycle

An illustration of a typical GCDO engagement during the investment lifecycle

See larger version of image (PNG 225KB)

Detailed description of diagram

Process flow diagram from left to right desribed below:

  • Investment intentions:
    • Strategic advice on alignment to the Strategy for a Digital Public Service and outcomes (Strategy, Planning and Investment team).
    • Guidance and advice on digital government enterprise architecture (Government Enterprise Architecture team).
  • Investment planning:
    • Investment advice and support of business cases (Strategy, Planning and Investment team).
    • Advice and support on assurance planning, including agreeing an engagement plan with the Senior Responsible Owner (System Assurance team).
  • Investment delivery:
    • Quality review of assurance plans, terms of reference for independent assurance reviews and assurance reports (System Assurance team).
    • Manage the quality of assurance providers on the GCDO Assurance Services Panel (System Assurance team).
  • Investment realisation:
    • Engage with agencies and communities of practice to share lessons learned and system level themes (System Assurance team).

Learn more about engaging with central agencies and functional leads:

2. Value of Assurance

2.1 Assurance is integral to good governance

Whilst assurance itself does not deliver outcomes, effective risk management and assurance are critical components of good governance. Good governance helps to build trust and confidence in digital public services.

The governance body plays a key role in supporting the Senior Responsible Owner (SRO) to exercise their decision-making authority. An effective governance body provides oversight and challenge with a focus on key risks and issues. This includes making sure there is a robust assurance regime in place.

High-quality assurance information enables better conversations about risks to successful delivery and helps governance bodies to focus on actions that will ‘make the difference’.

Case study – Canadian Federal Government Transformation of Pay Administration Initiative (Phoenix)

The objective of this initiative was to transform the way in which the Canadian Government processed its pay for its 290,000 employees. An independent report completed 18 months after go-live by the Auditor-General of Canada found that:

  • there were over 494,500 outstanding pay requests
  • about half a billion dollars was owed to staff who had been paid incorrectly
  • 49,000 employees had been waiting for more than a year to have a pay request processed.

The audit found that ‘there was no real or independent oversight of the massive project; that executives did not understand the importance of the warnings they received; and that the decision to implement the system was unreasonable.’

2.2 Role of the Senior Responsible Owner (SRO)

The SRO has overall accountability for the success of the investment and is the key decision maker. Their role is to make sure the delivery team is focused on achieving its objectives and provide confidence to the Chief Executive that the investment will deliver the expected outcomes and benefits.

A key responsibility of the SRO is to make sure the assurance approach is fit-for-purpose. This means the SRO needs to engage with and approve the following artefacts:

  • assurance plan
  • terms of reference for independent assurance reviews
  • assurance reports.

Being engaged in assurance planning enables the SRO to insist on a tailored and insightful review. This means they can be confident in making an informed decision based on accurate information about risks and issues and their impact on outcomes.

Case study – NZ Police HRMIS Programme

In July 2014, New Zealand Police commenced the implementation of a new payroll and HR management information system.

“It would be difficult to overstate the value which independent assurance brought to this project. It was a vital tool for the project governance team and an important consideration for the Ministerial oversight.

Assurance not only had a key role in the delivery of the project but in addition greatly upskilled the Police team.

I would never consider undertaking such a project without comprehensive independent assurance.”

John Bole – Senior Responsible Officer

2.3 Improving delivery confidence

It’s easy to get caught up in the day-to-day activity of delivery. We are managing issues every day and have them under control. So why do we need assurance? The reality is that we often cannot see the ‘wood for the trees’ and underestimate the likelihood of risks impacting on us (optimism bias).

Assurance can help us step back from the day-to-day activity and identify potential ‘blind spots’ so that we have early warning and can rectify them before they start to impact on outcomes.

Assurance can help to reduce optimism bias by providing an objective and evidence-based view of the likelihood of key risks occurring and their potential impact on outcomes. By focusing on the areas of greatest risk and making sure actions are in place to manage them, delivery teams will be in a stronger position to provide delivery confidence to key stakeholders.

Case study – NZ Customs Joint Border Management System

The Joint Border Management System (JBMS) was put in place to deliver technology to modernise ageing computerised border systems and improve risk management and intelligence activities. The programme was a large multi-agency programme, to be delivered over multiple years in partnership with a multinational software vendor.

“Assurance needs to be planned and monitored for effectiveness from the outset. To get the most out of assurance it is important to target assurance activity to specific audiences and to have a clearly defined purpose of what it is setting out to achieve. Plan for early technical assurance where necessary and focus your assurance on forward-looking activities that assess risks to delivery of programme / organisational outcomes, rather than solely on compliance activities.”

Murray Young – Senior Responsible Owner

2.4 Value of an integrated assurance approach

Assurance is most effective when government organisations adopt an integrated approach to planning and coordinating assurance activities that considers a range of different stakeholder needs.

This is becoming increasingly important where there are a number of government organisations and other partners involved. Multiple stakeholders are likely to have different assurance needs, and an integrated approach to assurance planning can help to establish a single set of requirements that will meet all stakeholders’ needs.

An integrated approach, based on a shared view of risk across all stakeholders, helps to avoid assurance gaps and makes sure focus is on the key risks. It also helps to reduce the compliance burden on delivery teams and maximise value for money.

Case study – MSD Simplification Programme

Simplification was a programme of work designed to simplify the provision to clients of financial assistance and support by redesigning the experience for clients with a simpler service that made far greater use of digital channels and automating and streamlining transactional processing. The outcomes achieved were a better client experience, reduced cost and increased accuracy, efficacy and timeliness in processing client transactions.

“For Simplification the value proposition of assurance was to integrate ‘assurance by design’ in everything we did, across the layers of the programme through its delivery stages This meant we were able to identify early any divergence or risk of divergence from the path to our benefits realisation. We incorporated regular reviews and advice assurance activities from a variety of internal and external individuals and groups to help manage risk and improve delivery confidence.”

Liz Jones – Programme Director

3. Overview of framework

3.1 Purpose

The purpose of the All-of-Government (AoG) Portfolio, Programme and Project Assurance Framework is to support government organisations to implement a fit-for-purpose assurance approach for their digital investments.

Effective assurance provides confidence to your Chief Executive and other key stakeholders, including Ministers, that the expected investment outcomes and benefits will be achieved.

The framework is supported by detailed guidance and templates to help you apply the principles of good assurance.

Guidance and templates for Portfolio, Programme and Project Assurance

3.2 Audience

The target audience for the framework is:

  • Senior Responsible Owners (SROs) and governance bodies
  • project and programme managers
  • internal audit functions
  • enterprise portfolio or programme management offices (EPMOs).

3.3 Our definition of assurance

An independent and objective assessment that provides credible information to support decision-making.

The key words in our definition are ‘independent and objective’. There are varying degrees of independence and objectivity, but assurance is most effective when it is integrated across all ‘3 lines of defence’ in the 3 Lines of Defence model.

3.4 Three Lines of Defence model

  • The first line of defence is the day-to-day project management processes and controls you have in place, including quality management.
  • The second line of defence is the governance and oversight arrangements that exist, including clear and signed-off terms of reference for all governance bodies.
  • The third line of defence is the independent assurance you obtain from Internal Audit and third-party assurance providers.

The focus of our framework is on assurance that is performed by competent and impartial people outside of the delivery team (that is, at the second and third lines of defence). Examples of assurance activities within the scope of our framework include:

  • regular governance and oversight activities, such as governance meetings, executive project status reports, Audit and Risk Committee oversight
  • health checks performed by an internal EPMO
  • risk reviews performed by an internal Risk function
  • internal audit reviews
  • third-party assurance reviews, including Independent Quality Assurance (IQA) and Technical Quality Assurance (TQA) reviews
  • quantitative risk analysis
  • Gateway reviews.

3.5 Applicable government organisations

The framework is mandated for the following government organisations:

  • Public service departments
  • Non-public service departments
  • District health boards
  • Certain crown entities (ACC, EQC, NZQA, NZTA, HNZC, NZTE, TEC).

All central government organisations — State Services Commission (SSC)

Note: The framework is not limited to the above government organisations. It can be used by any government or private sector organisation as a guide to good assurance practice to support the successful delivery of investments to grow New Zealand’s economy and enhance the wellbeing of its people.

3.6 Applicable investments

The framework applies to all digital investments.

As a guiding principle:

An investment is defined as a digital investment if it uses technology as the primary lever for achieving the expected outcomes and benefits.

This is typically an investment that has a significant technology component. Note that for the purposes of the framework, digital investments include the following types of investments:

  • Investments that are looking to transform the way in which citizens interact with government — these investments will likely use technology to provide services in new and innovative ways
  • Investments that are looking to improve the efficiency and effectiveness of business operations — these investments will likely use technology in more traditional ways to automate tasks.

3.7 GCDO assurance oversight role

The GCDO has a core responsibility to provide Ministers and other key stakeholders with confidence that the system of assurance supporting digital government outcomes is effective. To enable the GCDO to fulfil this responsibility, the System Assurance team has an independent assurance oversight role over high-risk digital investments to make sure they:

  • have a fit-for-purpose assurance plan in place
  • obtain high-quality assurance information to support decision-making.

This requires the System Assurance team to work closely with SROs and the monitoring departments of high-risk digital investments to provide assurance planning advice and support.

Whether an investment falls into the high-risk category will be assessed as part of the Treasury’s Risk Profile Assessment (RPA) process. While the RPA process assesses the inherent risk to the system, we encourage government organisations to follow the principles of good assurance and use the supporting guidance and templates for all their digital investments, regardless of whether they are high risk or not.

Risk profile assessment — Treasury

3.8 Core expectations of government organisations

The following core expectations of government organisations apply to high-risk digital investments. However, the System Assurance team is happy to provide assurance planning advice and support to government organisations for non-high-risk digital investments.

All high-risk digital investments must:

  • attend an initial SRO briefing with the System Assurance team and agree an ongoing engagement plan
  • have an up-to-date and fit-for-purpose assurance plan in place that has been endorsed by the governance body and Internal Audit and approved by the SRO
  • submit the following artefacts to the System Assurance team for a quality review to make sure they are fit-for-purpose and meet the GCDO’s quality standards:
    • assurance plans
    • terms of reference for independent assurance reviews
    • assurance reports
  • use the GCDO Assurance Services Panel for third-party assurance reviews. Note: All applicable government organisations are required to use the GCDO Assurance Services Panel regardless of whether the digital investment is high risk or not.

4. Applying the principles of good assurance

The System Assurance team has developed a set of principles for good assurance practice based on our lessons learned. When applied, these principles support good practice assurance planning.

Figure 2: Principles of good assurance

Detailed description of diagram

This image shows the 6 principles of good assurance (in no particular order):

  • Assurance by design
  • Flexible
  • Informs key decisions
  • Independent and impartial
  • Risk and outcomes based
  • Accountability.

In moving to a principles-based framework, assurance becomes less about compliance and more about demonstrating good assurance thinking based on a clear understanding of delivery risk and the outcomes being sought.

The principles should be tailored to enable a fit-for-purpose approach based on the risk and complexity of the investment.

A principles-based approach provides confidence in the delivery of outcomes without resulting in excessive levels of assurance.

4.1 Assurance by design

Assurance is not a one-time activity. It’s the way we do things here…

  • Plan for assurance from the outset and continue to monitor and iterate throughout the investment lifecycle.
  • Make sure all business cases are supported by an assurance plan.
  • Budget for assurance activities in your business case.
  • Make sure assurance is integrated and operating effectively across all ‘3 lines of defence’.
  • Incorporate lessons learned from similar initiatives into your assurance approach.
  • Cleary define and coordinate roles and responsibilities across assurance providers to reduce the compliance burden on delivery teams.
  • Undertake risk assessments when designing new systems, processes and policy, including for core delivery partner activities.

4.2 Flexible

Assurance is adaptable to meet changes in scope, approach, solution or risk profile.

  • Significant changes to the scope, approach, solution, or risk profile of the initiative trigger a review of the assurance plan by the governance body.
  • Tailor assurance to the delivery approach – for example, in an Agile / DevOps environment there may be greater reliance on assurance activities embedded into day-to-day project delivery and governance activities.
  • Use the results of assurance activities to inform the forward assurance plan.
  • Make sure assurance covers inter-agency, sector and AoG impacts, including stakeholder engagement activities, where an investment goes beyond the boundaries of your organisation.
  • The governance body regularly reviews the assurance plan to make sure it continues to be fit-for-purpose and that the agreed assurance activities are undertaken.

4.3 Informs key decisions

Assurance provides timely, credible information to inform key decisions.

  • Make sure there is a clear relationship between the planned assurance activities and key decision points:
    • critical milestones / off-ramps
    • contract stage gates
    • key dependencies.
  • Make sure assurance reports are unambiguous and support informed decision-making based on an assessment of delivery confidence.
  • Assess ongoing viability and alignment to strategic outcomes before moving to the next phase.
  • Technical quality assurance is vital in assessing progress and quality and should be planned for as early as possible in the investment lifecycle.
  • Make sure assurance reviews at project handover include both business readiness and ICT operational readiness to accept the change.

4.4 Risk and outcomes-based

Assurance assesses the risks to successful delivery and their impact on outcomes.

  • Make sure assurance is risk-based – in other words, there is a clear link between the planned assurance activities and the risks to achieving the investment outcomes.
  • Make sure assurance is forward-looking and assesses delivery confidence rather than focusing solely on adherence to methodology.
  • Undertake due diligence on vendors to identify risks to delivery such as capacity, capability, overreliance on key people, location of vendor (offshore, onshore), and so on.
  • Use a phased delivery approach with clear and agreed off-ramps and acceptance criteria that measure real progress against outcomes.
  • The governance body regularly reviews risks to make sure they are being managed in accordance with the organisation’s risk tolerance level.

4.5 Independent and impartial

Assurance is performed by competent people outside of the delivery team who are not unduly influenced by key stakeholders.

  • Identify key members of the review team in the terms of reference and make sure they have the right skills and experience to effectively assure an investment of your scale and complexity.
  • Follow formal procurement processes to engage third-party assurance providers. Note: All applicable government organisations are required to use the GCDO Assurance Services Panel regardless of whether the digital investment is high risk or not.
  • Make sure any conflicts of interest are clearly identified and managed, including:
    • making sure personal relationships between government organisations and providers don’t threaten independence and objectivity
    • performing an assurance review where the provider has or is currently providing project management or technical services
    • fixing issues identified during the course of an assurance review.

4.6 Accountability

Assurance roles and responsibilities at the governance level are understood.

  • Clearly document assurance roles and responsibilities in role descriptions and the governance body terms of reference.
  • Regularly review the composition of the governance body to make sure it has the right skills and experience.
  • Make sure assurance artefacts — for example, assurance plans, terms of reference for independent assurance reviews and assurance reports are endorsed by the governance body and approved by the SRO.
  • Make sure the SRO includes a management response to accept the findings in the assurance report or to record if there is a disagreement over a finding or recommendation.
  • Make sure the governing body receives copies of all assurance reports in full.
  • Track and regularly report the status of issues raised in assurance reports to the governance body.

5. Engaging with the System Assurance team

The nature and frequency of engagement with the System Assurance team is based on a number of factors. We consider:

  • the complexity and risk of the investment
  • the organisation’s previous experience with digital investments
  • the level of oversight by Internal Audit
  • our previous experience of similar initiatives.

The level of engagement will be tailored to suit the needs of the investment. For example, an increase in the risk to delivery may require an increase in System Assurance oversight.

5.1 Triggers for engagement

The most common trigger for GCDO engagement is the Risk Profile Assessment (RPA). Government organisations must complete an RPA for all significant investments identified in multi-year plans and provide any RPA that has a Medium or High-risk rating to the Treasury IMS team.

The IMS team engages with the GCDO, central agencies and other functional leads to agree the final risk rating. If you are unsure whether your investment meets this criterion, you should complete an RPA.

As part of our System Lead response on your RPA, the GCDO will provide a means for your agency to contact us for an initial engagement.

Risk profile assessment — Treasury

5.2 Initial SRO briefing

As the key decision maker, it’s vital the SRO has a clear understanding of the value of assurance and the GCDO’s core expectations for good assurance practice.

To support the SRO to fulfil their core assurance accountabilities, the System Assurance team will run an initial briefing to make sure the SRO has a clear understanding of their key responsibilities and how to apply the principles of good assurance.

As an output of the briefing, we will agree an engagement plan with the SRO.

As part of this engagement plan, we will agree if assurance documents e.g., assurance plans, terms of reference and assurance reports should be submitted before/after internal SRO approval. We typically require 10 working days with these documents and recommend the relevant teams contact us during the draft phase.

5.3 GCDO Assurance Services Panel

The GCDO Assurance Services Panel (GCDO Panel) makes it easy for government organisations to access highly qualified providers of independent assurance services — including Independent Quality Assurance (IQA) and Technical Quality Assurance (TQA) — for digital investments.

GCDO Assurance Services Panel

5.4 How to contact us

Contract the System Assurance team for advice on digital investment assurance.

Email: systemassurance@dia.govt.nz.

Guidance and templates for Portfolio, Programme and Project Assurance

For a full list of guidance and templates, see:

Guidance and templates for Portfolio, Programme and Project Assurance

Glossary

This table lists definitions of terms used in this guidance.
Term Definition
Central agencies New Zealand’s central agencies are the Department of the Prime Minister and Cabinet (DPMC), the State Services Commission (SSC), and the Treasury.
Functional lead Functional leads are assigned by the State Services Leadership Team to chief executives to drive performance across the state services in functional areas such as policy, finance, data and analytics, communications, procurement, digital, property, human resources, health and safety, legal, and investment management and asset performance.
Governance body A governance body is a group of people with the authority to challenge and exercise oversight over the portfolio, programme or project.
Independent Quality Assurance (IQA) and Technical Quality Assurance (TQA)

GCDO assurance services for digital investments fall into two broad categories:

  • Independent Quality Assurance (IQA) – Portfolio assurance, Programme assurance and Project assurance
  • Technical Quality Assurance (TQA) – Technical design assurance and Technical implementation assurance.
Off-ramp An off-ramp is an exit clause in a contract.
Senior Responsible Owner (SRO)

The SRO role can also be described as a sponsor, executive or executive sponsor. It describes the role with overall accountability for the success of the investment and as the chair of the governance body, the key decision maker.

Within this framework, the SRO or equivalent has accountability for ensuring that the assurance approach is fit-for-purpose.
Three Lines of Defence model The 3 Lines of Defence model is used as a clear and effective way to strengthen communications on risk management, assurance, and control by clarifying essential roles and duties for various parts of governance, management, and day-to-day operations.
Was this page helpful?
Thanks, do you want to tell us more?

Do not enter personal information. All fields are optional.

Last updated