ASAE 3500

This ASAE deals with direct engagements undertaken by an assurance practitioner to provide a reasonable assurance report on an activity’s performance evaluated against identified criteria.  This ASAE may be applied to limited assurance engagements, adapted and supplemented as necessary in the engagement circumstances. (Ref: Para A1)

This ASAE addresses assurance engagements on performance:

1. of any activity, whether within an entity or across multiple entities;
2. evaluated against identified criteria selected or developed by the assurance practitioner or the engaging party;
3. providing a reasonable assurance conclusion; and
4. for either restricted use, by the engaging party or specified third parties, or to be publicly available, through tabling in Parliament or other means of distribution.

Agreed‑upon procedures engagements, where procedures are conducted and factual findings are reported but no conclusion is provided, and consulting engagements, for the purpose of providing advice on performance are not assurance engagements and are not dealt with in this ASAE.  Agreed‑upon procedures engagements are addressed under Standard on Related Services, ASRS 4400.^[1]

Nature of a Performance Engagement

The essential elements of performance engagements are: (Ref: Appendix 1 and Appendix 2)

1. a three party relationship involving:
   1. an assurance practitioner, including a State, Territory or Commonwealth Auditor-General;
   2. a responsible party or a number of responsible parties involved in the activities which are the subject matter of the performance engagement; and
   3. intended users, which are often the responsible party, Parliament and the general public;
2. an appropriate activity (the subject matter);
3. suitable criteria;
4. sufficient appropriate evidence; and
5. a written assurance report.

Performance engagements are most commonly conducted on an activity delivered or controlled by the Government.  The objective of a performance engagement is to evaluate the performance of an activity, with respect to economy, efficiency and/or effectiveness against the identified criteria.  The scope of a performance engagement is either determined by an Auditor‑General, based on the assessed information needs of Parliament, or of the general public, or by the engaging party based on the information needs of the engaging party and other identified users. (Ref: Para A2-A3)

Performance engagements are usually initiated by a State, Territory or the Commonwealth Auditor‑General and will not involve an engaging party, but may also be accepted by an assurance practitioner from an engaging party in the private sector.  The authority of an Auditor‑General to conduct a performance engagement derives from their legislative mandate, consequently the party responsible for the activity does not initiate the performance engagement and their agreement to the terms of engagement is not usually required.  The roles and responsibilities of the parties to a performance engagement initiated by an Auditor‑General are illustrated in Appendix 3. (Ref: Para A4)

Relationship with ASAE 3000, Other Pronouncements and Other Requirements

This ASAE adapts the requirements in ASAE 3000 Assurance Engagements Other than Audits or Reviews of Historical Financial Information, which is written for attestation engagements, as necessary, to direct engagements on performance and identifies the requirements of ASAE 3000 which the assurance practitioner is required to comply with in conducting a performance engagement in addition to the requirements of this ASAE.  The Framework for Assurance Engagements, which defines and describes the elements and objectives of an assurance engagement, provides the context for understanding this ASAE and ASAE 3000.

This ASAE requires the assurance practitioner to apply the ASAE 3000 requirement to comply with relevant ethical requirements related to assurance engagements.  It also requires the Audit Office of the Auditor‑General to apply ASQC 1^[2] or the lead assurance practitioner to be a member of a firm that applies ASQC 1.

An assurance engagement performed under this ASAE may be part of a larger engagement. If multiple standards are applicable to the assurance engagement, the assurance practitioner applies, either:

1. if the engagement can be separated into sections, the standard relevant to each section of the engagement, including this ASAE for the section on performance; or
2. if the engagement cannot be separated into sections, the standard which is most directly relevant to the subject matter.

In circumstances when an assurance engagement performed under this ASAE includes a compliance section, the assurance practitioner applies both ASAE 3100 Compliance Engagements and ASAE 3500 as applicable, in conducting the assurance engagement.

Assurance conclusions on performance may be required by Parliament, legislation, industry bodies or other users in conjunction with assurance conclusions on historical financial statements, other historical financial information, compliance, controls and/or other subject matters.  In these performance engagements the subject matter, identified criteria against which that subject matter is evaluated and the level of assurance sought may vary, in which case different standards will apply.  Assurance reports can include separate sections for each subject matter, identified criteria or level of assurance in order that the different matters to be concluded upon are clearly differentiated.

A table showing the AUASB Standards that apply to certain engagements, depending on the subject matter and engagement circumstances, is contained in Appendix 4.

The assurance practitioner shall not represent compliance with this ASAE unless the assurance practitioner has complied with the requirements of this ASAE and the requirements of ASAE 3000 identified in this ASAE as relevant to performance engagements, adapted as necessary for direct engagements.  ASAE 3000 contains requirements and application and other explanatory material specific to attestation assurance engagements but it may also be applied to direct engagements, adapted and supplemented as necessary in the engagement circumstances.^[3]

Inability to Comply with Relevant Requirements

Where in rare and exceptional circumstances, factors outside the assurance practitioner’s control prevent the assurance practitioner from complying with a relevant requirement in this ASAE, the assurance practitioner shall:

1. if possible, undertake appropriate alternative evidence-gathering procedures; and
2. document in the working papers:
   1. the circumstances surrounding the inability to comply;
   2. the reasons for the inability to comply; and
   3. justification of how alternative evidence-gathering procedures achieve the objectives of the relevant requirement.

When the assurance practitioner is unable to undertake appropriate alternative evidence‑gathering procedures, the assurance practitioner shall assess the implications for the assurance report.

As required by ASAE 3000, the assurance practitioner shall comply with relevant ethical requirements related to assurance engagements.^[4] (Ref: Para A5)

The assurance practitioner shall initiate, where the assurance practitioner has the legislative mandate to do so, or accept a performance engagement only when:

1. the assurance practitioner has no reason to believe that relevant ethical requirements, including independence, will not be satisfied;
2. the assurance practitioner is satisfied that those persons who are to perform the engagement collectively have the appropriate competence and capabilities;
3. the preconditions for an assurance engagement are present, as required by ASAE 3000;^[5] and
4. the basis on which the engagement is to be performed has been communicated and where relevant, agreed by the assurance practitioner and either:
   1. the engaging party, in written terms of engagement, including the assurance practitioner’s reporting responsibilities; or
   2. the responsible party, in an engagement initiated by the assurance practitioner where there is no engaging party, by issuing a written communication advising the responsible party of the planned engagement.

Preconditions for the Assurance Engagement

When establishing whether the preconditions for an assurance engagement are present, the assurance practitioner shall determine, based on their preliminary knowledge of the performance engagement circumstances, whether: (Ref: Para A6-A12)

1. the activities (underlying subject matter) which are to be evaluated are appropriate;
2. the criteria identified, selected or developed by the assurance practitioner or agreed with the engaging party are suitable in evaluating the activities, including that they exhibit the characteristics of suitable criteria,^[6] and will be available to users;
3. the assurance practitioner expects to be able to obtain the evidence needed to support the assurance practitioner’s conclusion, which will be contained in a written report; and
4. the engagement has a rational objective.

When identifying, selecting or developing suitable criteria or determining whether the identified criteria selected by the engaging party are suitable, the assurance practitioner shall consider whether the identified criteria are reasonable quantitative or qualitative measures of performance against which the activity’s performance may be assessed. Suitable criteria for a performance engagement shall reflect the overall objective/s, the assertions to be addressed (economy, efficiency and/or effectiveness) and have the following characteristics: (Ref: Para A13-A18)

1. relevance: relevant criteria contribute to conclusions that assist decision-making by the intended users.
2. completeness: criteria are sufficiently complete when relevant factors that could affect the conclusions in the context of the performance engagement circumstances are not omitted.
3. reliability: reliable criteria allow reasonably consistent evaluation or measurement of the activity, including when used in similar circumstances by similarly qualified assurance practitioners.
4. neutrality: neutral criteria contribute to conclusions that are free from bias.
5. understandability: understandable criteria contribute to conclusions that are clear, comprehensive, and not subject to significantly different interpretations.

Agreeing on or Communicating the Terms of the Performance Engagement

If the performance engagement is initiated by an engaging party, the assurance practitioner shall agree the terms of engagement with the engaging party in writing. (Ref: Para A19-A20)

If the performance engagement is initiated by a State, Territory or the Commonwealth Auditor‑General and does not involve an engaging party, then the assurance practitioner shall communicate the terms of engagement with the responsible party. (Ref: Para A21)

The assurance practitioner shall implement quality control procedures as required by ASAE 3000.^[7]

The assurance practitioner shall apply professional scepticism, exercise professional judgement and apply assurance skills and techniques in planning and performing a performance engagement.^[8]

Planning

The assurance practitioner shall plan the performance engagement so that it will be performed in an effective manner as required by ASAE 3000^[9] and achieves the objectives as communicated and/or agreed in the terms of engagement. (Ref: Para A2, A22-A25)

Materiality

The assurance practitioner shall consider materiality when determining the nature, timing and extent of procedures.

The assurance practitioner shall identify any matter relating to the activity as material if it is significant to the performance of the activity in relation to economy, efficiency and/or effectiveness evaluated against the identified criteria.  During the performance engagement the assurance practitioner shall reassess the materiality of any matter if there is any indication that the basis on which the materiality was determined has changed.

The assurance practitioner shall also consider materiality when evaluating the effect of any identified variations, taken individually and in combination, to the performance of the activity as evaluated against the identified criteria.  Material variations are those which could impact performance in relation to economy, efficiency and/or effectiveness and be reasonably expected to influence relevant decisions of the intended users of the assurance report. (Ref: Para A26-A34)

Understanding the Activity and Other Performance Engagement Circumstances

The assurance practitioner shall obtain an understanding of the activity, which is included in the scope of the performance engagement, and other engagement circumstances sufficient to enable the assurance practitioner to be able to identify and assess any risks of material variations in the activity’s performance in relation to economy, efficiency and/or effectiveness as evaluated against the identified criteria. (Ref: Para A35)

In doing so, the assurance practitioner shall obtain an understanding of internal controls the assurance practitioner considers are relevant to the evaluation of the activity’s performance against the identified criteria.  This includes evaluating the design of those controls pertinent to the objective of the performance engagement and, if relevant, determining whether they have been implemented by performing procedures in addition to inquiry of the responsible party. (Ref: Para A36-A39)

The assurance practitioner shall implement non‑compliance with laws and regulations procedures as required by ASAE 3000.^[10]

Based on the assurance practitioner’s understanding obtained in paragraphs 32, 33 and 34 the assurance practitioner shall: (Ref: Para A40-A42)

1. identify and assess the risks of material variation in the activity’s performance to be concluded upon;
2. consider the impact of assessed risks on the appropriateness of the performance engagement objective and the suitability of the identified criteria and, if necessary, seek to amend the objective and/or identified criteria;
3. design and perform assurance procedures to respond to assessed risks identified in paragraph 35(a); and
4. obtain sufficient appropriate evidence to support the assurance practitioner’s conclusion.

Work Performed by an Assurance Practitioner’s Expert

When the assurance practitioner plans to use the work of an assurance practitioner’s expert, the assurance practitioner shall comply with the requirements in ASAE 3000.^[11]

Work Performed by Another Assurance Practitioner, a Responsible Party’s or Evaluator’s Expert or an Internal Auditor

If the assurance practitioner plans to use information prepared by another party as evidence, the assurance practitioner shall comply with the requirements of ASAE 3000.^[12]

Written Representations

The assurance practitioner shall endeavour to obtain written representations, as appropriate for the performance engagement, from the responsible party or parties. (Ref: Para A43-A45)

(Ref: Para A46)

The assurance practitioner shall evaluate the impact of identified variations in the entity’s performance of the activity which are material, individually or in combination, on the assurance practitioner’s conclusion.  The assurance practitioner shall describe the extent and impact of those variations and conclude whether the activity partially performed or did not perform against the identified criteria in the assurance report.^[13] (Ref: Para A46)

When relevant to the performance engagement, the assurance practitioner shall consider the effect on the activity’s performance of events that become known to the assurance practitioner up to the date of the assurance report, and shall respond appropriately to facts that become known to the assurance practitioner after the date of the assurance report, that, had they been known to the assurance practitioner at that date, may have caused the assurance practitioner to amend the assurance report.  The extent of consideration of subsequent events depends on the assurance practitioners’ judgement of the potential for such events to affect the performance of the activity and to affect the appropriateness of the assurance practitioner’s conclusion.  However, the assurance practitioner has no responsibility to perform any procedures regarding performance of the activity after the date of the assurance report. (Ref: Para A47-A48)

The assurance practitioner shall evaluate the sufficiency and appropriateness of the evidence obtained in the context of the performance engagement, and if necessary, attempt to obtain further evidence. If the assurance practitioner is unable to obtain necessary further evidence, the assurance practitioner shall consider the implications for the assurance practitioner’s conclusion.  The assurance practitioner shall state in their conclusion that there was not sufficient or appropriate evidence to conclude whether the activity was free of material variation, in terms of economy, efficiency and/or effectiveness, as evaluated against the identified criteria.^[14]

The assurance practitioner shall form a conclusion about whether the performance of the activity as evaluated against the identified criteria is free of material variation.  In forming that conclusion, the assurance practitioner shall consider the outcomes of procedures performed in paragraphs 39, 40 and 41.

The assurance report shall be in writing and shall contain a clear expression of the assurance practitioner’s reasonable assurance conclusion about the activity’s performance against the objectives communicated and/or agreed in the terms of the performance engagement. (Ref: Para A49)

The assurance practitioner’s conclusion shall be clearly separated from other sections of the assurance report containing information or explanations that are not intended to affect the assurance practitioner’s conclusion, including findings and recommendations. (Ref: Para A52-A54)

Assurance Report Content

The assurance report shall include at a minimum the following base elements, to the extent that it is not inconsistent with relevant legislation or regulation: (Ref: Para A50-A51)

1. a title, indicating that it is an independent assurance report;
2. an addressee;
3. identification of the scope of the performance engagement including:
   1. the responsible party or (parties) and a description of their responsibilities;
   2. the activity which was the subject matter of the performance engagement;
   3. a description of the objective of the performance engagement;
   4. identification of the criteria for evaluating the performance of the activity and the party specifying those criteria, if it was not the assurance practitioner;
   5. if appropriate, a description of any significant inherent limitations associated with the evaluation of the activity’s performance against the identified criteria; and
   6. the assurance practitioner’s responsibilities.
4. a statement that the performance engagement was performed in accordance with ASAE 3500 Performance Engagements;
5. a statement that the assurance practitioner complies with the independence and other relevant ethical requirements related to assurance engagements;
6. a summary of the work performed by the assurance practitioner to obtain reasonable assurance and to provide a basis for the assurance practitioner’s conclusion;
7. the assurance practitioner’s conclusion about the performance, in terms of economy, efficiency and/or effectiveness, of the activity as evaluated against the identified criteria;
8. when the assurance practitioner has been unable to obtain sufficient appropriate evidence or has identified material variations in the activity’s performance in terms of economy, efficiency and/or effectiveness as evaluated against the identified criteria, the assurance report shall contain:
   1. a description of the extent and impact of those matter(s); and
   2. the assurance practitioner’s conclusion that either the activity did not perform in certain material respects, did not perform in all material respects, or there was not sufficient or appropriate evidence to conclude whether the activity was performed.
9. signature of the assurance practitioner, the Audit Office or location in the jurisdiction where the assurance practitioner practices, and the date of the assurance report.

If the assurance practitioner is required to conclude on other subject matters under different AUASB standards in conjunction with an engagement to report under this ASAE, the assurance report shall include a separate section for each subject matter in the assurance report, clearly differentiated by appropriate section headings.

Scope Limitation

A limitation on the scope of the assurance practitioner’s work may be imposed by the terms of the engagement, if the engagement was initiated by an engaging party, or by the circumstances of the particular engagement. When the limitation is imposed by the terms of the engagement, and it is likely to prevent the assurance practitioner from reaching a conclusion, the engagement shall not be accepted, unless required to do so by law or regulation.

When a scope limitation is imposed by the circumstances of the particular engagement, the assurance practitioner shall attempt to perform alternative procedures to overcome the limitation.  When a scope limitation exists and remains unresolved, the wording of the assurance practitioner’s conclusion shall describe the limitations on their engagement and the matters on which they are unable to conclude. (Ref: Para A54-A55)

If during the course of the performance engagement the assurance practitioner identifies any material variations in the activity’s performance, the assurance practitioner shall report those variations to the responsible party(ies) on a timely basis in order to allow the responsible party sufficient time to investigate and respond to the identified variations.

The assurance practitioner shall consider whether, pursuant to the terms of the performance engagement, if applicable, and other engagement circumstances or legislative requirements, any matter has come to the attention of the assurance practitioner that is to be communicated with Parliament, the responsible party, the engaging party (if applicable) or others, as required by ASAE 3000.^[15]

The assurance practitioner shall determine whether there is a responsibility or legislative requirement for the assurance practitioner to report the occurrence or suspicion of fraud or other misconduct to a party outside the entity, including Parliament, a regulator or government agency.  Any such reporting shall be in accordance with the relevant legislation.

The assurance practitioner shall prepare documentation in accordance with ASAE 3000.^[16] In documenting the nature, timing and extent of procedures performed as required by ASAE 3000, the assurance practitioner shall record: (Ref: Para A56-A58)

1. the identifying characteristics of the activity’s performance being tested;
2. who performed the work and the date such work was completed; and
3. who reviewed the work performed and the date.

(Ref: Para 3)

If the assurance practitioner initiates or accepts a limited assurance engagement on the performance of an activity, in adapting this ASAE for that purpose, the assurance practitioner ensures:

1. the users understand the lower level of assurance which the assurance practitioner will obtain as a basis for their conclusion;
2. the needs of users will still be met by a limited assurance conclusion; and
3. the assurance conclusion clearly communicates that the procedures performed vary in nature and timing from and are less in extent than for a reasonable assurance engagement and so the level of assurance obtained is substantially lower than in a reasonable assurance engagement.

(Ref: Para 7-8)

The objectives of a performance engagement may be expressed in various ways and are often presented as a statement of purpose or “questions” which are considered in the context of the responsible party’s responsibilities with respect to economy, efficiency and effectiveness.  In these circumstances, the assurance practitioner exercises professional judgement in determining the use of the most appropriate terminology throughout the performance engagement and especially in the assurance report. (Ref: Para 28)

Performance engagements may address a broad range of activities including:

1. systems for planning, budgeting, authorisation, control and evaluation of resource allocation;
2. systems established and maintained to ensure compliance with an entity’s mandate as expressed in policies or legislation;
3. resource management framework;
4. measures aimed at deriving economies of scale, such as centralised resource acquisition, sharing common resources across a number of business units;
5. measures aimed at improving economy, efficiency and/or effectiveness;
6. governance structures, including the assignment of responsibilities and accountability;
7. measures to monitor outcomes against predetermined objectives and performance benchmarks;
8. program or service delivery; and
9. implementation of government policy.

In the public sector, the conduct of performance engagements by Auditors-General is legislated in the respective jurisdictions. While the legislative requirements may have either a narrow or broad scope, performance engagements may include examination of:

1. economy, efficiency and/or effectiveness:
   1. in terms of management systems or an entity’s management in order to contribute to improvements;
   2. of the operations of an entity or an activity of an entity;
   3. in the implementation of government policies or programs and the application of government grants;
   4. in terms of financial prudence in the application of public resources; and
   5. of administrative arrangements.
2. intended and unintended impacts of the implementation of government policies or programs and the extent to which community needs and stated objectives of an activity or entity have been met; or
3. probity processes and identification of weaknesses.

(Ref: Para 20)

Relevant ethical requirements include the following fundamental principles with which the assurance practitioner is required to comply:

1. integrity;
2. objectivity, including independence;
3. professional competence and due care;
4. confidentiality; and
5. professional behaviour.

(Ref: Para 21-25)

Preconditions for the Assurance Engagement

In the public sector, if a performance engagement is initiated by the assurance practitioner, some of the preconditions for the assurance engagement may be assumed to be present if they are set out in legislation, such as the roles and responsibilities of the responsible party and the right of access to information by the assurance practitioner.

When initiating or accepting a performance engagement, in order to satisfy themselves that those persons who are to perform the performance engagement collectively have the appropriate competence and capabilities, the assurance practitioner may need to either assemble a multi‑disciplinary team or be a specialist in the relevant discipline.

When multi‑disciplinary teams are used in a performance engagement, adequate direction, supervision and review are particularly important so that the team members’ different perspectives, experience and specialties are appropriately used.  It is important that all team members understand the objectives of the particular performance engagement and the terms of reference of work assigned to them.  Adequate direction, supervision and review are important so that the work of all team members is executed properly and is in compliance with this ASAE and meets the quality control requirements of ASAE 3000.

Assessing the appropriateness of the subject matter

When assessing the appropriateness of the activity as the subject matter of the performance engagement, the assurance practitioner considers whether:

• the activity is identifiable, and its performance capable of consistent evaluation against identified criteria; and
• the information about it is capable of being subjected to procedures for gathering sufficient appropriate evidence to support a conclusion.

If after initiating or accepting the performance engagement, the assurance practitioner concludes that the activity is not an appropriate subject matter, the assurance practitioner assesses whether to:

• change the scope of the performance engagement or, if terms of the performance engagement have been agreed with the engaging party, seek to amend those terms; or
• withdraw from or discontinue the performance engagement.

In a performance engagement initiated by the assurance practitioner, the identification of the subject matter and development of criteria will be an iterative process which evolves as the audit objective/s are clarified and refined, based on the information gathered during the performance engagement.  As the assurance practitioner gains a better understanding of the performance engagement circumstances they may revise their assessment of the matters which address the needs of users.

In the event that the assurance practitioner is unable to change the scope or terms of, or withdraw from or discontinue, the performance engagement, under paragraph A10 of this ASAE, the assurance practitioner needs to consider the implications for the assurance report.

Assessing the Suitability of the Criteria

Criteria are the measures used to assess the performance of the activity. They may be based on relevant legislation, guidelines, internal policies and procedures, industry standards or best practice. Criteria which address each objective or sub-objective are developed or identified in planning the performance engagement. In assessing the suitability of the criteria, the assurance practitioner considers whether the criteria are derived from sources such as:

1. regulatory bodies, legislation or policy statements;
2. industry standards, relevant benchmarks, and relevant practice guides developed by professional bodies, associations or other recognised authorities;
3. statistics, measures or practices developed by the responsible party or by similar entities; or
4. those developed by the assurance practitioner themselves, in which case the assurance practitioner ordinarily documents why the identified criteria are suitable.

The assurance practitioner assesses the suitability of the criteria to evaluate or measure the performance of the activity, with respect to economy, efficiency and/or effectiveness to be addressed within the scope of the performance engagement.

Criteria may range from general to specific.  General criteria are broad statements of acceptable and reasonable performance.  Specific criteria, often referred to as sub‑criteria or lines of enquiry, are derived from general criteria and are more closely related to an entity's governing legislation or mandate, objectives, programs, systems and controls.

Criteria are either established or specifically developed. Ordinarily, established criteria are suitable when they are relevant to the needs of the intended users. Specific users may, however, develop a more detailed set of criteria that meet their specific needs in which case the assurance report may state, if it is relevant to the intended users:

• that the criteria are not embodied in laws or regulations, or issued by authorised or recognised bodies of experts that follow a transparent due process; and
• that the assurance report is only for the use of the intended users and for their purposes.

If after initiating or accepting the performance engagement, the assurance practitioner concludes that the identified criteria are not suitable, the assurance practitioner may either:

• identify or develop suitable criteria;
• seek to change the terms of the performance engagement, if necessary, such as when the terms have been agreed with an engaging party; or
• withdraw from or discontinue the performance engagement.

In the event that the assurance practitioner is unable to change the terms of, or withdraw from or discontinue, the performance engagement, the assurance practitioner considers the implications for the assurance report.

Agreeing on or Communicating the Terms of the Performance Engagement

The terms of the performance engagement normally identify:

1. the objectives of the engagement;
2. that the engagement is a reasonable assurance engagement;
3. the activity to be evaluated in the engagement;
4. the period to be covered by the engagement;
5. whether economy, efficiency and/or effectiveness is to be addressed and suitable criteria, in so far as the criteria have been identified, against which the activity will be evaluated;
6. the intended users of the assurance report;
7. the base elements of the assurance report; and
8. any other matters required by law or regulation to be included in the terms of engagement.

The terms of engagement may also seek the responsible party’s agreement that they acknowledge and understand their responsibility to provide the assurance practitioner with:

1. access to all information, such as records, documentation and other matters of which the responsible party is aware are relevant to the activity’s performance;
2. all additional information that the assurance practitioner may request from the responsible party for the purposes of the performance engagement; or
3. unrestricted access to persons engaged in the activity from whom the assurance practitioner determines it necessary to obtain evidence.

If there is no engaging party, such as for performance engagements initiated by an Auditor‑General, the existence of a legislative mandate may obviate the need to agree on the terms of the performance engagement.  Even in those circumstances it may be useful for the assurance practitioner to communicate the terms of engagement to the responsible party, including referral of any legislative requirements imposed on the responsible party to provide access to information or people relevant to the activity.

(Ref: Para 28-31)

Planning involves developing an overall strategy for the scope, emphasis, timing and conduct of the performance engagement. The performance engagement plan, consists of a detailed approach for the nature, timing and extent of evidence-gathering procedures to be undertaken and the reasons for selecting them. Ordinarily, adequate planning:

• helps to devote appropriate attention to important areas of the performance engagement, identify potential risk areas on a timely basis and properly organise and manage the performance engagement in order for it to be conducted in an effective and efficient manner;
• assists the assurance practitioner to properly assign work to performance engagement team members, and facilitates their direction and supervision and the review of their work; and
• assists, where applicable, the coordination of work done by other assurance practitioners and experts.

The nature and extent of planning activities will vary with the performance engagement circumstances, for example the size and complexity of the activity and the assurance practitioner’s previous experience with it. Examples of the main matters to be considered include:

• the terms of the performance engagement.
• the characteristics of the activity and the identified criteria.
• the performance engagement process and possible sources of evidence.
• the assurance practitioner’s understanding of the activity and other performance engagement circumstances.
• identification of intended users and their needs, and consideration of materiality and the assessment of risk.
• personnel and expertise requirements, including the nature and extent of involvement by experts.

Planning is not a discrete phase, but rather a continual and iterative process throughout the performance engagement.  As a result of unexpected events, changes in conditions, or the evidence obtained from the results of evidence‑gathering procedures, the assurance practitioner may need to revise the overall strategy and performance engagement plan, and as such the resulting planned nature, timing and extent of further evidence‑gathering procedures.

In planning the performance engagement, if the scope of the engagement is based on overall objectives, then the assurance practitioner may identify sub‑objectives from which they can identify, select or develop the criteria, against which the activity’s performance can be evaluated.

Materiality

Professional judgement about materiality is made in light of surrounding circumstances, but is not affected by the level of assurance.  Materiality for a reasonable assurance engagement is the same as for a limited assurance engagement because materiality is based on the information needs of intended users.

The identified criteria may discuss the concept of materiality in the context of the preparation and presentation of the assurance report and thereby provide a frame of reference for the assurance practitioner in considering materiality for the engagement.  Although identified criteria may discuss materiality in different terms, the concept of materiality generally includes the matters discussed in paragraphs A28–A34.  If the identified criteria do not include a discussion of the concept of materiality, these paragraphs provide the assurance practitioner with a frame of reference.

Variations in performance, including omissions, are considered to be material if they, individually or in combination, could reasonably be expected to influence relevant decisions of intended users taken on the basis of the assurance report. The assurance practitioner’s consideration of materiality is a matter of professional judgement, and is affected by the assurance practitioner’s perception of the common information needs of intended users as a group. In this context, it is reasonable for the assurance practitioner to assume that intended users:

1. have a reasonable knowledge of the activity, and a willingness to study the assurance report with reasonable diligence;
2. understand that the assurance report is prepared and assured to appropriate levels of materiality, and have an understanding of any materiality concepts included in the identified criteria;
3. understand any inherent uncertainties involved in the measuring or evaluating the activity; and
4. make reasonable decisions on the basis of the assurance report taken as a whole.

Unless the performance engagement has been designed to meet the particular information needs of specific users, the possible effect of variations in performance on specific users, whose information needs may vary widely, is not ordinarily considered.

Materiality is considered in the context of qualitative factors and, when applicable, quantitative factors.  The relative importance of qualitative factors and quantitative factors when considering materiality in a particular performance engagement is a matter for the assurance practitioner’s professional judgement.

Qualitative materiality factors may include such things as:

• the number of persons or entities affected by the subject matter.
• the interaction between, and relative importance of, various components of the activity when it is made up of multiple components, such as a report that includes numerous performance indicators.
• the wording chosen with respect to the activity that is expressed in narrative form.
• the characteristics of the presentation adopted for the assurance report when the identified criteria allow for variations in that presentation.
• the nature of a variation, for example, the nature of observed variations from a control when the assurance report includes a statement that the control is effective.
• whether a variation affects compliance with law or regulation.
• in the case of periodic reporting on an activity, the effect of an adjustment that affects past or current activities or is likely to affect future activities.
• whether a variation is the result of an intentional act or is unintentional.
• whether a variation is significant having regard to the assurance practitioner’s understanding of known previous communications to users, for example, in relation to the expected outcome of the measurement or evaluation of the underlying subject matter.
• whether a variation relates to the relationship between the responsible party, the measurer or evaluator, or the engaging party or their relationship with other parties.
• when a threshold or benchmark value has been identified, whether the result of the procedure deviates from that value.
• when the underlying subject matter is a governmental program or public sector entity, whether a particular aspect of the program or entity is significant with regard to the nature, visibility and sensitivity of the program or entity.

Quantitative materiality factors relate to the magnitude of variations relative to reported amounts for those aspects of the assurance report, if any, that are:

• expressed numerically; or
• otherwise related to numerical values (for example, the number of observed deviations from a control may be a relevant quantitative factor when the assurance report is a statement that the control is effective).

When quantitative factors are applicable, planning the performance engagement solely to detect individually material variations overlooks the fact that the combination of uncorrected and undetected individually immaterial variations may cause the assurance report to be materially misstated.  It may therefore be appropriate when planning the nature, timing and extent of procedures for the assurance practitioner to determine a quantity less than materiality as a basis for determining the nature, timing and extent of procedures.

Materiality relates to the information covered by the assurance report.  Therefore, when the performance engagement covers some, but not all, aspects of the information communicated about an underlying subject matter, materiality is considered in relation to only that portion that is covered by the performance engagement.

Concluding on the materiality of the variations identified as a result of the procedures performed requires professional judgement. For example:

• the identified criteria for a value for money engagement for a hospital’s emergency department may include the speed of the services provided, the quality of the services, the number of patients treated during a shift, and benchmarking the cost of the services against other similar hospitals. If three of these identified criteria are satisfied but one applicable criterion is not satisfied by a small margin, then professional judgement is needed to conclude whether the hospital’s emergency department represents value for money as a whole.

Understanding the Activity and Other Performance Engagement Circumstances (Ref: Para 32-33)

Obtaining an understanding of the activity and other performance engagement circumstances is an essential part of planning and conducting a performance engagement. That understanding provides the assurance practitioner with a frame of reference for exercising professional judgement throughout the performance engagement, for example, when:

• considering the characteristics of the activity.
• assessing the suitability of criteria.
• assessing systems established and maintained for ensuring compliance with an entity’s mandate or internal controls as expressed in policies and legislation.
• identifying where special consideration may be necessary, for example factors indicative of wastage or fraud, and the need for specialised skills or the work of an expert.
• establishing and evaluating the continued appropriateness of quantitative levels of performance (where appropriate), and considering qualitative materiality factors or benchmarks.
• developing expectations for use when undertaking analytical procedures.
• use of data analytical tools to undertake the engagement.
• designing and undertaking further evidence-gathering procedures to reduce risk to an appropriate level.
• evaluating evidence, including the reasonableness of the responsible party’s oral and written representations.

In a performance engagement, understanding internal controls relevant to the activity assists the practitioner in identifying the types of variations and factors that affect the risks of material variation.  Professional judgment is needed to determine which controls are relevant in the engagement circumstances.

When the objective of a performance engagement is to assess the design or implementation of controls over a process (for example, a process for dealing with patients in a hospital emergency room), the assurance practitioner may consider, during the initial planning phase,  identifying the internal controls to the extent necessary to inform the engagement scope and the risk assessment.  The assurance practitioner considers the evaluation of the design or determines the implementation of the controls later in the engagement as internal controls form the activity for this performance engagement.

When the objective of a performance engagement is to conclude on a specific outcome of a process, controls may not be relevant to that engagement.  For example, an assurance engagement may be designed to reach a conclusion regarding whether the time taken to process specific items (for example, applications to receive a service) over a specified period of time exceeds what is permitted under stated policies.  The practitioner might simply examine all the items processed during the specified period and conclude on whether there were material variations with the stated policies.

Evaluating the design of a control involves considering whether the control, individually or in combination with other controls, is capable of effectively preventing, or detecting and correcting, material variations.  Implementation of a control means that the control exists and that the entity is using it.  There is little point in assessing the implementation of a control that is not effective, and so the design of a control is considered first.  An improperly designed control may represent a significant deficiency in internal control.

(Ref: Para 35-38)

Sufficiency is the measure of the quantity of evidence.  Appropriateness is the measure of the quality of evidence; that is, its relevance and its reliability.  The assurance practitioner ordinarily considers the relationship between the cost of obtaining evidence and the usefulness of the information obtained.  However, the matter of difficulty or expense involved is not in itself a valid basis for omitting an evidence‑gathering procedure for which there is no alternative.  The assurance practitioner uses professional judgement and exercises professional scepticism in evaluating the quantity and quality of evidence, and thus its sufficiency and appropriateness, to support the conclusions in the assurance report.

Performance engagements require the application of assurance skills and techniques and the gathering of sufficient appropriate evidence as part of an iterative, systematic assurance engagement process.  For further guidance on the nature, timing and extent of evidence‑gathering procedures for performance engagements, refer to ASAE 3000.

In a performance engagement if the assurance practitioner becomes aware of a matter that leads the assurance practitioner to question whether sufficient appropriate evidence has been obtained, the assurance practitioner ordinarily pursues the matter by undertaking other evidence‑gathering procedures sufficient to enable the assurance practitioner to report.

Written Representations

If the performance engagement is initiated by the assurance practitioner, the assurance practitioner may not be in a position to obtain representations from the responsible party, particularly as the responsible party may not be a party to the performance engagement.

Representations by the responsible party cannot replace other evidence the assurance practitioner could reasonably expect to be available.  An inability to obtain sufficient appropriate evidence regarding a matter that has, or may have, a material effect on the evaluation or measurement of the activity, when such evidence would ordinarily be available, constitutes a limitation on the scope of the performance engagement, even if a representation from the responsible party has been received on the activity.

Written representations may include that the responsible party:

1. acknowledges its responsibility for conducting the activity, intended to achieve a certain level of performance;
2. has provided the assurance practitioner with all relevant information and access agreed to, as set out in paragraph A20;
3. has disclosed to the assurance practitioner any of the following of which it is aware may be relevant to the performance engagement:
   1. variations in achievement of intended performance; or
   2. any events subsequent to the period covered by the assurance practitioner’s report up to the date of the assurance report that could have a significant effect on the assurance practitioner’s report.

(Ref: Para 39)

The assurance practitioner needs to consider the impact of material variations in the performance of the activity when evaluated against the identified criteria, on the conclusions in the assurance report. A variation is material when, in the assurance practitioner’s judgement, it has the potential to affect:

1. decisions made by intended users about the performance (economy, efficiency and/or effectiveness) of an activity; or
2. the discharge of accountability by the responsible party or the governing party of the entity.

Further guidance on the qualitative and quantitative factors for the assurance practitioner to consider with regard to variations in performance of an activity refer to A30-A34.

(Ref: Para 40)

The extent of consideration of subsequent events that come to the attention of the assurance practitioner depends on the potential for such events to affect the activity and to affect the appropriateness of the assurance practitioner’s conclusions.  Consideration of subsequent events in some performance engagements may not be relevant because of the nature of the activity.

The assurance practitioner does not have any responsibility to perform procedures or make any enquiry after the date of the report.  If however, after the date of the report, the assurance practitioner becomes aware of a matter identified, the assurance practitioner may consider re‑issuing the report.  In a performance engagement the new report discusses the reason for the new report under a heading “Subsequent Events”.

(Ref: Para 43-48)

There may be circumstances where an Auditor‑General, having conducted a performance engagement, decides not to report to Parliament or to publish an assurance report.  The Auditor‑General usually has discretion under their mandate to choose whether and to whom they will report on performance engagements.  Assurance reports which are tabled in Parliament become available to the public.  In certain circumstances it may be necessary for the confidentiality of the assurance report to be maintained, in which case the report may, in accordance with relevant legislation be provided to the relevant Parliamentary Committee or other appropriate user, in confidence.  The Auditor‑General considers the public interest in determining whether the  assurance report will be made publicly available.

This ASAE does not require a standardised format for reporting on performance engagements even though paragraph 45 identifies the basic elements of the assurance report. For instance, under:

• paragraph 45(a), the title of the assurance report may differ depending on whether the assurance practitioner is an Auditor-General or a practitioner in the private sector. However, in both instances the title would convey that it is an independent report.
• paragraph 45(g), the assurance practitioner’s conclusions may be drafted as appropriate to recognise local legislation or custom and may be worded in terms of a response to the statement of purpose or the audit question.

Therefore, assurance reports are tailored to the specific performance engagement circumstances with the assurance practitioner using professional judgement in deciding how best to meet the reporting requirements detailed in paragraph 45 in conveying the conclusion(s). The assurance practitioner includes the matters in paragraph 45 as a minimum and reports in the manner and to the extent necessary to facilitate effective communication to the intended users. Whilst the assurance conclusion makes a clear statement communicating the assurance practitioner’s conclusion, the assurance report may include other matters which the assurance practitioner considers meet the information needs of the intended users, such as:

• terms of the performance engagement;
• overall objectives and sub-objectives of the performance engagement;
• identified criteria applied;
• findings relating to particular aspects of the performance engagement; and
• in some cases, recommendations.

Ordinarily, any findings and recommendations are clearly separated from the assurance practitioner’s conclusion on the performance of the activity.

Reporting Findings, Recommendations and Responsible Party Comments

The assurance practitioner may expand the assurance report to include other information and explanations, including:

• relevant background information and historical context.
• the assurance approach.
• underlying facts and identified criteria applied.
• disclosure of materiality levels.
• findings relating to particular aspects of the performance engagement.
• analysis of the causes of variations in the activity’s performance.
• recommendations to address variations identified.
• comments received in response to the report from the responsible party/ies.

The decision to include any such information depends on its significance to the needs of the intended users.  Additional information is clearly separated from the assurance practitioner’s conclusion and worded in such a manner so as not to affect that conclusion.

Variations in the Activity’s Performance

If material variations are identified, the assurance practitioner’s conclusion clearly reflects that either:

1. the activity did not perform, in terms of economy, efficiency and/or effectiveness, with respect to the identified criteria of the activity or certain objectives or sub-objectives of the performance engagement;
2. the activity did not perform, in terms of economy, efficiency and/or effectiveness with respect to the identified criteria of the activity or the objective of the performance engagement, as a whole; or
3. the assurance practitioner was unable to conclude on the activity’s performance when the assurance practitioner was unable to obtain sufficient appropriate evidence regarding the activity’s performance as a whole.

The assurance practitioner’s conclusions described in paragraph A54, are equivalent modified conclusions under ASAE 3000 and the equivalent terms in ASAE 3000^[17] are:

• a qualified conclusion – circumstances described in sub-paragraph A54(a).
• an adverse conclusion – circumstance described in sub-paragraph A54(b).
• a disclaimer of conclusion – circumstance described in sub-paragraph A54(c).

(Ref: Para 52)

Documentation includes a record of the assurance practitioner’s reasoning on all significant matters that require the exercise of professional judgement, and related conclusions.  The existence of difficult questions of principle or judgement, calls for the documentation to include the relevant facts that were known by the assurance practitioner at the time the conclusion was reached.

In applying professional judgement to assessing the extent of documentation to be prepared and retained, the assurance practitioner considers what is necessary to provide an understanding of the work undertaken, the results of that work, the evidence obtained and the basis of the principal decisions taken to another experienced assurance practitioner, who has no previous connection with the performance engagement.  It is, however, neither necessary nor practicable to document every matter the assurance practitioner considers during the performance engagement.

Identifying characteristics of the activity’s performance being tested that the assurance practitioner may document include:

1. subject matter; and
2. assertions being tested.