(Ref: Para 3)

If the assurance practitioner initiates or accepts a limited assurance engagement on the performance of an activity, in adapting this ASAE for that purpose, the assurance practitioner ensures:

1. the users understand the lower level of assurance which the assurance practitioner will obtain as a basis for their conclusion;
2. the needs of users will still be met by a limited assurance conclusion; and
3. the assurance conclusion clearly communicates that the procedures performed vary in nature and timing from and are less in extent than for a reasonable assurance engagement and so the level of assurance obtained is substantially lower than in a reasonable assurance engagement.

(Ref: Para 7-8)

The objectives of a performance engagement may be expressed in various ways and are often presented as a statement of purpose or “questions” which are considered in the context of the responsible party’s responsibilities with respect to economy, efficiency and effectiveness.  In these circumstances, the assurance practitioner exercises professional judgement in determining the use of the most appropriate terminology throughout the performance engagement and especially in the assurance report. (Ref: Para 28)

Performance engagements may address a broad range of activities including:

1. systems for planning, budgeting, authorisation, control and evaluation of resource allocation;
2. systems established and maintained to ensure compliance with an entity’s mandate as expressed in policies or legislation;
3. resource management framework;
4. measures aimed at deriving economies of scale, such as centralised resource acquisition, sharing common resources across a number of business units;
5. measures aimed at improving economy, efficiency and/or effectiveness;
6. governance structures, including the assignment of responsibilities and accountability;
7. measures to monitor outcomes against predetermined objectives and performance benchmarks;
8. program or service delivery; and
9. implementation of government policy.

In the public sector, the conduct of performance engagements by Auditors-General is legislated in the respective jurisdictions. While the legislative requirements may have either a narrow or broad scope, performance engagements may include examination of:

1. economy, efficiency and/or effectiveness:
   1. in terms of management systems or an entity’s management in order to contribute to improvements;
   2. of the operations of an entity or an activity of an entity;
   3. in the implementation of government policies or programs and the application of government grants;
   4. in terms of financial prudence in the application of public resources; and
   5. of administrative arrangements.
2. intended and unintended impacts of the implementation of government policies or programs and the extent to which community needs and stated objectives of an activity or entity have been met; or
3. probity processes and identification of weaknesses.

(Ref: Para 20)

Relevant ethical requirements include the following fundamental principles with which the assurance practitioner is required to comply:

1. integrity;
2. objectivity, including independence;
3. professional competence and due care;
4. confidentiality; and
5. professional behaviour.

(Ref: Para 21-25)

Preconditions for the Assurance Engagement

In the public sector, if a performance engagement is initiated by the assurance practitioner, some of the preconditions for the assurance engagement may be assumed to be present if they are set out in legislation, such as the roles and responsibilities of the responsible party and the right of access to information by the assurance practitioner.

When initiating or accepting a performance engagement, in order to satisfy themselves that those persons who are to perform the performance engagement collectively have the appropriate competence and capabilities, including having sufficient time to perform the engagement, the assurance practitioner may need to either assemble a multi‑disciplinary team or be a specialist in the relevant discipline.

When multi‑disciplinary teams are used in a performance engagement, adequate direction and supervision of engagement teams and review of their work are particularly important so that the engagement team members’ different perspectives, experience and specialties are appropriately used.  It is important that all engagement team members understand the objectives of the particular performance engagement and the terms of reference of work assigned to them.  Adequate direction and supervision of engagement teams and review of their work are important so that the work of all engagement team members is executed properly and is in compliance with this ASAE and meets the quality management requirements of ASAE 3000.

Assessing the appropriateness of the subject matter

When assessing the appropriateness of the activity as the subject matter of the performance engagement, the assurance practitioner considers whether:

• the activity is identifiable, and its performance capable of consistent evaluation against identified criteria; and
• the information about it is capable of being subjected to procedures for gathering sufficient appropriate evidence to support a conclusion.

If after initiating or accepting the performance engagement, the assurance practitioner concludes that the activity is not an appropriate subject matter, the assurance practitioner assesses whether to:

• change the scope of the performance engagement or, if terms of the performance engagement have been agreed with the engaging party, seek to amend those terms; or
• withdraw from or discontinue the performance engagement.

In a performance engagement initiated by the assurance practitioner, the identification of the subject matter and development of criteria will be an iterative process which evolves as the audit objective/s are clarified and refined, based on the information gathered during the performance engagement.  As the assurance practitioner gains a better understanding of the performance engagement circumstances they may revise their assessment of the matters which address the needs of users.

In the event that the assurance practitioner is unable to change the scope or terms of, or withdraw from or discontinue, the performance engagement, under paragraph A10 of this ASAE, the assurance practitioner needs to consider the implications for the assurance report.

Assessing the Suitability of the Criteria

Criteria are the measures used to assess the performance of the activity. They may be based on relevant legislation, guidelines, internal policies and procedures, industry standards or best practice. Criteria which address each objective or sub-objective are developed or identified in planning the performance engagement. In assessing the suitability of the criteria, the assurance practitioner considers whether the criteria are derived from sources such as:

1. regulatory bodies, legislation or policy statements;
2. industry standards, relevant benchmarks, and relevant practice guides developed by professional bodies, associations or other recognised authorities;
3. statistics, measures or practices developed by the responsible party or by similar entities; or
4. those developed by the assurance practitioner themselves, in which case the assurance practitioner ordinarily documents why the identified criteria are suitable.

The assurance practitioner assesses the suitability of the criteria to evaluate or measure the performance of the activity, with respect to economy, efficiency and/or effectiveness to be addressed within the scope of the performance engagement.

Criteria may range from general to specific.  General criteria are broad statements of acceptable and reasonable performance.  Specific criteria, often referred to as sub‑criteria or lines of enquiry, are derived from general criteria and are more closely related to an entity's governing legislation or mandate, objectives, programs, systems and controls.

Criteria are either established or specifically developed. Ordinarily, established criteria are suitable when they are relevant to the needs of the intended users. Specific users may, however, develop a more detailed set of criteria that meet their specific needs in which case the assurance report may state, if it is relevant to the intended users:

• that the criteria are not embodied in laws or regulations, or issued by authorised or recognised bodies of experts that follow a transparent due process; and
• that the assurance report is only for the use of the intended users and for their purposes.

If after initiating or accepting the performance engagement, the assurance practitioner concludes that the identified criteria are not suitable, the assurance practitioner may either:

• identify or develop suitable criteria;
• seek to change the terms of the performance engagement, if necessary, such as when the terms have been agreed with an engaging party; or
• withdraw from or discontinue the performance engagement.

In the event that the assurance practitioner is unable to change the terms of, or withdraw from or discontinue, the performance engagement, the assurance practitioner considers the implications for the assurance report.

Agreeing on or Communicating the Terms of the Performance Engagement

The terms of the performance engagement normally identify:

1. the objectives of the engagement;
2. that the engagement is a reasonable assurance engagement;
3. the activity to be evaluated in the engagement;
4. the period to be covered by the engagement;
5. whether economy, efficiency and/or effectiveness is to be addressed and suitable criteria, in so far as the criteria have been identified, against which the activity will be evaluated;
6. the intended users of the assurance report;
7. the base elements of the assurance report; and
8. any other matters required by law or regulation to be included in the terms of engagement.

The terms of engagement may also seek the responsible party’s agreement that they acknowledge and understand their responsibility to provide the assurance practitioner with:

1. access to all information, such as records, documentation and other matters of which the responsible party is aware are relevant to the activity’s performance;
2. all additional information that the assurance practitioner may request from the responsible party for the purposes of the performance engagement; or
3. unrestricted access to persons engaged in the activity from whom the assurance practitioner determines it necessary to obtain evidence.

If there is no engaging party, such as for performance engagements initiated by an Auditor‑General, the existence of a legislative mandate may obviate the need to agree on the terms of the performance engagement.  Even in those circumstances it may be useful for the assurance practitioner to communicate the terms of engagement to the responsible party, including referral of any legislative requirements imposed on the responsible party to provide access to information or people relevant to the activity.

(Ref: Para 28-31)

Planning involves developing an overall strategy for the scope, emphasis, timing and conduct of the performance engagement. The performance engagement plan, consists of a detailed approach for the nature, timing and extent of evidence-gathering procedures to be undertaken and the reasons for selecting them. Ordinarily, adequate planning:

• helps to devote appropriate attention to important areas of the performance engagement, identify potential risk areas on a timely basis and properly organise and manage the performance engagement in order for it to be conducted in an effective and efficient manner;
• assists the assurance practitioner to properly assign work to performance engagement team members, and facilitates the direction and supervision of engagement team members and the review of their work; and
• assists, where applicable, the coordination of work done by other assurance practitioners and experts.

The nature and extent of planning activities will vary with the performance engagement circumstances, for example the size and complexity of the activity and the assurance practitioner’s previous experience with it. Examples of the main matters to be considered include:

• the terms of the performance engagement.
• the characteristics of the activity and the identified criteria.
• the performance engagement process and possible sources of evidence.
• the assurance practitioner’s understanding of the activity and other performance engagement circumstances.
• identification of intended users and their needs, and consideration of materiality and the assessment of risk.
• personnel and expertise requirements, including the nature and extent of involvement by experts.

Planning is not a discrete phase, but rather a continual and iterative process throughout the performance engagement.  As a result of unexpected events, changes in conditions, or the evidence obtained from the results of evidence‑gathering procedures, the assurance practitioner may need to revise the overall strategy and performance engagement plan, and as such the resulting planned nature, timing and extent of further evidence‑gathering procedures.

In planning the performance engagement, if the scope of the engagement is based on overall objectives, then the assurance practitioner may identify sub‑objectives from which they can identify, select or develop the criteria, against which the activity’s performance can be evaluated.

Materiality

Professional judgement about materiality is made in light of surrounding circumstances, but is not affected by the level of assurance.  Materiality for a reasonable assurance engagement is the same as for a limited assurance engagement because materiality is based on the information needs of intended users.

The identified criteria may discuss the concept of materiality in the context of the preparation and presentation of the assurance report and thereby provide a frame of reference for the assurance practitioner in considering materiality for the engagement.  Although identified criteria may discuss materiality in different terms, the concept of materiality generally includes the matters discussed in paragraphs A28–A34.  If the identified criteria do not include a discussion of the concept of materiality, these paragraphs provide the assurance practitioner with a frame of reference.

Variations in performance, including omissions, are considered to be material if they, individually or in combination, could reasonably be expected to influence relevant decisions of intended users taken on the basis of the assurance report. The assurance practitioner’s consideration of materiality is a matter of professional judgement, and is affected by the assurance practitioner’s perception of the common information needs of intended users as a group. In this context, it is reasonable for the assurance practitioner to assume that intended users:

1. have a reasonable knowledge of the activity, and a willingness to study the assurance report with reasonable diligence;
2. understand that the assurance report is prepared and assured to appropriate levels of materiality, and have an understanding of any materiality concepts included in the identified criteria;
3. understand any inherent uncertainties involved in the measuring or evaluating the activity; and
4. make reasonable decisions on the basis of the assurance report taken as a whole.

Unless the performance engagement has been designed to meet the particular information needs of specific users, the possible effect of variations in performance on specific users, whose information needs may vary widely, is not ordinarily considered.

Materiality is considered in the context of qualitative factors and, when applicable, quantitative factors.  The relative importance of qualitative factors and quantitative factors when considering materiality in a particular performance engagement is a matter for the assurance practitioner’s professional judgement.

Qualitative materiality factors may include such things as:

• the number of persons or entities affected by the subject matter.
• the interaction between, and relative importance of, various components of the activity when it is made up of multiple components, such as a report that includes numerous performance indicators.
• the wording chosen with respect to the activity that is expressed in narrative form.
• the characteristics of the presentation adopted for the assurance report when the identified criteria allow for variations in that presentation.
• the nature of a variation, for example, the nature of observed variations from a control when the assurance report includes a statement that the control is effective.
• whether a variation affects compliance with law or regulation.
• in the case of periodic reporting on an activity, the effect of an adjustment that affects past or current activities or is likely to affect future activities.
• whether a variation is the result of an intentional act or is unintentional.
• whether a variation is significant having regard to the assurance practitioner’s understanding of known previous communications to users, for example, in relation to the expected outcome of the measurement or evaluation of the underlying subject matter.
• whether a variation relates to the relationship between the responsible party, the measurer or evaluator, or the engaging party or their relationship with other parties.
• when a threshold or benchmark value has been identified, whether the result of the procedure deviates from that value.
• when the underlying subject matter is a governmental program or public sector entity, whether a particular aspect of the program or entity is significant with regard to the nature, visibility and sensitivity of the program or entity.

Quantitative materiality factors relate to the magnitude of variations relative to reported amounts for those aspects of the assurance report, if any, that are:

• expressed numerically; or
• otherwise related to numerical values (for example, the number of observed deviations from a control may be a relevant quantitative factor when the assurance report is a statement that the control is effective).

When quantitative factors are applicable, planning the performance engagement solely to detect individually material variations overlooks the fact that the combination of uncorrected and undetected individually immaterial variations may cause the assurance report to be materially misstated.  It may therefore be appropriate when planning the nature, timing and extent of procedures for the assurance practitioner to determine a quantity less than materiality as a basis for determining the nature, timing and extent of procedures.

Materiality relates to the information covered by the assurance report.  Therefore, when the performance engagement covers some, but not all, aspects of the information communicated about an underlying subject matter, materiality is considered in relation to only that portion that is covered by the performance engagement.

Concluding on the materiality of the variations identified as a result of the procedures performed requires professional judgement. For example:

• the identified criteria for a value for money engagement for a hospital’s emergency department may include the speed of the services provided, the quality of the services, the number of patients treated during a shift, and benchmarking the cost of the services against other similar hospitals. If three of these identified criteria are satisfied but one applicable criterion is not satisfied by a small margin, then professional judgement is needed to conclude whether the hospital’s emergency department represents value for money as a whole.

Understanding the Activity and Other Performance Engagement Circumstances (Ref: Para 32-33)

Obtaining an understanding of the activity and other performance engagement circumstances is an essential part of planning and conducting a performance engagement. That understanding provides the assurance practitioner with a frame of reference for exercising professional judgement throughout the performance engagement, for example, when:

• considering the characteristics of the activity.
• assessing the suitability of criteria.
• assessing systems established and maintained for ensuring compliance with an entity’s mandate or internal controls as expressed in policies and legislation.
• identifying where special consideration may be necessary, for example factors indicative of wastage or fraud, and the need for specialised skills or the work of an expert.
• establishing and evaluating the continued appropriateness of quantitative levels of performance (where appropriate), and considering qualitative materiality factors or benchmarks.
• developing expectations for use when undertaking analytical procedures.
• use of data analytical tools to undertake the engagement.
• designing and undertaking further evidence-gathering procedures to reduce risk to an appropriate level.
• evaluating evidence, including the reasonableness of the responsible party’s oral and written representations.

In a performance engagement, understanding internal controls relevant to the activity assists the practitioner in identifying the types of variations and factors that affect the risks of material variation.  Professional judgment is needed to determine which controls are relevant in the engagement circumstances.

When the objective of a performance engagement is to assess the design or implementation of controls over a process (for example, a process for dealing with patients in a hospital emergency room), the assurance practitioner may consider, during the initial planning phase,  identifying the internal controls to the extent necessary to inform the engagement scope and the risk assessment.  The assurance practitioner considers the evaluation of the design or determines the implementation of the controls later in the engagement as internal controls form the activity for this performance engagement.

When the objective of a performance engagement is to conclude on a specific outcome of a process, controls may not be relevant to that engagement.  For example, an assurance engagement may be designed to reach a conclusion regarding whether the time taken to process specific items (for example, applications to receive a service) over a specified period of time exceeds what is permitted under stated policies.  The practitioner might simply examine all the items processed during the specified period and conclude on whether there were material variations with the stated policies.

Evaluating the design of a control involves considering whether the control, individually or in combination with other controls, is capable of effectively preventing, or detecting and correcting, material variations.  Implementation of a control means that the control exists and that the entity is using it.  There is little point in assessing the implementation of a control that is not effective, and so the design of a control is considered first.  An improperly designed control may represent a significant deficiency in internal control.

(Ref: Para 35-38)

Sufficiency is the measure of the quantity of evidence.  Appropriateness is the measure of the quality of evidence; that is, its relevance and its reliability.  The assurance practitioner ordinarily considers the relationship between the cost of obtaining evidence and the usefulness of the information obtained.  However, the matter of difficulty or expense involved is not in itself a valid basis for omitting an evidence‑gathering procedure for which there is no alternative.  The assurance practitioner uses professional judgement and exercises professional scepticism in evaluating the quantity and quality of evidence, and thus its sufficiency and appropriateness, to support the conclusions in the assurance report.

Performance engagements require the application of assurance skills and techniques and the gathering of sufficient appropriate evidence as part of an iterative, systematic assurance engagement process.  For further guidance on the nature, timing and extent of evidence‑gathering procedures for performance engagements, refer to ASAE 3000.

In a performance engagement if the assurance practitioner becomes aware of a matter that leads the assurance practitioner to question whether sufficient appropriate evidence has been obtained, the assurance practitioner ordinarily pursues the matter by undertaking other evidence‑gathering procedures sufficient to enable the assurance practitioner to report.

Written Representations

If the performance engagement is initiated by the assurance practitioner, the assurance practitioner may not be in a position to obtain representations from the responsible party, particularly as the responsible party may not be a party to the performance engagement.

Representations by the responsible party cannot replace other evidence the assurance practitioner could reasonably expect to be available.  An inability to obtain sufficient appropriate evidence regarding a matter that has, or may have, a material effect on the evaluation or measurement of the activity, when such evidence would ordinarily be available, constitutes a limitation on the scope of the performance engagement, even if a representation from the responsible party has been received on the activity.

Written representations may include that the responsible party:

1. acknowledges its responsibility for conducting the activity, intended to achieve a certain level of performance;
2. has provided the assurance practitioner with all relevant information and access agreed to, as set out in paragraph A20;
3. has disclosed to the assurance practitioner any of the following of which it is aware may be relevant to the performance engagement:
   1. variations in achievement of intended performance; or
   2. any events subsequent to the period covered by the assurance practitioner’s report up to the date of the assurance report that could have a significant effect on the assurance practitioner’s report.

(Ref: Para 39)

The assurance practitioner needs to consider the impact of material variations in the performance of the activity when evaluated against the identified criteria, on the conclusions in the assurance report. A variation is material when, in the assurance practitioner’s judgement, it has the potential to affect:

1. decisions made by intended users about the performance (economy, efficiency and/or effectiveness) of an activity; or
2. the discharge of accountability by the responsible party or the governing party of the entity.

Further guidance on the qualitative and quantitative factors for the assurance practitioner to consider with regard to variations in performance of an activity refer to A30-A34.

(Ref: Para 40)

The extent of consideration of subsequent events that come to the attention of the assurance practitioner depends on the potential for such events to affect the activity and to affect the appropriateness of the assurance practitioner’s conclusions.  Consideration of subsequent events in some performance engagements may not be relevant because of the nature of the activity.

The assurance practitioner does not have any responsibility to perform procedures or make any enquiry after the date of the report.  If however, after the date of the report, the assurance practitioner becomes aware of a matter identified, the assurance practitioner may consider re‑issuing the report.  In a performance engagement the new report discusses the reason for the new report under a heading “Subsequent Events”.

(Ref: Para 43-48)

There may be circumstances where an Auditor‑General, having conducted a performance engagement, decides not to report to Parliament or to publish an assurance report.  The Auditor‑General usually has discretion under their mandate to choose whether and to whom they will report on performance engagements.  Assurance reports which are tabled in Parliament become available to the public.  In certain circumstances it may be necessary for the confidentiality of the assurance report to be maintained, in which case the report may, in accordance with relevant legislation be provided to the relevant Parliamentary Committee or other appropriate user, in confidence.  The Auditor‑General considers the public interest in determining whether the  assurance report will be made publicly available.

This ASAE does not require a standardised format for reporting on performance engagements even though paragraph 45 identifies the basic elements of the assurance report. For instance, under:

• paragraph 45(a), the title of the assurance report may differ depending on whether the assurance practitioner is an Auditor-General or a practitioner in the private sector. However, in both instances the title would convey that it is an independent report.
• paragraph 45(g), the assurance practitioner’s conclusions may be drafted as appropriate to recognise local legislation or custom and may be worded in terms of a response to the statement of purpose or the audit question.

Therefore, assurance reports are tailored to the specific performance engagement circumstances with the assurance practitioner using professional judgement in deciding how best to meet the reporting requirements detailed in paragraph 45 in conveying the conclusion(s). The assurance practitioner includes the matters in paragraph 45 as a minimum and reports in the manner and to the extent necessary to facilitate effective communication to the intended users. Whilst the assurance conclusion makes a clear statement communicating the assurance practitioner’s conclusion, the assurance report may include other matters which the assurance practitioner considers meet the information needs of the intended users, such as:

• terms of the performance engagement;
• overall objectives and sub-objectives of the performance engagement;
• identified criteria applied;
• findings relating to particular aspects of the performance engagement; and
• in some cases, recommendations.

Ordinarily, any findings and recommendations are clearly separated from the assurance practitioner’s conclusion on the performance of the activity.

Reporting Findings, Recommendations and Responsible Party Comments

The assurance practitioner may expand the assurance report to include other information and explanations, including:

• relevant background information and historical context.
• the assurance approach.
• underlying facts and identified criteria applied.
• disclosure of materiality levels.
• findings relating to particular aspects of the performance engagement.
• analysis of the causes of variations in the activity’s performance.
• recommendations to address variations identified.
• comments received in response to the report from the responsible party/ies.

The decision to include any such information depends on its significance to the needs of the intended users.  Additional information is clearly separated from the assurance practitioner’s conclusion and worded in such a manner so as not to affect that conclusion.

Variations in the Activity’s Performance

If material variations are identified, the assurance practitioner’s conclusion clearly reflects that either:

1. the activity did not perform, in terms of economy, efficiency and/or effectiveness, with respect to the identified criteria of the activity or certain objectives or sub-objectives of the performance engagement;
2. the activity did not perform, in terms of economy, efficiency and/or effectiveness with respect to the identified criteria of the activity or the objective of the performance engagement, as a whole; or
3. the assurance practitioner was unable to conclude on the activity’s performance when the assurance practitioner was unable to obtain sufficient appropriate evidence regarding the activity’s performance as a whole.

The assurance practitioner’s conclusions described in paragraph A54, are equivalent modified conclusions under ASAE 3000 and the equivalent terms in ASAE 3000^[17] are:

• a qualified conclusion – circumstances described in sub-paragraph A54(a).
• an adverse conclusion – circumstance described in sub-paragraph A54(b).
• a disclaimer of conclusion – circumstance described in sub-paragraph A54(c).

(Ref: Para 52)

Documentation includes a record of the assurance practitioner’s reasoning on all significant matters that require the exercise of professional judgement, and related conclusions.  The existence of difficult questions of principle or judgement, calls for the documentation to include the relevant facts that were known by the assurance practitioner at the time the conclusion was reached.

In applying professional judgement to assessing the extent of documentation to be prepared and retained, the assurance practitioner considers what is necessary to provide an understanding of the work undertaken, the results of that work, the evidence obtained and the basis of the principal decisions taken to another experienced assurance practitioner, who has no previous connection with the performance engagement.  It is, however, neither necessary nor practicable to document every matter the assurance practitioner considers during the performance engagement.

Identifying characteristics of the activity’s performance being tested that the assurance practitioner may document include:

1. subject matter; and
2. assertions being tested.