Report 29 - Access to Information Review Task Force

NEW REPORTING FRAMEWORK FOR ASSESSING THE PERFORMANCE OF THE ACCESS TO INFORMATION PROGRAM

4.0 ATI Performance Reporting Practices and Suggested Improvements

This section looks at current ATI performance reporting practices, identifies gaps and shortcomings in these and suggests improvements that will lead to a more effective reporting system.

4.1 Current Reporting

Currently, the public and Parliamentarians have access to three basic reports on the performance of Access to Information: (1) annual reports from departments and agencies; (2) a statistical roll-up of these reports prepared by the Treasury Board Secretariat (TBS); and, (3) the annual report of the Information Commissioner (IC).

The first two sets of performance reports are quite similar. Their format follows the guidelines set out by the TBS in the Information Circular entitled Annual Reports and Statistical Reports of the Access to Information Act and the Privacy Act. These reports contain a wide variety of statistics including: the number of requests, source of requests, distribution of requests completed, exemptions invoked, exclusions cited, completion times, extensions applied, translations required, method of access, fees charged and the costs of the ATIP offices. Some of the departmental/agency reports include a limited analysis of the data. The TBS roll-up of these statistics is designed to give the public and Parliamentarians an idea of the general ATI picture. It contains no analysis of the data, and therefore is of limited use in understanding and improving the general performance of the government on ATI.

The Information Commissioner, by way of contrast, collects data such as the status of complaints, complaint findings, turnaround time profiles, geographic distribution of complaints, notices of extensions and number of deemed refusals on account of delays. They would like access to more data. They believe that institutions are currently collecting information for internal purposes and that this data could be of use to the TBS, themselves and others interested in the health of the access system. The OIC argued that institutions should be required to submit this data via a modified TBS form. This form, they suggested, should include a more comprehensive set of indicators - such content analysis, extension profiles, etc. A copy of the OIC's suggested reporting format is attached as Appendix E.

Return to Table of Contents

4.2 Gaps and Deficiencies in Reporting

Based on our document review and given set of interviews, the following gaps and deficiencies have been observed in the reporting of ATI Program information:

1) None of the current statistical reports (i.e., from institutions, the TBS, or the annual report of the IC) provide an accurate² or balanced picture of the Access to Information's performance across the federal government.

For example, the institutions and TBS lack client satisfaction data and have a paucity of analysis relating to current challenges and emerging trends. The IC report suffers from an almost-exclusive focus on complaints. It is analogous to having the "complaint department" report on how well a business is performing. Even the most successful businesses have complaints, but these are balanced by how well it does in terms of satisfied customers, quality improvements (leading to more profitability), gains in client reach, etc.

2) There is no single, integrated coherent report on the performance of Access to Information as a whole. This places a burden on Parliamentarians and the public.

3) The collected data profiles do not identify the cause and extent of problems and/or trends if they exist - thereby, making it difficult to pinpoint the origins of performance issues and avenues for correcting them.

This observation is further supported in part by the work of Professor Alasdair Roberts, Jonathan DeWolfe and Christopher Stack - School of Policy Studies, Queen's University ("An Evidence-Based Approach to Access Reform", July, 2001). Their report suggests a substantial amount of ATI data is available for analysis on federal systems databases like CAIRS and ATIPflow, but are not being used to their full potential. They suggest many possible analyses - e.g., who uses the Act and for what purpose; the distribution of requests by category of requester and subject matter; distinct patterns of usage within the ATIA; patterns of activity across organizational sectors; profiling of complexity of request based on proxy indicators such as number of pages reviewed by the ATIP office, the perceived sensitivity of the request, the difficulty of conducting a search for requested records, the need for multi-consultations with other departments/agencies, etc.; disclosure practices by type of requester, subject-matter, or institutional sector; and, profiles of fee collection practices.

This latter gap is the biggest challenge faced by most ATIP offices. Thus, due to these shortcomings, interested parties are often left with incomplete and, in some instances, competing performance 'stories' or conclusions.

Return to Table of Contents

4.3 Proposed Improvements

Several improvements are suggested:

1. Departmental/agency reports should include:

• data on resource utilization;
• data on client satisfaction; and,
• enhanced data analysis.

2. The TBS report should:

• make the same improvements as the departmental/agency reports;
• incorporate a government-wide analysis, and trends, including longitudinal statistics (when feasible); and,
• provide a balanced, integrated report on the overall, government-wide performance of Access to Information (including selected excerpts or digests of the IC annual report).

3. The TBS integrated report should try to tell the Access to Information's performance story. The performance model described in section 2.0 should be considered as one way to tell this story. It suggests addressing such questions as:

i) What conditions (i.e., backlogs, existing and new demands for information, etc.) did the institution expect to deal with during the coming year?

ii) What actions, if any, did the institution decide to take to address these conditions?

iii) What happened during the course of the year in review?

• This would discuss both access requests under the Act and their processing, as well as proactive releases of information designed to increase public access to information and to reduce the ATIP workload.
• It would also include the identification of any untoward or unanticipated demands and what ATIP did to address these.

iv) How successful were the actions taken over the last year?

v) What is the plan for ATI in the coming year?

Such a format would provide the public and Parliamentarians with a complete and balanced picture of Access to Information's performance - past, present and future. It would constitute a learning mechanism for the institutions and could generate a healthy competition between institutions in publicizing successes and best practices.

Return to Table of Contents

5.0 Concluding Remarks and Suggested Next Steps

Concluding Remarks. Section 3 of this report outlined the way in which access to information is measured in terms of success or failure of the ATIP office to meet the mandatory deadlines of the legislation. It discussed the types of reports being generated to measure this performance and then listed the strengths and weaknesses associated with the data collected for these reports. As a means of concluding this section it is fitting to reflect upon one of the questions asked in the interviews - namely: Do the statistics now in the public domain tell an adequate story on the access performance of government/your department? When asked this question, all interviewees answered "no." For all the reasons noted in section 3.0, all interviewees (even the OIC) felt confident in saying that the current statistics do not offer an accurate, or fair, measure of the government's efforts to offer access to information.

Suggested Next Steps. There are several important steps that need to be carried out before proceeding with the aforementioned "proposed improvements". It is suggested that:

1. The Access to Information Review Task Force consider recommending the Performance Model described in this report as the basis for the standard framework for assessing the performance of the ATI Program in the federal government.

2. Further vetting and refining of the Performance Model would be required if the Task Force decides to recommend this framework as the government standard in this regard. For example, all key stakeholders in the ATI Program would need to be interviewed and consulted for their inputs/comments/etc. on the Model itself. This step would seek to harmonize the needs of the key stakeholders in adapting such a framework approach to the assessing of performance of the ATI Program.

It should be emphasized that the introduction of such an approach would be done gradually, as only one system of performance measurement and management is sought and this will take time and effort to bring all key stakeholders into full use - e.g., a systematic phased implementation plan would need to be developed and agreed upon by key stakeholders; such a plan would also investigate and address issues of resourcing, responsibility, systems interface, reporting content, format and frequencies.

3. Further research into the types of ATI Program requests, including the possible modeling of complexity of requests³ , is warranted as part of improving the eventual performance measurement and reporting of ATI requests.

4. As part of its leadership role for Access to Information, the Treasury Board Secretariat should consider developing and testing a questionnaire that would measure the satisfaction of ATI clients. It could make this questionnaire available to departments and agencies to encourage them to improve their performance measurement and performance reporting.

5. Once a more comprehensive and reliable database is established, the TBS should encourage research to enhance the understanding of the Canadian federal ATI system and its performance.

²Based on the document review and the small set of interviews for this review, the ATIP data used by the departments/agencies, the TB and the IC appear to be statistically valid (i.e., conceptually sound or justifiable). However, perceptions persist with the use and interpretation of compliance rates that may shed doubt on this validity (e.g., situations where an unfilled request after 30 days is incorrectly deemed a refusal; or, where no account of valid extensions is offered for cases taking over 30 days to complete; or, where there is no account of legitimate debates about the releasability of data - if not releasable, then the 30 days limit is not an appropriate measure of timeliness).

³Supra Note 1 Related to this analytical 'next step' are suggestions for further research by Alasdair Roberts, Jonathan DeWolfe and Christopher Stack in a report entitled: "An Evidence-Based Approach to Access Reform." They found that focused-analytic research on ATIA databases will assist in improving government decision and policy making related to the ATIA itself. The report outlines several key research questions (and data analysis paths) to achieve this objective; namely: 1) what causes delay? 2) are sensitive requests handled differently? 3) do different users receive different treatment? 4) are frivolous and vexatious requests a serious problem? 5) what changes to the fee structure would be beneficial to the client and the Program?

Next Page

Return to Table of Contents