Report 29 - Access to Information Review Task Force

NEW REPORTING FRAMEWORK FOR ASSESSING THE PERFORMANCE OF THE ACCESS TO INFORMATION PROGRAM

3.0 Study Findings - Current Measures of Performance

3.1 The ATI Processes

Under the Access to Information Act, Canadians have the legal right to information recorded in any form and controlled by federal government agencies. The statute, regulations, and Treasury Board Secretariat Policy and Guidelines clearly encourage institutions to provide individuals access to information to which they are entitled without having to resort to the formal procedures provided for in the legislation, which is intended to complement, not replace, existing procedures for obtaining government information. Requesters, however, access the formal request mechanisms available to them. (There were more than 20,000 requests made in 2000-2001).

Formal requests for information to a department are made to the ATIP Coordinator (in this report we refer to the Coordinator and his/her staff as the Access to Information and Privacy office). The ATIP office is responsible for ensuring that government institutions furnish information to the requester within 30 days of its receipt. Extensions may be applied by the government institutions if there are many records to examine and meeting the 30 day deadline would interfere with the operations of the department; other government agencies need to be consulted; or, third parties need to be notified.

In general, the ATIP office receives requests from a number of sources including the general public, the media, Members of Parliament, businesses, and other non governmental organizations. The ATIP office is then responsible for reviewing the requests, determining the most appropriate region/sector to respond to the request (termed Office of Primary Interest or OPI), and then assigning the request to the OPI. Officers within the ATIP office are responsible for liaison between the institution and the requester. This entails the delivery of acknowledgement, discussion over clarification of the request, as well as the forwarding of a fee estimate, if required.

When an ATI request is sent to an OPI, it is often accompanied by a time in which the response is expected back in the ATIP office. The standard time for an OPI response is seven or eight days. It is the task of the OPI to collect all the information and files relevant to the request and prepare the appropriate information package for transfer to the ATIP office. Included with the material requested, some OPI's also prepare a letter of recommendation that will alert the ATIP officers to information in the records that must or should be protected; that contain third party or personal information; and/or, that requires consultations. This recommendation letter may also provide rationale or background information that will assist the ATIP officer in his/her review of the records. If the request is deemed sensitive, the OPI officer may also prepare an impact statement that will alert: i) senior officials to the anticipated consequences of disclosure, protection, or the fact that the record does not exist; and, ii) communication officials to the need to prepare for enquiries in the subject matter area.

In general, once the package of reports is prepared, the OPI sends it back to the ATIP office where it is processed for disclosure. The ATIP office is responsible for ensuring that the file addresses the needs of the requester within the bounds of the Access to Information and Privacy Acts. The officer responsible for the file will paginate all the records; do a line by line review of the contents - marking parts of the records that contain information exempted or excluded under the Act; physically cut and paste, or otherwise block out exempted or excluded information; and, prepare the package for disclosure to the requester.

At this point it is important to note that institutions differ in the sign-off authority given to an access to information file. Some require sign-offs in the OPI (ADM, RDG) before going to the ATIP office for final review; some require the ATIP office to send the file back to the OPI for sign off after the final review; and others don't require sign-off at this level at all. Similarly, some ATIP offices have delegated authority for final release of all or some information, while others require the sign off at the Deputy Minister or Ministerial level. The point to be made here is that there is no consistency with regards to delegated authority on release of information.

Return to Table of Contents

3.2 Measurements of Performance

How well is a department doing in its obligation to "provide information to a requester?" One of the key measures of a department's success or failure in the area of transparency/openness (i.e., concerning its provision of information to requesters) is the performance of its ATIP office. Unfortunately, there is no standard measure of performance used for these offices. In order to analyze the current measures it was imperative for us to look at the types of information the different ATIP offices were providing to bodies that could hold them accountable for their performance. Through the course of our interviews we found that ATIP offices were providing performance-based information to the Treasury Board Secretariat, the Office of the Information Commissioner and their own senior management. The remainder of this section will outline the types of information these bodies required from the ATIP offices and the rationale behind their requests.

3.2.1 The Treasury Board Secretariat

Section 72 of the Access to Information Act requires that the head of every Government institution prepare and submit to Parliament an Annual Report on the administration of the Act within the institution during each fiscal year.

All individuals interviewed for this project noted that they prepare their Annual Report's to Parliament and submit them to the House, the Senate and the Treasury Board Secretariat annually.

This report contains a completed copy TBS/SCT form 350-62 (Report on the Access to Information Act). The statistics collected on this form include:

I. Volume of requests.
II. Disposition of requests.
III. Exemptions invoked.
IV. Exclusions cited.
V. Completion time.
VI. Extensions granted.
VII. Translations required.
VIII. Method of access.
IX. Fees collected/waived.
X. Costs.

In general, the Annual Reports to Parliament reviewed by GGI include these forms with a brief description of the statistical findings. Some reports also included brief discussions of measures such as:

• The sources of requests.
• The number and type of complaints to the Information Commissioner.
• Number of requests for consultations.
• Number of informal requests.

In addition to statistical analysis, some reports detail the organization of the ATIP function in their departments. Only one Annual Report to Parliament reviewed outlined a plan for improving a department's performance in providing access to information.

3.2.2 The Office of the Information Commissioner

One important aspect of the mandate of the Information Commissioner is the role as special ombudsman (appointed by Parliament) to investigate complaints that, contrary to the provisions of the Act, the government has denied access to information. Such complaints are made under the Access to Information Act.

The Office of the Information Commissioner (OIC) is fundamentally interested in institutions' compliance with the Act - as measured by substantiated complaints recorded in each institution. Since delays make up the majority of complaints (42% last year), the OIC is particularly interested in looking at the ability of an institution to comply with the 30-day turn-around. If the institution does not meet its 30-day turnaround requirement, a requester has the right to file a complaint with the Information Commissioner.

Complaints to the Information Commissioner become public knowledge when they are used as examples of where an institution and/or Government are not fulfilling its promises to the Canadian public of transparency and accountability. In an effort to forestall problems in this area, the OIC therefore monitors completion times registered by institutions in their Annual Reports to Parliament. If an institution has a number of delays that did not result in an official complaint, the Information Commissioner may initiate a complaint himself. Similarly, when investigating complaints, the OIC may find other instances of non-compliance within a file and register additional complaints.

In terms of reporting to the OIC, most institutions indicated that they do not provide formal reports to this office. Most information used by the OIC is from their own records of complaints, as well as their review of the Annual Reports to Parliament. The Information Commissioner, in highlighting the problem of delay complaints, has also conducted studies of delays in response times in several major government institutions. Since 1998/99, the OIC has issued "report cards" to these select institutions, adopting as their measure of performance, the percentage of access requests which have become "deemed refusals" under subsection 10(3) of the Access to Information Act. The OIC measure therefore counts cases that have missed either the original 30-day deadline or any extension of the deadline claimed by the institutions in accordance with the Act. Those institutions targeted for Report Cards are often asked for specific statistical information on an ad hoc basis. Again, however, this information mostly pertains to a institution's compliance to the time-lines. The OIC feels it is important to note that they do not see their Report Cards as commenting on the overall performance of an institution, but rather comments on how well institutions are complying with the timeframes established in the Act.

Because of the Information Commissioner's legislated role of investigating complaints, their reports tend to highlight institutions with problematic compliance issues.

3.2.3 Senior Management

GGI found that Senior Management in each of the six institutions interviewed for this project had different reporting requirements. There was also a discrepancy as to level of management requesting reports. Some ATIP offices were required to send reports to the Minister's Office and/or the Deputy Minister's Office, others sent information only as high as the ADM level. One institution noted that the reports were only required by the DG responsible for ATIP.

For the most part, interviewees reported that senior management was primarily interested in timeline compliance rates. The main reason for this was reported to us that they wanted to make sure that they passed the scrutiny of the OIC. What was requested beyond this measure differed greatly by institution. We found a certain level of ambiguity among the interviewees with respect to the "working definition" for compliance rates and the 30-day turnaround requirement. For example, some interviewees believed the OIC reported compliance rates regardless of approved extensions while some interviewees reported that the OIC took such extensions into account in their pronouncements on compliance.

Some institutions noted that their senior management required weekly updates as to the content of requests. This would allow them to highlight areas of interest/sensitivity for close monitoring. Other institutions were requested to give senior management this type of information on an as-and-when required basis.

There was one institution that was looking at ATIP operations and wanting to assign accountability for delays. Senior management in this institution requested breakdowns as to where files were in the organization so they could make the ADM's of OPI's more accountable for compliance. A second institution is in the process of setting up this type of measure and a third mentioned a desire to do this as well.

In terms of annual performance reporting, only two institutions interviewed showed attempts at doing this. One institution's Annual report to Parliament set out clear goals and objectives for the ATIP office. The main goal outlined in this report focuses on the institution's ability to comply with the ATIA. It then outlines it performance in this regard and sets objectives for improvement. According to the interviewee from this institution, the ATIP office is measured, by senior management, on its ability to meet both the overarching goal of compliance as well as its ability to meet the objectives for improvement set out the previous year.

The second institution that has a formal performance report monitored by senior management does so because that institution has developed a corporate-wide approach to performance measurement. Client service and transparency to the public are prime measurements of the institution as a whole and since the ATIP office is a marker for this, setting and meeting its performance objectives are of interest to management. One of the noted initiatives in the institution is to enhance the ATI program and the outcome used to measure success in this area is noted as "shorter turn-around times and better service". Annual reports on the ability for the ATIP Office to plan for and meet these outcomes are therefore necessary.
What one can see from the efforts of institutions noted above is the development of two sets of reports - one for public reporting and the other for internal performance improvements.

Return to Table of Contents

3.3 Strengths and Weaknesses of Data Collected

During interviews with key informants in the institutions questions were raised regarding the usefulness of the statistics currently being gathered to meet the reporting requirements of the TBS, the OIC and senior management in their departments. What follows is a summary of their comments.

3.3.1 Strengths

Except for the two institutions moving towards a formalized performance management system, none of the departmental interviewees voluntarily offered opinions regarding the strengths of the data collected. They all focused their discussions on the problems with using the current data required as the measuring sticks for their performance.

In the conversation with the interviewee from the OIC, however, GGI was told that using timeliness (evaluated via % cases in deemed refusal) is a good measure of an institution's performance as it relates to its ability to comply with the time lines of the Act. According to the OIC, the ability of an institution to comply with the time-lines has proven to be an effective barometer for the performance of their ATIP office. They have reported that poor compliance with timelines can alert institutions to other more subjective problems (e.g., misapplication of extensions, being over zealous in use of exemptions, poor communications between the institution and requestors, etc.), which they can correct. Two interviewees agreed with this observation that the OIC's reports have prompted their institution to look at the root causes of their delays and develop plans to alleviate them.

3.3.2 Weaknesses

All interviewees noted a number of weaknesses with the data currently being used to judge the performance of the government's Access to Information program. The most prominent complaint from institutions was the fact there is no measure as to the complexity of requests. It was noted that there are a number of factors that make a request simple or complex (e.g., number of pages reviewed, sensitivity of topic, number of consultations required, number and types of exemptions/exclusions invoked, physical location of files, etc.) and it is unfair to aggregate all requests as equal. This does not mean that a complex request should be given more time than is legislated, but that institutions should be judged on the amount of effort required to respond to requests within the allotted time-frame. This argument stems from the fact that some institutions may receive large volumes of simple requests, while others receive the same volume but of more complex requests - current statistics would just show that they received the same volume.

Related to the above point, respondents were concerned that there is no measure reported that accurately assesses the time spent by an institution to respond to a request. This has implications for resourcing and costing the governments access program. It should be noted that the cost measure on the annual TBS report is only an estimate of the cost of staffing the ATIP office. It does not take into consideration the time spent within the OPI's to respond to an access request.

Another complaint regarding the current measures relates to the argument presented in the previous section. While some departments noted the positive results of the OIC's reports, most, if not all, suggested that the governments overwhelming desire to reduce delays has shifted the balance away from making sure quality responses are made and towards making sure the numbers are met. This, it was argued, has a few serious risks:

1) it increases the likelihood that information that should be protected could get out by mistake, with the potential for serious consequences for companies, individual privacy or Canada's relationship with other countries, for example;

2) it puts pressure on the already maximized resources in the ATIP office - causing high stress, burnout and turnover; and,

3) the focus on output at all costs has reduced the time available for planning ways to improve performance.

A fourth area that was highlighted as a weakness in the current reporting requirements is that lack of trend analysis being required - at both the institutions and government level. Any trend analysis being done now is mostly ad hoc in nature. If senior management and/or TBS collected more detailed data, they may be able to detect patterns in terms of the needs of requestors. This, in turn, at least for certain cases may lead to more proactive and informal release of information, thereby, reducing the burden on the ATIP offices and improving the government's overall performance in access for these types of cases.

Some interviewees noted that their ATIP offices became aware of trends through observation and were able to respond by putting systems in place to do just this.

For example, one institution had backlog and compliance problems that were typically associated with large batches of requests coming into the department at the same time. Over a long period of time they came to realize that most of these batched requests were coming from one particular person. They quickly responded to this situation by dedicating one resource to ensuring that this individual's needs were being met. The result was a more satisfied client and a significantly reduced burden on the institution, which now began to do more focused analysis on incoming requests.

Another institution inadvertently noted there was a trend in the number of days it was past its extensions. This caused them to re-evaluate their extension time requests. The result was longer extensions but more compliance with the regulations.

It can be argued that a more systematic approach to trend analysis in ATIP would help the overall performance of the Access to Information program by directly reducing the burden placed upon the institutions. For example, if it is found that there is documentation related to a large number of access requests, then the government (or a department) could make an informed decision about making this documentation more accessible. Similarly, if it can be discerned that a large number of requests are for protected information, then one can make informed and strategic decisions about communicating the need for confidentiality in particular areas.

Related to the above point, interviewees noted that there are no requirements for the ATIP office to formally acknowledge actions taken to improve the public's accessibility to information within their department - i.e., the transparency of their department. Several department interviewees noted that the use of a website might be used to provide an increased amount of information, and special arrangements made for researchers to access non-sensitive information without using ATI.

All interviewees offered verbal examples, like those noted above, where they implemented a plan that helped improve their performance in this regard. They suggested that any measure of the government's access to information performance should include such success stories. Included in these success stories, some noted, should be some mention of the institutions record in the informal release of information. While this would be a difficult measure to gather, ways could be used to detect this.

For example, institutions have set-up designated reading rooms so the public can review information released in previous ATI requests. These institutions could track the use of these rooms. Similarly, one interviewee noted that their ATIP office gets requests for information that is already available to the public via websites, pamphlets, etc. While these requests provide examples of when and how the government helps the public get access to information, they are not currently being tallied. This type of client service happens constantly throughout the government, and as it is, there is no attempt to collect data to prove that Canadians are being given such access. The only data being collected show where they are being denied access to information within 30 days of their request.

There are two other concerns voiced by interviewees that relate to the current data being collected. The first regards the Annual Report to Parliament. All interviewees, including the TBS and the OIC questioned the usefulness of the TBS/SCT Form 350-62 (Report on the Access to Information Act). Strong concerns were voiced concerning the validity of section 5 - Completion Time. As it is written, institutions can only indicate the number of requests they were able to complete in 30; 31-60; 61-120; or, 121+ days. Interviewees mentioned that the statistics on completion times did not take into account appropriate extended time frames.

Questions also arose regarding the need to report on the exemptions invoked, the exclusions cited, the extensions applied, or the translations required. Interviewees wondered who would be interested in this information? Some researchers in academe are interested in answers to such questions (e.g., School of Policy Studies researchers at Queens University). However, the interviewees were not probed in detail on the related availability and value of this type of information. Nevertheless, for some researchers this type of information seems to reflect on the overall heath or relevancy of the Access to Information data system.

The second concern that arose regarding the current data being collected related to the OIC's measure of complaints. Two institutions noted that the OIC's measures include incidents when complaints were lodged that were completely unfounded. These complaints often relate to a requestor's inability to retrieve information protected by the legislated exemptions or exclusions, or their inability to retrieve data that never existed. All interviewees reported that unfounded complaints should not be used as a measure of an institution's performance. Some institutions would like to see separate statistics kept on the complaints made by requesters compared to complaints made by the Information Commissioner. The OIC agrees that such statistics would be useful.

3.3.3 Barriers

During the interview process, GGI was interested in comments relating to the identification of barriers to the collection and reporting of effective performance information related to access. In this regard, all interviewees noted that it was relatively easy to fill in TBS/SCT form 350-62 (Report on the Access to Information Act). The TBS noted that the Annual Reports were handed in on time. The OIC noted that they do not have access to current government statistics when they do their Annual Report to Parliament and that they don't think that the TBS/SCT form is comprehensive enough.

All but one of the institutions represented in the interviews utilized ATIPflow as their primary tracking and statistical collection tool. They all commented on its ease and usefulness. A few did mention however, that one of the biggest barriers to effective statistical reporting is data entry error on the part of people using this system. They note that these errors are a natural part of business and are usually found and corrected. The one institution that does not use ATIPflow commented on its desire to do so. They felt this system would make data collection and reporting easier, but mentioned that the cost of the system was holding them back from implementation.

What was interesting to note during discussions on this topic, was that each institution had the ATIPflow system customized to suit its particular reporting needs. As a result, the system lost the benefits of consistency among institutions. This may make a government-wide roll up of data difficult. In 1998, the TBS announced the implementation of the Coordination of Access to Information Requests System (CAIRS). This system was intended to coordinate ATI requests interdepartmentally. Institutions are required to submit ATI data into a central system so an overall roll-up can occur. Unfortunately, this is not happening as originally envisaged¹ - as evidenced in the report by Alasdair Roberts entitled "An Evidence-Based Approach to Access Reform." Queen's University School of Policy Studies (July 2001).

¹Alasdair Roberts, Jonathan DeWolfe and Christopher Stack in An Evidence-Based Approach to Access Reform, Queen's University School of Policy Studies, July, 2001. One of the difficulties with CAIRS system noted in the Queen's study (p. 5) includes an under-reporting of requests by institutions (due to non-participation or inaccurate reporting) in the CAIRS database.

Return to Table of Contents

Next Page