Table of Contents | Print This Page

Analysis of Questionnaires | Synopsis From: The Internal Responsibility System In Ontario Mines

  • Revised: September 2009
  • Content last reviewed: June 2009
  • Final Report: The Trial Audit & Recommendations
  • Revised: Steering Committee 23 October 2000 (September 2000; 31 July 2000)
  • Ian M. Plummer, Peter W. Strahlendorf, Michael G. Holliday

For analysis purposes we grouped the questions into a series of "clusters", each with the same theme--e.g., questions involving the planning of the work are grouped together in the "planning cluster". Much of the report is concerned with investigating such questions as: "Why did the mines with the best safety performance score high in their responses?" Which questions, or clusters of questions contributed the most to a high score, and which did not-and why? It was an exercise that enabled us to make suggestions as to how the audit tool may be improved.

We approached this task from two standpoints. The first looked at the responses in terms of such IRS "concepts" as leadership (at all levels in the organization), work planning, and information flow. The second looked at responses from the perspective of each level in the organization-for example: how did the supervisors as a group do? or the mine managers?. The first was conceptual, while the second was more structural.

The exercise essentially consisted of an examination of the various clusters on a question by question basis. Such detailed work is virtually impossible to succinctly summarize except in the form of recommendations, and these form part of the final chapter of the report. Accordingly, we suggest that readers of this synopsis investigate the details presented in Chapters 4 and 5 to clarify, for themselves, the reasons for the report's recommendations.

IRS Performance Indicators

As a general observation, there did appear to be a relationship between the preponderance of questions directed to workers and the degree of correlation between IRS performance and the medical aid statistics. There may be several reasons for this. In part, the larger numbers of worker respondents helps. But the reason may have more to do with how abstractions have to filter down and be translated into concrete improvements in conditions and practices in order to reduce risk. It is workers who can see best if policies, systems, leadership activities and so on are being put into practice. Conversely, it might be said that the more senior people are usually quite aware of what the "correct" answers should be to "system oriented questions". The enthusiasm of senior people to "look good" on the survey may have influenced results, giving a more homogeneous response across mines regardless of OHS performance.

The Planning Cluster:

There were only eight questions in this cluster. Of these, three were directed to workers and three to front-line supervisors (FLSs). Thus there was a strong emphasis on the views of the people closest to the work. This cluster of questions had a good correlation with OHS performance. The workplaces in which workers and supervisors were involved heavily in planning the work tended to be the mines where there were fewer accidents (as measured by medical aid cases).

The Information Cluster:

There were only 6 questions in this cluster and only one of them was directed to workers. However, the answers to this cluster, taken as a whole, correlated reasonably with OHS performance. Workplaces where people doing the work get good OHS information quickly are going to be workplaces where people making daily, practical decisions about work and OHS will be making excellent decisions about risk reduction. It has always been said about the IRS that one of its most basic principles is openness about OHS . In law, this has been referred to as the "right to know", more accurately described as various "duties to tell" on the part of the employer and supervisors. In general, we can say that people are making better decisions throughout the IRS if they have the right information about hazards and controls. Information is the "lubricant" for successful problem-solving in the system.

The "Beliefs" Cluster:

There were 34 questions in this relatively large cluster and, taken as a whole, the answers gave a reasonable correlation with OHS performance. The questions were distributed among all workplace parties, with a slight emphasis on workers and supervisors, and covered the following topics:

  1. whether a person believed he or she could go to the next level in the IRS without fear of a bad reaction;
  2. whether a person believed in a fundamental conflict between OHS and production or profitability;
  3. whether a person believed that OHS was a part of quality management;
  4. whether a person believed that following procedures is in conflict with production;
  5. whether a person viewed OHS goals as being beyond "mere regulatory compliance"; and
  6. whether a person believed the role of the JHSC to be one of "internal auditor".

While there are problems reliably measuring beliefs, and while there are debates as to the relative importance of beliefs and behaviours, we assume here that good indicators of the effectiveness of the IRS will be the presence of certain fundamental ideas in the minds of individuals in the workplace. In general, we can say that there was usually good variation within the worker responses, variation that followed the OHS performance of the mine. Questions that required self-assessment by supervisors and managers usually resulted in relatively homogeneous (and positive, from the standpoint of actions by the respondents) responses across mines, thus contributing little to the correlation of the cluster with OHS performance.

The "H&S Concerns & Responses" Cluster:

There were 16 questions in this cluster and they dealt with the way people interacted between levels in the organization with respect to handling OHS concerns and complaints. And six of these focussed on the interaction between the worker and the direct supervisor. Taken as a whole, the responses to this cluster showed only a fair correlation with OHS performance. There tended to be a fairly homogeneous and positive response on questions dealing with self-reporting by managers and, hence, there was no correlation with the answers to this sub-set of questions and OHS performance.

We expect the central core of the IRS to be a "filtering mechanism" so that many small issues can be dealt with quickly by workers and front-line supervisors. But issues involving greater resources or having cross-organizational impact will be the larger issues that need to move up the organization. Few reach the top. The many small issues will tend to be direct causes of accidents and exposures. The few large issues will tend to be the system errors that are the root causes of the direct causes.

The "Initiatives & Responses" Cluster:

There were 26 questions in this cluster. Only three were directed to workers, and five to front-line supervisors. Middle managers were emphasized with seven questions directed to them. There was no real correlation between the answers, taken as a whole, and OHS performance.

We asked a lot of questions about people taking initiative on OHS in this cluster. The "concerns" cluster, above, was about traditional concerns, reports and complaints. That is, traditionally, people see problems-defects, contraventions, dangers-and they deal with the problem or bring it to someone else. But the quality approach tells us that we want people to go beyond the negative and the traditional. We want people to think of ways of improving the processes they are involved with.

The "Accountability" Cluster:

There were eight questions aimed at all levels of the direct line of responsibility except the workers. Although one might have expected a strong correlation between OHS performance and the degree to which people in the IRS were formally held accountable for OHS activities, the answers to the this cluster, either taken singly or as a whole, did not track OHS performance.

The "Demonstrating Leadership" Cluster:

This cluster did not correlate with OHS performance. On the face of it, this runs counter to the accepted wisdom that OHS success depends on commitment from the top. Our view is that leadership at all levels is important, but it is up to the more senior people in particular to set the tone, establish the corporate culture and drive the IRS . There were 39 questions in this cluster, but only two of these were directed to workers.

The "IRS Issues & Responses" Cluster:

This cluster did not show a correlation with OHS performance. This was the biggest cluster of questions-45 in total. Only two of those questions were directed to workers (i.e., 4% of the questions). This is likely the primary explanation for the poor correlation.

There were several sub-themes in this cluster:

  • reporting up the responsibility chain regarding IRS performance;
  • responding to such reports;
  • the role of the JHSC and worker representative regarding the IRS ;
  • the role of the OHS professional regarding the IRS ;
  • the use of "IRS analysis"; and
  • the existence of a "truncated" version of the IRS .

Given its size and its sub-themes, this cluster would be better broken out into separate clusters.

The "Responsibility" Cluster:

This cluster consisted of only four questions and dealt with responsibility at the worker/front-line supervisor interface, and was disappointing in an ironic way. Who would have thought that the responsibility cluster concerning the Internal Responsibility System would show a lack of correlation with OHS performance? All we can say is that there were few questions involved in this cluster and so it had little scope and impact.

Previous | Next