Conclusions and Recommendations

The data clearly demonstrate that answers to the questionnaires can be related to a mine's safety performance. The Question 1A and 1B combination, particularly as answered by mine workers, shows promise as an easily administered and interpreted tool that can give a quick assessment of the IRS at a particular mine.

In addition, if it is accepted that the questionnaires do provide a measure of how the IRS functions in the mine (and we have no reason to suspect that they do not), then one can conclude from the correlations that better performance in the IRS will lead to a better safety record. If the IRS is working well, the organization has fewer accidents, as measured by medical aids (and to a lesser degree, as measured by lost time).

Further Modifications of the Audit Tool

The time constraints for this project do not allow us the opportunity for another iteration of testing the audit tool. We do, however, suggest that this be done. In our question by question analysis of the trial audit questionnaires, we gained an impression of which questions appeared to work, and those that did not. We believe that the following types of modifications would give a better overall correlation with OHS performance, as well as improve the correlation of sub-groups and even individual questions.

Elimination of questions that tended to cause confusion
Decrease in the number of questions directed to senior management
Increase in questions to non-senior management about the beliefs and actions of senior management
Elimination of questions that had little contribution to the correlation with OHS performance
Change in wording of some questions so as to be more specific or more action oriented
Re-classification of a small number of questions in the thematic clusters of questions used to focus in on the causes of IRS performance
A change in the number and nature of the thematic clusters of questions The IRS Description

The description of the IRS is useful in and of itself. A single, authoritative description will reduce misunderstandings about the IRS . Too often in the past one got the sense that people were discussing the IRS while assuming that there was a shared understanding. We are better able to strengthen the IRS when there is a consistency of understanding. To be a little more forceful about it, the description of the IRS , reached as it was with the assistance of the IRS Steering Committee, is perhaps the most important outcome of this project.

The IRS Analysis

There is an outcome of this project that is conceptually part way between the IRS description and the IRS audit tool. That is, the IRS model can be used to analyze IRS situations, problems and opportunities in new ways. Most health and safety problems can be analyzed in terms of the root problems with the "people factors" in the organization. We call this "IRS analysis".

IRS analysis puts the people of the organization front and centre in the analysis.
Everyone should be engaged in IRS analysis to some degree. The health of the IRS itself should be a major concern of mid to senior management. The manager should be asking him or herself: "What authority or resources did they lack that they felt that they could not solve this problem?" and "If there was a lack of information, authority or resources, why was the problem not passed on up to a higher level of the organization?"
The IRS professional and the JHSC members would do a much better job of handling issues if they became accustomed to thinking in IRS terms. IRS analysis would take us beyond the question "is this work safe?", to "why wasn't this hazard dealt with by the people responsible within the day to day routine?"
Similarly, while we made no direct observations, it follows that a MOL inspector has greater success in obtaining compliance if he or she goes beyond compliance into the IRS . It is not that people would be prosecuted or be given orders expressly for "failures within the IRS ", but rather IRS analysis would locate individuals needing external stimulation or stimulation from the higher levels of the IRS .

The IRS Audit

We have shown very strong support for the principle that if the IRS is working well an organization will have fewer accidents. The IRS audit is a tool for measuring how well the IRS is working. But the IRS audit does more than that. It also enables the user to find out why the IRS is working the way it is, and thus provides some guidance as to what actions might be taken to improve the IRS .

It should be noted that the predictive value of the audit tool works even if there is no actual use of the tool. All one needs to do is identify the organizations with poor medical aid and lost time statistics and you have identified the organizations with weak IRS performance. There is a temptation to say that this is merely stating the obvious, but it must be recalled that this study showed the relationship between the "end of stream" statistics and IRS performance. The statistics predict which organizations need the more intense analysis based on the full set of questions--the analysis focussing on why the IRS is performing poorly.

It is vital that users of the audit tool do not rest with the final "scores", but use the tool to probe deep into the organization to see why the numbers turned out the way they did. This analysis is the most important use of the audit tool, particularly for the people within the workplace.

The main report makes a whole series of observations and comments regarding the administration of the audit tool. The following topics are covered:

Proper modification of the audit tool;
The need for a single official custodian of the audit tool;
Tinkering with analysis algorithms contained in an Excel spreadsheet, the electronic version of which formed part of the report;
Distribution of questions or model answers;
How to refresh the questions;
Time taken to administer the Audit;
Audit Frequency;
Using the audit results;
Evidence for answers;
Fear of a Poor "Score";
Sampling Strategy;
The cost factor
Ensuring that groups of respondents are matched;
Ensuring that all shifts are sampled;
Education and training for consistency of use;
Internal auditors;
External auditors; and
Following-up on the Audits.

This list is not intended to be exhaustive, but rather serves to illustrate many of the considerations that will need to be addressed before an audit tool is put into general use.

Postscript

The IRS Steering Committee reviewed the report in its "draft-final" form and discussed its contents and recommendations with us at a meeting on 18 August 2000. Subsequent to that meeting, and based on our recommendations on how the questionnaires might be improved, the committee asked us to make revisions to the questionnaires administered during the trial audit of the six mines. This was done, and revised questionnaires are appended to the main report. However, the Committee did not adopt all the recommendations we made for questionnaire change. Further, while we feel that the limited changes should lead to questionnaires that produce responses giving a better overall correlation with IRS performance, there is no way (short of conducting additional mine audits) of verifying this. This is because of the new questions that have been added.

The report spent considerable effort in analyzing, on a question by question basis, the questionnaires. This analysis showed that there were problems in both the way some of the trial-audit questions were asked, and how they were grouped and weighted to analyze IRS functions. We feel that a more focussed analysis of the considerable wealth of data collected in the trial audit, than was permitted by the time and resources available to us in this project, would considerably improve the audit tool as it now stands. Thus we caution that, while the Excel spreadsheet (which, in its electronic form, is part of this report) was useful in helping the analyses reported here, there are problems in the underlying algorithms. Accordingly, if the spreadsheet is used at all in its present form, its results need to be treated with due circumspection.