In 2007, DuPont undertook an internal safety improvement effort. As we began, then CEO Chad Holliday posed a very important question: At the end of the study will you be able to tell me why one plant site has outstanding safety performance and another in the same business has poor performance? In the past, we had never been able to quantify – let alone answer – this question.
Twelve years ago, DuPont safety and consulting professionals began to answer this question by adapting the Safety Culture Improvement Process, originally developed by Dr. James Stewart, to understand employee perceptions about their safety management systems, both at the individual and organizational level1. The goal of the Survey was – and still is – to evaluate employees’ perceptions of their safety program and identify behaviors, attitudes, and other factors that can derail a safety program. Our Safety Perception Survey benchmarking data spans the globe, with more than 632,000 responses covering 96 industries, 41 countries, and more than 3,383 locations
However, a critical issue we had been experiencing was insufficient data from truly world-class plant sites. The Survey database predominantly represented companies needing to improve their safety performance. Our new internal safety improvement effort would now allow us to analyze data from plants that had been demonstrating world-class performance for several years, as well as data from plants wanting to improve. When we benchmarked data from DuPont plants that wanted to improve, their performance was already better than comparable companies already in our database.
As a result, we believed our database held a key to understanding the subtle differences between world-class safety performance and mediocre performance. After methodical analysis and applying decades of our own safety consulting experience to this extensive database, we unlocked a key to understanding the subtle, yet critical differences between the two.
This paper will explain how we used this database, plus information derived from our many years of consulting experience, to gain better insights into what makes a world-class company.
We examined results from the DuPont Safety Perception Survey – collected from dozens of organizations – and looked at how the safety culture varied between organizations with poorer safety performance (by DuPont standards) and those with excellent performance. We used the OSHA Total Recordable Injury Rate as our measure of safety performance. We also compared safety performance at DuPont, and results from our Safety Perception Survey, to the results of qualitative assessments conducted on-site by DuPont Safety Resources consultants.
DuPont Safety Resources has been using the DuPont Safety Perception Survey since 1999 to measure the strength of the safety culture in organizations. The survey consists of 24 multiple-choice questions that measure an organization’s safety culture across the three elements of safety management: Leadership, Structure, and Processes and Actions. (See Appendix A for a description of these three elements and a list of the question topics). Organizations can benchmark their results from each Survey question to companies with excellent or poor safety performance.
The Survey captures perceptions and attitudes toward safety that are held by a cross-section of employees, supervisors, managers, professionals and hourly workers in the organization. Data is compared across departments, levels and responsibilities.
The Survey tests whether safety is a core value held by management and whether that message is effectively transferred to the worker on the floor. It will expose whether safety rules are followed and whether consequences and recognition are consistent across the organization. It will examine whether or not employees and managers are involved in audits, incident investigations and safety meetings. It will highlight the importance of safety versus quality, schedule and cost. And, finally, the Survey will examine attitudes about off-the-job safety.
Because the Survey involves every level of the organization, it provides leaders with an objective, data-driven process for understanding safety culture and benchmarking it. Responses are compared within the organization and to responses from employees in external benchmark companies. The benchmark companies include those with superior safety performance (“best”) as well as those with very poor safety performance (“worst”).
Benchmarking through the use of the Survey can identify a company’s own best and worst sites and make comparisons to uncover critical truths. For example, how does one site go years without recordable incidents while others have injuries routinely?
Finding the perceptual differences between best sites and worst sites can be difficult when reviewing safety performance data at face value. But by digging deeper into the values and beliefs at each site, some clues can be found in the Survey responses and comments.
Are Safety Culture and Safety Performance Related?
To determine whether safety culture, as measured by the DuPont Survey, is related to safety performance, as measured by OSHA’s Total Recordable Injury rate, we calculated a value called Relative Culture Strength (RCS), which is based on responses to all questions on every Survey from a given organization. We were thus able to compare Relative Culture Strength and OSHA Total Recordable Injury Rates from many different organizations to look for relationships.
The dataset we used consisted of over 169,000 Survey respondents at 41 companies and at more than 1,100 individual locations. More than 31,000 of the Surveys were from locations outside North America; over 24,000 were from Canada. The Surveys were conducted from 2002 through 2011. The primary industries surveyed included aircraft manufacturing, chemicals, electric utilities, engineering/construction, food, mining, oil refining, paper, steel and transportation.
These 169,000 Surveys represent less than one third of the total number of Surveys in the DuPont Safety Perception Survey database; however, for the other Surveys no corresponding OSHA Total Recordable Injury Rates were available and many companies outside the U.S. do not track this metric.
We then split the data into 346 “organizations,” which we defined as the smallest entity for which we had safety performance data. This was either 1) a site facility within a company, 2) a business, geographic or functional division within a company, or 3) the entire company.
The number of Survey respondents per organization ranged from 30 to over 20,000, with a median of 489 respondents. Ten organizations have taken the DuPont Safety Perception Survey four or more times. 94 additional organizations have taken the Survey at least two times.
The OSHA Total Recordable Injury Rates by calendar year were provided by the companies that were surveyed. We used an average of rates from the three consecutive years prior to the Survey to represent the safety performance of an organization. When a Survey was launched in the last six months of a year, that year’s rate was included in the three-year average. When a Survey was launched in the first six months of a year, we used the prior three years of rate data.
We compared the three-year average Total Recordable Injury Rate to the overall RCS (see Figure 1 on the PDF).
In general, greater cultural strength relates to a lower injury rate. We noticed that between overall RCSs of 40 and 80, the three-year total Recordable Injury Rate varied widely. So, we decided to divide the data into four groups according to overall RCS (See Figure 2 on the PDF).
The 346 organizations were divided into four groups according to their overall RCS. Each dot in the charts below represents a single organization. The median 3-year average Recordable Injury Rate for each group is plotted as a horizontal line. Groups with a greater Relative Culture Strength had a lower Total Recordable Injury Rate and a “tighter” injury rate distribution.
We saw that both the median and mean injury rates decreased as a group’s overall RCS increased. We also saw that variability among the injury rates decreased as culture strength increased. One of the beliefs that DuPont held was that Leadership was the key dimension that enabled outstanding Safety performance. And while our conclusions validated the notion that Leadership is key, we also concluded that the other two dimensions – Structure and Processes and Actions – are just as important to organizations that demonstrate outstanding performance.
A strong Leadership commitment facilitates implementation of a strong safety management system. The Structure dimension allows organizations to implement the changes necessary, and the Processes and Actions dimension defines what actions need to be taken to understand the effectiveness of the process.
Figure 3 on the PDF shows the similarities when you calculate RCS for each of the dimensions. We consider an organization to have a world-class safety culture when the RCS for each of the three dimensions is greater than 80
Finally, we summarized our findings in a DuPont Bradley Curve chart, shown in Figure 4 on the PDF.
This chart shows four categories of companies in terms of recordable injury rate versus RCS:
When you then incorporate RCS results into the DuPont Bradley Curve chart, you get an excellent picture of how Relative Culture Strength improves as a company moves from reactive to interdependent. This is shown in Figure 5 on the PDF.
We believe that when an organization has a weaker safety culture, it may “get lucky” and have one or more years of relatively good safety performance but that performance is less sustainable than in an organization with a stronger safety culture. Organizations with good safety performance but a weak safety culture are at larger risk of experiencing greater rate of injuries in the future than are organizations with a strong safety culture.
How quickly does safety culture change? Organizations need to understand that change has to be sustainable, and sustainability takes time. You can change the climate in an organization very quickly but culture can require greater time to change. For this reason, we do not believe you should take the Survey more frequently than every two or three years. Our research shows that safety improvements in organization performance over shorter periods generally produce little or no significant change in RCS. It is our belief that sustainable safety culture change comes through improving Relative Culture Strength and this does not happen quickly.
Many Benefits of Benchmarking
Benchmarking through the use of the Survey is also an effective process for assessing and catalyzing safety culture improvement within organizations at any portion of their journey to world-class safety. And it’s a focused way to rejuvenate an existing safety program through rapid penetration into all levels of an organization.
When making comparisons that use validated and respected global data, employees are more likely to accept the data rather than thinking it has been “manufactured” to make a point. Employees also begin to see themselves as part of a larger picture and become sensitive to the external environment.
In addition, corporate mind-sets will shift from relative complacency to a strong sense of urgency for ongoing improvement as they become aware of how the company is performing compared to its peers. Employees support the process because they have a role in establishing performance targets and focusing resources.
Finally, areas that need improvement are prioritized, and there can be a sharing of best practices between benchmarking partners.
This study has validated the DuPont Safety Perception Survey as a statistically based process utilizing benchmarking which is focused on the key drivers correlated to injury reduction. Our development of the Relative Culture Strength metric has allowed us to take qualitative data from the Survey and change it to quantifiable data to measure a client’s progress on their journey toward world-class safety.
In developing this metric we have been able to validate data from the DuPont Safety Perception Survey and have shown a correlation to both assessment scores and safety performance. We have confirmed improvements in safety performance and have proven that organizations need to focus on all three dimensions – Leadership, Structure, and Processes and Actions. Prior to this study, we believed that a relationship existed, but we had not quantified it; now we have confirmed this hypothesis with data.
In addition, by integrating Relative Culture Strength onto the DuPont Bradley Curve, it has allowed us to benchmark safety culture within a company, an industry and across different industries. By analyzing companies that have taken the Survey more than once, we have concluded that safety performance can improve year over year; however, it takes several years to change and sustain an improvement in safety culture. As such, this study has shown that understanding one’s safety culture can provide an insight into the company’s overall safety process, which is different from other traditional indicators. As a result, this indicator is a key element that each of us must come to know and understand in order to build an organization which demonstrates sustainable world-class safety performance.