WWTP Operator – The poor cousin? (Award winning paper from David Lee)

WWTP Operator – The poor cousin?
Opportunities for Better Control Room Design
David Lee MIChemE, CEng1*
1User Centered Design Services Inc., 48412 North Black Canyon Highway, PMB #99, New River, Arizona 85087, USA (*correspondence: dlee@mycontrolroom.com, 336.816.1419)

KEYWORDS
Control Room Design, Human Factors, ISO 11064, Ergonomics, Waste Water

ABSTRACT
It is an unfortunate fact of life that plant areas that are not ‘profit generating’ are not afforded the same level of investment or attention as their more glamorous counterparts. Often waste water treatment or effluent treatment plants are viewed in this light. This is most evident in the control rooms used to house individuals working in these areas. They are often old shacks attached to chemical or equipment storage areas that combine the control room, break room, laboratory, SCADA rack room, MCC and general storage area into a single room! If they are lucky there will be a rest room somewhere close by. This used to be, and to a certain extent, still is also true of outside operator field stations, left behind after the console operator and controls have been moved to the sparkling new remote centralized control room (CCR). However, there is often a safety case to give them something better, such as more ergonomic work stations, more screens for better process visibility and better lines of sight to the plant itself. So what should we be providing our personnel in these often remote areas of the facilities? Let’s not treat them as poor cousins making do with old stuff and hand me downs and provide an environment that supports the requirements of their jobs and adequately caters for their human needs.
This paper discusses opportunities for designers, engineers, operators and managers to learn from the experience gained in the design of similar multifunction buildings across a range of industries. In so doing the obvious parallel is drawn with the design of the central control room (CCR) and the use of standards such as ISO 11064.

Introduction
In my experience waste water treatment plant control rooms are treated as little more than a place for the operator to keep out of the weather. This seems to be true regardless of whether it is a standalone facility, such as for a local municipality or part of a much larger integrated facility such as a refinery. The significant improvements and lessons learned in the construction and operation of current best practice control rooms have not been leveraged in the design or redesign of control rooms for this often vital function.
This paper will attempt to present the state of the art today and how this can be applied to waste water control rooms.
The Historical Perspective
As we look around the various processing industries we find a great number of control rooms that were designed 15-20 years ago, and were considered state of the art at that time. These can often be characterized as cramped, dirty, noisy, dark and generally unpleasant places to work, but the operators, as adaptable as they are, make them function. They were often designed primarily to accommodate the technology and evolved along with the technology from panel board to DCS. From studies of these environments and the impact they had on operator performance, by organizations such as the Abnormal Situation Management (ASM) Consortium, we have a much better understanding today of how to do it right, or at least better.

State of the Art
So how do we tap into this knowledge? Unfortunately there is no one size fits all solution, every control room has its own unique requirements, from the tasks required to be performed in the room to the number of people required to perform them. However, there is a common factor that underlies all designs – the human operator. So rather than the previously prevalent technology based design, we need to look at a human centered design. To achieve this we look to the discipline of human factors engineering (HFE).
As we apply HFE techniques we quickly learn that a key element is Situation Awareness (SA). This can be defined as providing the operator with ability to quickly and accurately detect, diagnose and respond to changing plant conditions, especially those that may develop into abnormal operating conditions. In providing the operator with Situation Awareness we see there are several design elements that can be classified as SA Tools that have a direct impact on control room design:

• Human Machine Interface
• Alarm Management System
• Communications
• As these considerations have been applied to the design of the control room, along with advances in enabling technology, we see several major trends:
• Remote Centralized Control Rooms
• Dedicated console and dedicated field operators
• Theatre Style Designs
• Use of ergonomic furniture
• Removal of control system equipment from the control room
• High Performance Human Machine Interfaces
• Use of large screen displays
• Improved alarm management

Each of these is discussed below.
The trend to build remote centralized control rooms has, in part, been encouraged by the requirement to remove control rooms from hazardous plant areas. In so doing, advantages of improved communications and collaboration between functional areas has been facilitated by this new control room environment. However, it is more often the case than not, that certain functions on large facilities are not included within this new facility, one of which is often waste water treatment. The integration of this, and other areas that are based on higher levels of field operations, such as tank farms, has not been seen as desirable, especially when these areas are still manned by operators with inside and outside duties.

This control room trend has also required that control room operators are dedicated to that function, driven by a requirement for continued monitoring of the control system. It is often further driven by significant distances between plant and control room, making inside/outside operations impractical. Similarly dedicated field operators are now able to concentrate on outside activities, without having to worry about monitoring the control system.  We see that this works in many cases; however, there are some drawbacks to this arrangement, especially with remote control rooms. In these situations we are turning more and more to technology to address the loss of, for example, face to face communications between field and console operators.

The layout of the control room, with respect to the consoles, is also changing with the adoption of Theatre style designs. Many early control rooms adopted a functional design approach that resulted in consoles in circles leading to issues with noise, lighting and restricted collaboration. Theatre style layouts are characterized by all the consoles facing in one direction, typically onto a shared large screen display. This arrangement works well for environments where collaboration between consoles is vital to efficient plant operation.
Another component to the design of the control room is the use of console furniture that meets ergonomic requirements. No longer is it sufficient to introduce vendor standard furniture with excessive numbers of screens, often two high in design, with no adjustment, improper speaker positioning and no place to put paperwork and communications equipment. The use of properly designed and fit for purpose furniture is becoming more common, generally at a lower cost and with a smaller footprint, allowing savings in control room construction costs.

Technological advances have also allowed control system equipment, in particular the workstation CPU enclosure, to be removed from the control room, reducing the requirement for environmental controls for the hardware. This has the effect of further enabling console footprint to be reduced, reducing the need for raised computer flooring in the control room and facilitating easier access to hardware for maintenance activities such as upgrades. This remote mounting also reduces the equipment’s impact on control room temperature and noise levels.

The next set of improvements center more around the presentation of information to the operator. Human Machine Interfaces (HMI) have evolved, with the enabling technology, from the days of the stand-up panel board to today’s PC based systems. Unfortunately we did a very poor job of fully utilizing the potential of the technology, adopting poor practices that hindered, rather than helped the operator interact with his equipment. The next generation of HMI introduced the concept of process graphics, but based them on P&ID representations of the process. In order to see their entire process operators have requested more and more screens, and this had led to consoles with as many screens as 24 per operator! The ASM Consortium developed a set of principles, on which to base HMI design back in the 1990’s, but the adoption of these has been slow and inconsistent. Now we have moved on to the era of high performance HMI, further enhancing the design of not just the DCS graphics, but the entire console environment. This also embraces the increasing use of large screen displays as part of this working environment. However, as technology has improved, and the cost of large screen display hardware has plummeted; early adopters have done a poor job of taking full, or any, advantage of these devices. We often see control rooms with large screen displays that are unused by the operators, as they are seen to be providing them little, or no, benefit. Although the use by secondary users is a consideration, the primary use to provide operators with improved situation awareness by bringing back the big picture is poorly understood. More recent examples of well-designed systems should provide some much needed help for those wanting to use them.

The final important piece to the puzzle is the attention to correcting poor alarm management practices, still prevalent today. Again as we look back at the old panel boards there were very few alarms; today operators are faced with thousands of configured alarms, presented to them at rates that they cannot handle effectively. Standards and tools have come a long way in the last 10 years, and this is helping in addressing the performance of this very important operator tool. Ultimately, if we get everything else right the operator’s reliance on these systems, as a reactive prompt to action, should be significantly reduced. Instead a more ‘situationally aware’ operator should act before the alarm point is even reached.

How can we use this?
All this is great but how does it help us?
Well let’s start by making the statement that the principles employed in the design of these large fancy control rooms are equally as important in the design of a waste water control room, regardless of the size of facility or the number of operators housed there. Therefore we should be able to make use of the same set of standards in the design process.

First off we need to understand that no two control rooms are the same, so following the ISO 11064 approach to perform a functional task analysis as a foundational step in the design is essential. Doing a thorough job of this is obviously important, but perhaps even more so, is having everyone sign off on it. Changes at a later stage in the design process become more and more difficult and ultimately more expensive, or lead to a compromised solution.
Once we understand what is required we then start to look at where to best locate the building, and the consequences of that choice. For example are there risks associated with the process or security? Security is becoming a hot topic in industry, and may require a building vulnerability assessment to make sure all security threats are identified and addressed. An example of this is, ‘will there be a requirement for card key access?’ Most waste water control rooms are left unattended for large periods of time, and it is typical for contractors, and others, to wander in looking for the operator. At this time systems and equipment are vulnerable and this should be a significant concern.

Next we address the building layout; again it is typical for this type of control room to be a single room with perhaps a restroom off of it. However, we need to seriously consider separating out building functions such as HVAC and electrical, to address safety and security concerns. Having operators in the same room as MCC or Switchgear equipment, and their potential for arc flash, is obviously a personal safety concern. Similarly DCS or SCADA equipment, especially termination panels and servers, should be separate and secure. Then we need to look at things like the lab equipment and control of emissions from them, if appropriate, and even a proper look at the storage of lab chemicals and supplies. This may require a separate lockable cabinet or cupboard. Even little things like where to hang rain coats and PPE need to be carefully considered.
With respect to the location of the control console, we obviously need to start thinking about the technology and its application before we can tie this down. Looking at some of the points made above can help guide our thinking:

•  Remote Centralized Control Rooms – we need to consider how we communicate to other control rooms and operators. Even little things like staying in touch with morning meetings and sharing performance data can be facilitated by, for example, the use of collaborative tools such as Smart boards.
• Dedicated console and dedicated field operators – we need to consider how to staff the area, however, experience would suggest dedicated console and field operators are rare for this type of operation
• Theatre Style Designs – although at first you would think this doesn’t make sense some of the design features such as the use of large screen displays and the view they provide for secondary users may be important. Consider perhaps the location of exterior windows to provide a view on to the display for the operator to get a quick overview without entering the control room, or similarly for supervision.
• Use of ergonomic furniture – this is a given today, and the use of sit/stand furniture for an inside/outside operator can be extremely useful when they are busy.
• Removal of control system equipment from the control room – this is a small consideration but when consideration for security and environmental control is made this may be extremely beneficial.
• High Performance Human Machine Interfaces – essential if we are to truly move to an environment that allows the operator to have the best situation awareness, especially as in an inside/outside mode of operation, spending a long time looking for information can be a huge problem.
• Use of large screen displays – not something we have in too many waste water control rooms, but consider the benefits of a simple overview that can be used by the various primary and secondary users to get a situation update very rapidly. The cost should no longer be a barrier.
• Improved alarm management – should be a given and designed to complement the HMI design.

Following on from the design of the building, and the layout of the building, comes the detailed design of the console and the SCADA system, which rely heavily on the considerations above.
However, the design process does not end here; the documentation of the decisions and agreement with the design is essential before handing over to the building architect and subsequently the builder. Many times we have seen misinterpretations of the conceptual design into the final product, leading to a compromised control room design. The use of verification and validation techniques and methodologies such as SINTEF’s CRIOP® should be employed throughout the design lifecycle and indeed in to the operational life of the control room.
Summary
In summary, we have made many basic errors in the past, but across the various industries we have learnt and gradually improved, creating standards and best practices to guide us, and future designers. It is important that even though the design of a ‘simple’ waste water control room seems straightforward, we need to spend some time and follow the right process to make sure our initial vision is turned into reality. Consideration of some of the latest industry trends may surprise some in their applicability to this particular design problem. Whatever anyone says, waste water is as important as any process unit and we need to make sure the operator has the right tools and environment to ensure safe and efficient operation of this important asset.
—-
List of Acronyms:
ASM    Abnormal Situation Management
CRIOP®    Crisis Intervention and Operability
EEMUA    Engineering Equipment and Material and Users’ Association
HFE    Human Factors Engineering
HMI    Human Machine Interface
ISA    International Society of Automation
ISO    International Standards Organization
MCC    Motor Control Center
SINTEF    Stiftelsen For Industriell og Teknisk Forskning

Main Author Name:
David Lee, MIChemE, CEng.
David is a Chemical Engineer from the UK. David spent 22 years in the Chemical industry as an Engineer and Manager before becoming a human factors consultant with User Centered Design Services in 2007. Since that time he has been involved in well over 100 projects, across multiple industry segments across the globe, working with clients to address all aspects of operator situation awareness.
David is a senior member of ISA, ChemPID Safety and Human Factors Chair and VP of his local Section. David is also a clause leader and voting member on the ISA 101 committee and member of ISA 18 and ISA 106 committees.