A Utility’s Fifteen-Year Migration to a Single, Non-Proprietary Monitoring and Control System
Most people imagine a SCADA system as single computer interface overseeing a utility’s entire remote infrastructure. The reality more often resembles the system found in Ocala, Florida, which was developed over multiple years by a variety of system integrators. The budget for the original system only paid for remote telemetry units (RTUs) with proprietary HMI software at their water and wastewater treatment plants. Later, a different integrator installed another brand of software with programmable logic controllers (PLCs) in their lift station network. A third combination of hardware and software was added to their irrigation sites. A limited yearly capital budget meant that the utility had to continually choose between more bolt-on components and the steep cost of starting over with a single system. This resulted in a variety of issues.
- It was expensive to train staff on three different applications
- Each system could only see part of the process
- There was no central historical database for reporting and analysis
- Many servers were required to ensure redundant servers and services
- Multiple sets of software renewal fees and support contracts
In 1995, the Ocala began a fifteen-year transformation to a unified SCADA application. Once they found a way to keep all their PLCs and RTUs they were able to convert one system at a time so that each project did not exceed their yearly budget. The final system took advantage of a new approach to distributed historical data management that allowed them to lower costs by reducing the number of manned hours required at some of their remote sites.
What is SCADA?
Every SCADA application is different. However, most share the following core elements:
- Monitoring & control devices to collect data and control equipment at remote sites (e.g. PLCs, RTUs, and pump controllers)
- A communication network to relay commands and bring back process data
- Human machine interface (HMI) software to provide a graphic representation of the process
- A database of I/O points (tags) that defines important objects in the system
- A database of historical process information logged by the system
- A reporting package to generate scheduled or ad-hoc reports
- An alarm management system to alert on-call personnel and emergency responders to system problems
1995 – Three Disparate Systems
To meet the ever-growing needs of their sixty-five thousand customers, the Ocala Water & Sewer Department operates a twenty-five million GPD water plant, three water reclamation facilities with total capacity of thirteen million GPD, 125 lift stations, approximately two thousand acres of spray irrigation systems, and fifteen other assorted water and wastewater sites.
According to Darryl Muse, Ocala’s Deputy Director of Utility Systems, the utility became increasingly concerned about the rising cost of maintaining three SCADA systems. “Primarily, the issue was support,” says Muse. “We had people who specialized in the water side, the wastewater side, and the lift stations. We had three groups trying to support the automation side of the house.” Also, none of the applications could see the whole system creating blind spots for critical alarms and making it difficult to respond to all issues in a timely fashion. Additionally, the lack of a single historical database complicated the process of generating reliable system-wide reports and trends.
Since none of their software packages could communicate with all of their RTUs and PLCs, they could not simply standardize on one of those packages. “One of our biggest issues was the proprietary nature of our software packages,” says Muse. “We had to use specific hardware with each of them. It became problematic especially when we started to experience significant lead times to get equipment.” The limited yearly capital budget also prevented them from standardizing on one of their brands of remote hardware.
2001 – Step-by-Step Migration
To begin this transformation, the utility needed to find an open architecture software solution. “We had come across VTScada™ when talking to our neighbor, Gainesville Regional Utilities who were in conversation with [Trihedral, developers of the software]. We were searching for something we could afford that wasn’t proprietary and wouldn’t involve significant hardware expenditure. VTScada offered all of that. ”
“We initially moved just the lift stations onto VTScada,” recalls Muse. “On the day we installed it, we were literally polling all our sites within a few hours.” Rather than rebuild the tag database from scratch, a tag conversion utility allowed them to simply convert the existing database. With this, the software could automatically generate standard graphic displays for all data points. The new system no longer required third-party tools for critical components such as reports, trending, hot-backup, remote access, and alarm dialer since these are integrated into the software. These tools grow with the product as new versions of the product are released. However, the problems associated with supporting two SCADA systems remained.
2002 – A Single Unified SCADA System
After a year of running the SCADA applications side-by-side, the utility converted their water and wastewater plants as well. “After we proved that the new system was solid and reliable, everyone came by and saw what it could do. Once we had enough people involved in the process everyone wanted it at their plant. It wasn’t a hard sell at that point.” The software included graphic development tools that allowed them to create customized displays. “The flexibility in creating new screens that was a major plus for the treatment facilities,” recalls Muse. The reclamation facilities soon followed. Now authorized staff could monitor and control any remote site from a single interface and use integrated trending and reporting tools to analyse data from a central historical database (see image above). A redundant hot-backup server in the office provided server and database redundancy. A single set of security user accounts managed access to the whole application. The utility was also free to incorporate other brands of hardware in the future based on features and price.
2010 – A Better Approach to Historian Management
For eight years, the utility continued to leverage their historical data to identify problems and increase efficiency; leading to a new set of issues. Each site logged data over the network to a central historian in the main office which was backed up by an onsite redundant server. As a result, when the network was overloaded or offline, remote users could see local live data but could not access the historian. Worse, remote data collected during that period was lost creating gaps in the permanent record. Also, each remote site required a primary and redundant server in order to provide local fail-over requiring additional maintenance and software licensing.
In 2010, a new city-wide high-speed fiber network combined with new VTScada features inspired them to rethink their whole approach. “We wanted to ensure access to historical data at all our remote facilities where we have folks for sixteen hours a day,” says Muse. The new network allowed them to keep only one server at each location. Each one could take over for any other in the system. Each site also hosted its own local synchronized historical database with a central backup of all databases at the head office. This provides quick local access to live data with the benefits of a centralized database for reporting and analysis. This approach also reduced the server count by fifty percent increased the levels of server redundancy from two to six (see image right).
These were not the only savings from having distributed historians. “We eventually spoke to the Department of Environmental Protection about our wastewater plants,” says Muse. “We convinced them that we could un-man those sites for eight hours a day because the data was available offsite.” During those periods, these sites could be monitored and controlled from the water plant which was staffed twenty-four hours a day. “They were pleased with the availability of the data in case something went wrong on an off shift. They allowed us to go to un-manned shifts at three of our wastewater plants which was a major cost savings for us.”
Conclusion
The combination of time, good planning, and non-proprietary SCADA software allowed Ocala to transition to the unified system they needed without exceeding their yearly budget. “We have been pleased with the system. It’s flexible and robust enough that we can give our technicians some basic training on the things we need to do to support the system. We have been impressed that [Trihedral] have continued to grow the product from where it was fifteen years ago to what it looks like now.”
| Try It For Yourself Download the 90-day Trial | Download the Case Study as a PDF |
Images used with permission.