The use of Artificial Intelligence, IoT and IIoT, is often shrouded in a mist of confusion and theories for use cases. Practical applications and smart uses of technology at a real-world level are refreshing and demonstrate true innovation.
A collaboration of industry professionals and university students.
VTScada is a powerful, highly scalable SCADA software system that is used in thousands of applications globally, ranging from small, simple monitoring applications of a few I/O points up to systems with millions of points. Trihedral, the company behind VTScada, is an automation software company with over 33 years’ experience in the industry. The company has its headquarters in Bedford, Nova Scotia, and has office locations in ten different locations across Canada, the United States, and abroad. Their software has been used in many interesting applications and one such innovative application involves the use of drones.
BettsM Controls, Inc. is a technology-oriented product development company, premier solution provider and distributor of VTScada, based in Calgary, Alberta, Canada. The company provides leading edge hardware and software solutions across the water, oil, gas and power sectors.
The company was invited to collaborate on a Capstone Project with fourth year students from the University of Calgary, to test the skills that they had learned throughout their college journey.
Mervyn Betts, President of BettsM Controls Inc, worked with the students to define the scope of the project. This came about through meetings with the students and their Academic Advisor, Mohammad Moshirpour. The team zoned in on a real-world problem from which came the project concept. Drones are being used for a wide range of commercial uses such as photography, aerial reconnaissance, security, search and rescue, mapping and surveying along with many other applications. This led the team to discuss possible applications using this technology that were not obvious in the public domain.
How do you find a moment in hundreds of hours of footage?
Even in non-aerial applications, video capture and processing is a storage and time hungry requirement. A number of strategies can be applied to carry this out including live viewing of footage by personnel as it happens. This is massively labor intensive since it requires a person or persons to review every second of the footage and then make decisions based on what is seen in real time. Alternatively, this can be done after the fact where personnel review all the footage and identify the anomalies of interest that have occurred. Still not an efficient use of time.
The use of vision systems along with analytics engines are a major step forward but still require quite a lot of viewing of footage and then tuning of the algorithms.
The solution was alarming.
The team came up with a novel way of doing the latter, through a combination of strategies based around a DJA Phantom 3 drone. “What we want to see is when the video analytics detect something that’s abnormal or different, it can trigger an alarm in VTScada. With that timestamp we know exactly when the event occurred, and we can go find that on the video if we’d like to watch it again,” said Mr. Betts.
The use of vision systems in processing video is not a new idea by itself and requires teaching and tuning of sets of algorithms. This usually requires a lot of human intervention to physically watch for missed reads and good reads based on the ‘anomaly’ of interest. Then the vision system analytics are further refined to get to the point of a high probability of good reads. This is still very time consuming in any capacity since it requires the data analyst to wade through potentially hours’ or even weeks’ worth of footage to identify whatever is of interest. The second factor is that the drone flight software is handled by another dedicated application. Trying to get the vision system analytics software to interact with the flight control software is often a challenge. Teaching a vision system requires a coordinated approach between the vision system and the flight control software in order to reach a stable, high probability factor of good reads and efficient data storage.
With the project problem statement set, the team identified a suitable drone, onboard camera and video analytics engine, and lastly, flight controller hardware and firmware.
Three systems brought together by JSON.
JSON (JavaScript Object Notation) was the protocol of choice as a lightweight, efficient means to interact and provide data interchange between the video analytics system and the flight control software. This a familiar protocol to anyone in the IoT / IIoT sector allowing connectivity to a broad range of products on the marketplace.
The team next identified VTScada as the central software of choice, as it had a built-in JSON driver and provided the capability to be the flight control software, advanced data historian and alarm system capable of interacting with all the systems onboard the drone. VTScada’s highly intuitive, advanced scalable software, allowed the team to quickly build a flight control interface that allowed the drone to fly a predefined route right from takeoff, along waypoints, searching for ‘anomalies’ that were identified by the video analytics system, that were then timestamped in-sync with the video footage being captured.
An intuitive SCADA platform.
Mr. Betts was impressed with the ease with which the students built this interface using VTScada, “We did this project on VTScadaLIGHT which is a 50-tag free license providing a thin client. It allows you to use all the functionality of VTScada. It allows ten free licenses to anybody or any corporation that wants to make use of it. It didn’t take the students very long to integrate in VTScada. They did most of the integration in an afternoon. Really most of their effort was spent developing the video analytic programs and things like that as well as developing the automation on the drone,” said Mr. Betts.
In a video prepared by the students, BettsM Controls Inc. and VTScada by Trihedral, viewers can see how the entire system operates in unison. The video is embedded after this article.
It shows the VTScada flight control interface in the left window and the view from the onboard camera of the drone in a forward-looking attitude in the right. The map in the flight control interface shows the exact position of where the drone is at any given moment via the ‘slippy’ maps window. Flight and control data from the drone, like battery capacity, ground speed, altitude, heading location, flight pattern and guidance modes are also displayed here. These flight patterns are entered and selected here, allowing the vehicle to fly completely autonomously using any given number of search patterns, looking for ‘anomalies’ before returning to base.
“What it’s searching for, and this is all automated, is a blue square. It takes off and it’s going to go to the first waypoint and all the way along the way it is searching. The video analytics are running. As it proceeds here it will come to the first waypoint and it will not find what it’s looking for. Then, it drops out to circle at a greater distance. Once that occurs, we will move on to the next waypoint. So, here we found the item and an alarm just came in. You can see it on the left-hand side in VTScada telling us that we actually found the blue square. With that timestamp we can now go back to the video and know exactly where to look”, explains Mr. Betts. In the video, he further goes on to reinforce the point that their intention was, “to get the video analytics to happen right on the camera or in this case right on the drone with the camera. The reason that’s really interesting is because we use a thousandth to a millionth of the video that we actually take so when you look at video that’s happening at like six frames per second or more, all of that data gets stored and put someplace but nobody ever looks at it because they had no reason to look at it. The only time they need to look at it is when an event occurs.”
A solution for numerous industries.
He later explains that there are many immediate use cases for this application like oil and gas pipeline leak detection for instance. In the water and wastewater sector, everything from algal bloom identification to waterway pollutant detection after storm events where combined sewer overflows (CSOs) are in use. He also cites many wildlife tracking/marine or fisheries use cases. He says that he is looking forward “to working with other video analytic companies.”
Mr. Betts wishes to thank the University of Calgary Team of Dominic Hul, Brandon Lee, Nick Norrie, Veronica Eaton, Szymon Czarnota and Academic Advisor, Mr. Mohammad Moshirpour.
VTScada by Trihedral is proud to be a partner in this project. For further information about this application, please contact BettsM Controls Inc. at www.bettsm.com.For further information on applications for VTScada or to download a FREE full 50-point development license of VTScadaLight, please visit our website at www.vtscada.com or call Toll-free: 1 (800) 463-2783 (North America). Follow us on LinkedIn.