Vulnerable and extremely outdated technology was relied upon to keep the clients manufacturing lines running hour-by-hour. The plant’s production lines were running without backups and limited space on a day-to-day basis. The controls network team decided that the server and supporting IT infrastructure needed to upgrade to a virtualized server network complete with backup and replication services to eliminate the threat of a technology outage forcing downtime. Lack of dedicated IT personnel at the plant required outside support going forward for the solution.

Project Overview: Manufacturing Network Design & Support

STA partnered with a leading engineering firm to assist in the outline and execution of a new high-availability virtual network and server deployment to replace the clients outdated and underperforming infrastructure. Concerns were abundant about the current server deployment’s ability to be recovered in the event of both virtual server and host hardware failures.

The deployment plant ultimately consisted of:

  • 12+ virtual servers running multiple controls
  • Historian data server
  • In-batch software which controlled much of the plant operations remain in-service
  • High-speed data access for virtual servers
  • Continued maintenance plan for securing edge devices
  • 24/7/365 monitoring by STA to respond to and resolve critical issues

Technical Challenges

  • Space constraints on the current hardware combined with the large application footprint required both a host level and physical level redundant server backup
  • A new layout for the server hardware had to be engineered and integrated by STA into the plant’s updated fiber network to allow for backup processing and storage to happen at a separate end of the plant.
  • Critical systems in production mode needed to have a smooth transition to the new systems while lines were secured from each other during the process.

Solution

Working through past experience with other manufacturing plants, STA worked hand-in-hand with the internal Power and Controls Team to identify:

  • Each server’s processing and space requirements
  • Total host sizing
  • Internal support team’s knowledge
  • Recovery time objectives
  • A comprehensive strategy for the manufacturing network support & design
  • Ongoing 24/7/365 network and virtual server infrastructure monitoring though STA
  • Management and securing of edge networking devices used in backup, management and DR processes
  • Continued support of critical infrastructure through scheduled and customized maintenance

After reviewing hardware needs, STA specified the purchase of three new virtual host machines, the most appropriate virtual host software licensing packages, and a fully compatible virtual host backup & recovery solution. STA developed a full networking map of the final solution, ordered and installed the hardware, and worked in partnership with the internal Process and Controls Team to implement the solution successfully. On-going virtual network management services were requested by the client from STA.

 

Outcomes

  • Only the replacement of essential networking gear in the event of a full mid-plant server room loss due to fire or other catastrophes is required to get the plant lines up and running again. This is a result of the new server layout design and a third host designed with double capacity for the production servers.
  • If a physical host fails at the mid-plant server room the system is designed to transition production to virtual servers designated as DR hosts in a separate location in the plant. This was achieved by utilizing the new fiber network to incur little to no extra latency back to the production lines and other running servers.
  • Constant monitoring by a third-party (STA) saves on internal costs and allows for 24/7/365 response to any type of issue and critical infrastructure data and reliability.