The objective of this project was to provide a platform for deploying a standard set of security controls for Utility Monitoring Control System/Energy Management and Control Systems (EMCS), including building automation, micro-grid control, smart metering, load sensors, and other energy management Platform Information Technology systems installed in Department of Defense (DoD) buildings and critical infrastructure. The security platform and work within this project have enabled patching and continuous monitoring of these systems to support simplifying the security posture and management of assets. The schedule of work was on time and on budget and made possible through the support of the site team and FoxGuard. The completion of this project brings an enhancement to the current system that includes patch management and vulnerability management support through the use of the Sentrigard Enterprise Appliance.

FoxGuard Solutions provided a security platform to assist with continuous monitoring of the EMCS, as well as patch management of the virtual infrastructure that supports the environment. The security platform can help detect changes to device/application configurations, changes to system/device baseline configurations, and network anomalies such as new devices or new network communication paths. The security platform also includes centralized log collection to collect events from various devices in one location. Current practices in the industry often include a manual review of device/application configurations and periodic review of log files and event logs. These technologies reduce manual effort and make it easier to detect when something has changed in the environment.

The patch management portion of the solution allows for centralized scanning and deployment of operating systems and many third-party patches. This is along with FoxGuard Solutions’ Patch Availability Report and Patch Binary Acquisition service: subscription-based services that provide monthly reporting on patch updates and monthly delivery of patch update files, which make it easier for system operators to determine which patches are required in their environment and in many cases lead to deployment of those patches to all systems at once.

The demonstrated security platform provided features to facilitate configuration and baseline change detection, network anomaly detection, centralized log collection, and patch management. During the demonstration, these technologies were successfully employed to help reduce manual efforts and provide additional insight into the state of the Energy Management and Control Systems network, as well as keep systems up to date with the latest security patches.

With the Sentrigard in place, the Closed Restricted Network (CRN) 2.0 is a subset of operational technology (OT) and information technology components found on the Facility-related Control Systems (FRCS) production network. CRN 2.0 consists of a ESXI server that contains the same virtual machines that are part of the production network. A single software defined network switch provides network connectivity between devices. A JACE 8100 building automation controller gets data from a single EasyIO which is monitoring an air conditioning system in building 361. These devices provide the same network protocols found on the production network. One electric meter is also part of the CRN 2.0 network that represents the electric meters and protocols. The components and data from a previous ESTCP microgrid project have been integrated into the network. All these systems provide users with real data from actual hardware in a controlled environment for OT training, cybersecurity testing, and software/firmware testing. Using the Sentrigard system administrators must apply any software or firmware patches to the CRN 2.0 components before any changes are made to the production network. New hardware and software can be added to the system for integration testing, product comparisons, and vulnerability analysis. The network is designed to scale up or down to meet user requirements. CRN 2.0 can be accessed remotely to allow users to conduct testing at a significant cost savings. CRN 2.0 was recently used by a Department of Energy project focused on OT cybersecurity. The stakeholders remotely connected to the CRN 2.0 network and conducted all required testing, experiments, and analysis. Future efforts for CRN 2.0 include extending the software defined network to a contractor facility allowing for software sustainment activities (updates and patches) to be conducted remotely. This will significantly reduce the amount of costs required to maintain a system’s authority to operate. Another future effort will be to push the building automation system data to a DoD approved cloud for analysis.

No procurement issues were encountered for implementation of this technology solution. The Sentrigard server uses standard commercial off-the-shelf hardware and a combination of readily available open-source and commercial software components. The network anomaly detection module is sourced from a specific vendor and uses hardware provided by that vendor. As of the time of writing this report, global supply chain issues relating to the manufacturing of chips are certainly a procurement concern, but once those issues have been addressed the required hardware should be readily available.

During implementation, issues were discovered with the software-defined networking configuration already present within the demonstration environment and the lack of standard support for mirroring traffic (as compared to traditional switching hardware). For many switch vendors, mirroring traffic is well documented and only takes a handful of commands to implement. The software-defined networking technology in use within the demonstration environment required additional effort to come up with a configuration that provided the same level of traffic visibility required for the network anomaly detection software to function as intended. Prior to the implementation of future instances of this technology, a more thorough review of the networking configuration should be completed to determine the capabilities of the existing network infrastructure to support mirroring traffic. In addition, subject matter experts for the network infrastructure should be consulted early on to ensure they are ready to assist and test any changes early in the process of implementation.

Existing configurations for the host-based security system (HBSS) were preventing the baseline configuration module from functioning as intended. The HBSS configuration had to be adjusted to allow the necessary traffic through the host-based firewall on the Windows-based endpoints. This issue is not a major concern for future implementations of the technology, and simply requires changes to the HBSS configuration to allow the required traffic.