Resilient Utility Infrastructure – Part 2: Construction and Operations

by Matt Woo, P.E., RCDD, LEED AP BD+C and Steve Grayson, P.E., LEED AP

This article is part of Wood Harbinger’s newsletter series.

Resilient infrastructure is a primary component of a sustainable built environment; that is, one that can withstand and bounce back from a compromise of or interruption of services and continue operating close to normally. Part 1 of this two-part article explored planning and design considerations for achieving resilient utility infrastructure. In Part 2, we tackle the challenges of making a resilient facility plan come to life in construction and operation.

Construction

Scopes of work, schedules, construction areas, and existing system capacities or configurations may not permit a remodel or new construction from taking place without first taking down or interrupting a portion of the existing service(s). For example, key components of the existing service(s) may be in the area of remodel or new construction and will need to be relocated or temporarily re-fed in order to facilitate the work. The standby and utility services may be the project itself and require detailed step-by-step construction phasing to provide uninterrupted day-to-day operations of the facilities. This is particularly challenging in a critical facility that must remain operational with minimal to no impact.

There are several important methods for maintaining continuous operations: construction phasing, providing temporary services, and cohesive service cutover. For all of them, highly coordinated planning and execution is the key to success.

Communications/staff training and construction phasing enable manageable portions of the facility to be taken out of service or temporarily providing alternate sources of service(s) while construction is performed. One must first determine scope of work, identify key components that interface with each other, identify affected loads, coordinate shutdown date(s)/time(s)/duration(s) with owner and facility personnel, coordinate cut-overs to temporary services and back to permanent services, and conduct testing, operational training, and commissioning of new system(s).

Construction in a critical care or other essential/sensitive operations facility may require specific construction utilities to be provided to perform scope of work for remodels and new construction.  These power, water, air, gas, ventilation, etc. utilities may be independent sources, so as not to disrupt or compromise existing utilities serving adjacent spaces.  Existing utilities in or adjacent to the remodel or new construction area may not be adequate, in which case new independent or larger capacity/volume utilities would need to be brought in to meet construction requirements.

Where re-feeding services from adjacent spaces isn’t feasible, temporary services must be brought in to maintain facility operations in a reduced or full capacity. Consideration should be given to upgrading the network of utilities that will both serve the immediate construction demands as well as provide more flexibility and maintainability for future operations and construction. Electrical services can be provided with generators, fuel cells, and/or uninterruptible power supplies (UPSs) for temporary power production along with developing grid-type distribution services. Mechanically, we can provide temporary compressors, chillers, pumps, boilers, air handlers, exhaust fans, cooling towers—really anything we design can be installed on the back of a truck bed or on a skid that can be lifted by a pallet jack to allow for fast installation of temporary systems.

The trick for making temporary systems easier to use brings us back to planning for resilience. With a stub-out for temporary connections added on the existing headers or ducts, a contractor doesn’t have to modify anything to get a temporary system up and running. This can be critical for a data center if they need supplemental cooling, or a manufacturing facility that can’t lose production time. Pre-planning allows for a temporary chiller to just plug in and meet the immediate need. Many installations are found to have limited loops and minimum valves due to budget constraints or value engineering efforts. However, in the long run, it always ends up costing significantly more in cash outlays and interruptions to day-to-day operations if such measure are eliminated.

Another consideration is having access space nearby the central plant or mechanical and electrical equipment rooms, where a truck or skid with a temporary generator, air handler, or other large device can park, and where distribution lines can be easily routed.

For example, when a chiller failed at one of Boeing’s plants, we worked with them to bring in a temporary chiller on a truck and installed about 100 feet of hose to connect to the system and kept their operations running at full capacity. This was possible because fittings and planning was already in place to accommodate this emergency.  In another example, we took an entire mechanical room out of service and provided all temporary services during a renovation of a medical and dental clinic that remained open during new construction.

Utility cutovers, whether to temporary services or upgraded permanent utilities, require a coordinated game plan. Before a utility cutover can occur, one must first determine what loads are affected, what the power demand for these loads are, what schedule and time constraints to cutover are, and what infrastructure and equipment will be needed. To help minimize downtime and interruptions, temporary and permanent infrastructure should be built-out to the fullest extent possible, prior to cutover occurring.

New remodel or construction may require access or new pathways through existing critical care areas and can impact operations and adjacent operational spaces. Existing pathways may need to be removed and/or relocated in order to provide adequate space, headroom or clearance for remodel or new construction.  Extensive field investigation and surveying of existing conditions and tracing of existing circuits may be required to determine what existing utilities and infrastructure may be in the way or impacting the proposed construction.  These existing utilities and infrastructure can then be isolated, removed, re-routed, or re-fed as required to maintain operations and clear the area or free up capacity for construction scope of work.

In hospitals, isolation and infection control are other factors that compound the challenge. Additional containment may be required by isolating certain doorways, ducts, piping or conduit which are key to hospital operations but routing through scope of work areas.  To limit or mitigate utility interruptions, noise, vibrations, dust/dirt, and other construction-related elements in critical adjacent areas of operation, construction scope of work must be performed during off-hours or restricted times.  After these construction periods, utilities/services will need to be returned to adjacent operational areas.  Special cleaning, decontamination, and re-certification of equipment/zones to meet health care standards may be required, which must be factored into the schedule.

Commissioning and Training

A key element of the commissioning process for mission critical environments with complex infrastructure is a “blackout test.” This test is designed to “assure that all systems operate as expected on backup generator power, since utility power can be expected to fail occasionally but ongoing normal operations are essential.” A blackout test also makes clear what staff need training and where the issues are with controlled and clear communications among the key staff to carry out the outage emergencies and how recovery is accomplished. Read our colleague Shaun May’s article all about blackout tests for a play-by-play of what this process looks like!

Commissioning also provides the opportunity to catch small, seemingly harmless details that may impact utility equipment’s ability to perform when needed. For example, our team recently investigated problems with HVAC system components shutting down during routine generator testing and having to be manually restarted. For an environment that depends on reliable cooling in the event of a switchover to generator power, this was a significant problem. The initial thought was that power-quality issues were causing the shutdowns, but the end result was a combination of controls settings, equipment communication, and aging UPS equipment. These are the kinds of nuanced interactions that initial commissioning and ongoing commissioning is designed to address so that systems consistently operate as intended. Recommissioning is particularly valuable when infrastructure systems are updated or changed to make sure the whole interconnected system is optimized, performing as expected, and operating as efficiently and reliably as it can.

Operation and Maintenance

Keeping up with regular testing, calibration, and system maintenance is a crucial element of maintaining reliable resilient infrastructure. Generators should be routinely started and run under at least 30% load, for at least 30-minutes, to test that the system can start and carry the intended loads when called upon.  Emergency generators must be tested monthly per NFPA 110 standards. Batteries must be in good condition and have adequate cold cranking amp capacity to quickly and reliably start the generator.  Fluids (e.g. oil, coolant) must be clean, in good condition, and topped off regularly.

Uninterruptible power supply (UPS) system batteries should be in good working order and have a battery cell-level monitoring and charging system to help identify any potential weak battery cells which can impact a battery string and reduce UPS runtime. UPS inverter electronic components (e.g. capacitors, etc.) should be replaced at regular maintenance intervals, so that the UPS operates and can carry the loads that it was designed for.

Routine maintenance and testing of critical systems are not only code required, but provide valuable operational and condition baseline information of equipment to help schedule routine maintenance and help budget and save for equipment replacement/upgrades.

Ongoing training and access to operation and maintenance information is another important step in assuring resilience. A systems manual is a valuable support tool for operations and maintenance personnel. The systems manual includes not only information about the equipment components of each system, but goes further to describe each system, and how those systems integrate. This gives O&M staff a clear picture of the comprehensive interaction of components in a system and the inter-system activities and reactions that make up building performance. Read our colleague Jeff Yirak’s article about what makes a good systems manual.

Training best practice also includes running the operations and maintenance staff through all the “what if,” scenarios that were addressed in planning and designing the infrastructure. The best design becomes moot if the policies, procedures, and training for staff don’t get the same level of attention. Emergency situations are chaotic; lack of training and poor communications can only exacerbate the problem. Conversely, ongoing training and preparedness among essential staff responsible for maintaining the systems and the day-to-day operations of the facilities helps eliminate additional stress and contributes to successful outcomes.

The End Goal

Critical facilities such as hospitals and airports have been evacuated due to utility systems failures. You can imagine the operational nightmare and immense cost that comes from such an event. Utility infrastructure that is comprehensively planned, well designed, properly constructed, and regularly maintained by personnel with relevant policies, procedures, resources, and training is the path to creating effective resilience, by design.