Natural disasters, hurricanes, ice storms, fires, earthquakes: Is your Outside Plant (OSP) at risk? Do you have a plan? You can design and build the highest quality OSP infrastructure, but Mother Nature will always be there to test it. Are your infrastructure, technicians, and operations ready for the test?
If you are responsible for the operation of the OSP infrastructure, a natural disaster is always a concern. A hurricane, blizzard, winter ice storm, or flood warning may only allow a few days to prepare. Will that be enough time for your system and crews? A fire or an earthquake could occur at any moment. In the event of an earthquake, distant and wide areas can be affected by ensuing tsunamis. In all extreme situations, catastrophic damage is expected.
This article offers guidelines to prepare your outside plant infrastructure and crew to quickly respond to outages and damaged OSP infrastructure.
Crucial points to consider before a disaster or outage.
Minimize the effect of the event by implementing a routine maintenance schedule.
Restore service faster and minimize costs with an emergency restoration plan.
PREPARING IS ESSENTIAL
Preparing for emergency restoration begins with the initial design and installation of the OSP infrastructure. The OSP should be designed with solid and proven engineering principles and detailed in its design. It needs to be built with premium materials designed for the OSP’s environment and utilizing trained, qualified construction crews.
Quality assurance inspections and on-site observations are a must to ensure the crews are following the detailed design.
Documentation should be complete and include the following:
As-builts and redlines
Maintenance hole location
Equipment, vault, and handhole locations
Pole locations (all with GPS coordinates if possible)
Splicing records and testing results are needed for
copper, fiber optic, and coaxial cable.
The records shall be kept current and available with GPS coordinates if applicable, to provide quick access to the availability of vacant ducts, spare fiber strands, and copper pairs or to be utilized in restoration efforts.
Availability of redundant routing should also be considered at this time to help in either rerouting services or minimizing the effect.
Additionally, having a plan for spare equipment and ensuring the ability of crews to have 24-hour access to supplies, and additional labor forces (if required), is a crucial step in the ultimate restoration of service.
MAINTENANCE IS REAL!
Emergency restoration is triggered by an event, while maintenance is an ongoing process. At first, these two functions may seem unrelated. But a robust routine maintenance plan can greatly enhance disaster preparedness.
Once the initial design and installation have been completed a maintenance plan needs to be implemented. At the core of any maintenance, the operation is the on-staff technicians performing the work. Technicians need to be readily available and have the knowledge, skills, and abilities to perform the day-to-day tasks that are required to keep the OSP in prime condition.
Items addressed during routine maintenance will benefit the OSP infrastructure in the case of a natural disaster.
Conduct regular pole inspections:
Check that poles, messenger, guys, and anchors are in good condition and up to code
Verify bonding and grounding
Ensure there are no unauthorized attachments that could create a dangerous or hazardous electrical condition
Inspect pole attachment zones, aerial terminations, and splice enclosures
Implement an annual tree trimming maintenance plan
Confirm that all terminal cases, (aerial and buried), cabinets, and pedestals are sound, labeled, and clean, with no wildlife intrusions.
Check the conditions inside the pedestals, maintenance holes, vaults, and cabinets
Verify bonding and grounding
Ensure vacant ducts have duct plugs in place
Replace or refresh pest control in cabinets, pedestals, and vaults
Travel infrastructure routes
Verify that route markers are in place.
Confirm labeling on the poles, cabinets, and equipment.
Perform Right of Way Inspections
Clear or trim trees or shrubbery and confirm that the property owner has not blocked access.
Inspect maintenance holes or tunnels to verify that they have not been damaged by traffic load, ground thaw/movement, or construction.
Maintenance staff will often conduct an emergency restoration. Ensure that all of your on-staff technicians have been properly trained on the products that they will be repairing/replacing. Another best practice is to establish redundancy in your crew’s skill sets, including cross-training and hands-on field restoration training.
A disaster area is not the time for new product introductions or training. The maintenance staff needs to be well trained in safety, and standards to ensure that safe work practices are followed, and the system is restored to proper standards and code compliance. An ongoing training program should train not only the initial skills for routine maintenance, but also continually develop the workforce to perform disaster recovery or emergency service restoration.
MAINTENANCE PLAN BENEFICIAL IN MANY AREAS IN ADDITION TO ENSURING A SKILLED WORKFORCE, A MAINTENANCE PLAN CAN BE BENEFICIAL IN MANY OTHER AREAS.
Complete documentation of the OSP infrastructure (and areas adjoining, if available) is critical to restoration.
If this is not already available, the technicians can gather the information during their day-to-day activities.
Dispatch can process a work order to provide the crews the time allocation to gather and/or create this information on days where other maintenance requirements are relatively “light.”
These activities would include documenting circuit layout records (CLRs for all circuits (copper and fiber), creating cable and pathway records with status (rod and mandrel history), maintenance hole (MH) butterfly detail drawings, and overall system drawings.
Documentation of all currently installed equipment should be available for reference.
As part of any maintenance plan, a well-documented storage location for repair materials and supplies is key.
Technicians should be trained and familiar with the process for 24/7/365 access to materials, supplies, tools, and test equipment required for emergencies.
They should also know how to charge, calibrate and maintain test equipment and reorder necessary materials.
Ideally, emergency restoration equipment, tools, and supplies should not be utilized for routine maintenance. Where this is not possible, critical test equipment, tools, and supplies that are drawn for routine maintenance must be documented to be immediately reassigned in the event of a disaster.
The maintenance plan should address the standard practice for what is considered routine maintenance and support of the system, and what would be considered a natural disaster or emergency. A schedule for testing and calibrating all equipment (if required) should be included in the plan. This includes, but is not limited to, dielectric testing (rubber gloves, mats), meter calibration (OTDRs, LS/PMs, certification test sets), annual maintenance records (bucket trucks, trenchers, plows), etc.
With a robust maintenance plan in place, the OSP infrastructure should always be in the best condition to withstand a natural disaster. A maintenance plan also provides a procedure to address most of the challenges faced during the disaster.
Once the restoration area has been deemed safe, the first action is to call a well-trained team that has the knowledge, skills, and ability to perform the repairs. They should be well trained and know the procedures and their specific responsibilities.
Be sure to remember that your technicians may have also been severely affected or impacted by the same disaster. They may be dealing with emotions that can impact their performance. Therefore, a technician-only escalation hotline is recommended so you can deal with the realities they are facing at home or work.
ENSURE that there are enough materials to complete the restoration and that there is a plan to replace or replenish items and consumables (cable, poles, terminals, cabinet, splices, tapes, compound) used for the restoration.
IMPLEMENT a procedure to document all repairs, replacements, and any circuits that have been reassigned which will allow any temporary repairs to be made permanent, and facilitate reporting for insurance purposes.
DOCUMENTATION AND PRIORITIZATION of all circuits, and priority circuits, should exist. Priority circuits need to be addressed first in any restoration.
Properly designing and constructing the OSP infrastructure to codes and standards, using premium materials, should prepare it to withstand the event. Following routine maintenance will keep the OSP infrastructure in the best possible condition as it ages, and allow it to be repaired or restored quickly if disaster strikes. Lastly, in addition to the emergency restoration plan, keeping the technicians fully trained and having restoration materials readily available will aid in shortening the impact and duration of the event.
Do not wait for the disaster happen! Murphy’s Law implies that if something can go wrong it will, at the worst possible critical location, and at the worst possible time. To prevent Mr. Murphy from having a gleeful grin at our expense, stop him in his tracks by remembering the five Ps: Prior Planning Prevents Poor Performance!
Set your plan in place and perform a test run to verify. This will allow you to find and correct any errors or omissions in the plan. Is it expensive to do? Absolutely! But is it better to find out before your OSP infrastructure has been affected and customers are out of service, or before? If it is your responsibility, I think you now know the answer.
Article first published in ISE Magazine - August 2021 Written by Light Brigade Instructor, Jeffrey A. Noble, RCDD, OSP, TECH FOT, FOSP, CA, CT