Disaster recovery learning from experience

Many companies based in Grand Cayman had the unenviable task of having their disaster preparedness tested by Hurricane Ivan in 2004. However, it presented companies like LIME with a unique opportunity to strengthen their disaster recovery planning and in so doing, benefit their customers.  

“Ivan was something that taught us lessons on a local, regional and international scale. We went through and did comprehensive research of what took place during Ivan, we looked at our network and we definitely have built in resiliency,” says Donnie Forbes, head of service support and delivery for LIME Cayman Islands. 

Lessons learned during Hurricane Gilbert led to LIME Cayman doing a comprehensive exercise to see whether the infrastructure it had built would be able to withstand a direct hit from a Category 5 hurricane and as it turned out the buildings that housed LIME’s personnel and equipment at the time would not have been able to stand up to the winds and flooding that would accompany a Category 5 storm. 

“In early 2000 we did an extensive set of planning, in December 2003 we started moving people and equipment into One Technology Square, and in September 2004 we got hit. Had the planning and everything not taken place I think the Island would have been a totally different place in the aftermath of Ivan,” says Forbes. 

Apart from the need for telecommunications in Cayman to be up and running as soon as possible in the aftermath of a storm, businesses also want to remain up and running and have their data being safe and secure. 

LIME offers colocation space at its One Technology Square building, which has proven its worth during Hurricane Ivan. It is a purpose built telecommunications centre, built to withstand a Category 5 hurricane with a basic wind speed rating above 200 miles per hour. The ground floor is 10 feet above sea level, with all network critical infrastructure as well as the data centre and associated infrastructure located on the second floor, well clear of potential flood damage. 

The facility is also serviced by two backup generators with enough fuel stored on site to run the generator for up to 10 days. This is supported with a redundant uninterruptable power supply system to ensure continuous service. 

This is backed up by cooling and humidity control measures to protect the equipment, as well as fire suppression and extensive security to ensure the safety of the facility. 

“This is one of the most advanced buildings in the entire Caribbean in terms of being able to provide disaster recovery. It is the heartbeat of the Caymanian telecommunications network and all disaster recovery really begins and ends right here,” says Rick Bengle, team lead corporate accounts for LIME Cayman Islands. 

Bengle suggests that businesses looking at their disaster recovery and business continuity plans question their prospective service suppliers on their facilities and their disaster readiness. 

“We have walked many of our customers through this building, and when they come in they don’t understand the exact nature of what we provide to this Island, and when they leave they have a totally different understanding. When they see the multiple generators upstairs that supply power, the backup plans that we have, they start to understand that,” says Bengle. 

As a result of Hurricane Ivan, LIME also worked on hardening the rest of its local infrastructure, including the cables running off island as well.  

“We did a lot of work in East End to harden that area. In the old days the track for the duct used to be on the sea side of the road, and as a result of Ivan where the road got washed out and we sustained some damage we moved to the land side. As a way to harden that we dug a channel into the ironshore, put in our duct work and poured it back in with concrete,” says Forbes. 

Starting in the 90s LIME has also invested heavily in the Cayman Jamaica Fibre System, which provides connectivity to Jamaica and onwards from there. The company is also in partnership with the consortium that owns the MAYA1 cable, which provides connectivity into Central America, North America and South America. This means that should any one of the cables be out of commission due to the impact of a hurricane or earthquake and alternative route for the data can be found. 

According to Forbes one of the biggest weaknesses that Hurricane Ivan exposed was not preparing for a hurricane but preparing for the aftermath and the recovery phase. 

“Now that everything is broken, how do you fix it? We did an extensive amount of work on that – our hurricane plan went from one volume to three volumes. We created tasks for everything we thought we needed to maintain the business,” says Forbes. 

This led LIME to create a recovery team that focuses on how to recover after a hurricane, including damage assessment and having the correct people on the ground and the correct resources in place. 

As part of a regional approach to disaster recovery, LIME keeps an up-to-date list of available resources throughout the region, whether physical resources or human resources, so the necessary skills and equipment can be tracked down quickly and brought to bear on any disaster recovery situation. 

“If Cayman gets hit I know that if I need something there is a database that says which country has it and I can get it shipped in at short notice. We also have equipment stockpiled in the US, in our closest point of contact. We have put together an extensive plan, and we do not anticipate there will be this mad rush after a hurricane that there was after Ivan to get everything in place, because a lot of that is already in place,” says Forbes. 

The availability of those resources is something that Bengle is quick to point to as a competitive advantage that LIME holds over its competitors. 

“The investment of staff that LIME has here is something to consider. How many technicians do you have, how many people do you have supporting the network that’s built? Again, when customers ask those hard questions they get very different answers from us than they might from some of our competitors on the Island. Those are things that we encourage prospective clients to ask,” says Bengle. 

According to Jefferson Tibbetts, enterprise account manager, LIME is able to offer custom designed disaster recovery solutions based on a company’s needs and budget on the back of their infrastructure. 

These can include business continuity solutions like running soft client applications on smart phones like the iPhone in order to provide a seamless experience for users should a business lose its office in the wake of a disaster, allowing users to answer calls to their office number on their mobile. Businesses are also turning to MiFi devices to provide data capacity in the event that Internet access at the offices goes down. 

“All these things are in play behind the scenes. With the hardiness of the network and the build out of the network here is just so much you can do,” says Tibbetts. 

Even though disaster recovery covers a wide range of potentially catastrophic events, the Cayman Islands are especially prone to hurricanes, which is why these receive particular attention in the disaster recovery plans of local companies.