AT&T is spending around $120 million annually on network resiliency work
As Hurricane Ian spun toward the Florida coast in late September, AT&T was making some risky calculations about where to stage pallets of equipment and tractor-trailers full of generators that would help power its network recovery after the storm passed. Usually a telecom provider—and other infrastructure providers, such as power companies—choose a staging site somewhere outside the storm’s expected path, 50 to 100+ miles away, and move in as quickly as they can afterwards. For the first time, AT&T was weighing whether to stage equipment not only in multiple places around the state, but directly in the hurricane’s path: At a tiny Florida airstrip operated by Lee County for mosquito spraying, just seven nautical miles from downtown Fort Myers, Florida, the town that would ultimately have 97% of its structures damaged or destroyed by up to 18 feet of storm surge.
Chris Sambar, AT&T’s president of network and the former head of its FirstNet network team, recalled the staging as an example of one of the times that AT&T has pulled in data from the advanced climatological model that it is using to inform its network infrastructure decisions and investments in resiliency and response.
“We took a lot of risk, actually, by staging a whole bunch of equipment in the path of the storm. We knew it,” he said. But in addition to the model, Sambar added, AT&T’s own experts saw the airfield as the right choice for placing some physical resources that would withstand riding out the storm— not including equipment with tall masts which would be more susceptible to damage, for example. Ultimately, FEMA and some local first responders joined AT&T in staging at the airfield—but not any other telcos, Sambar said. AT&T locked down its equipment and then employees left, to hunker down at a hurricane-rated AT&T building.
“We chained [the equipment] down to the airfield, and the hurricane literally went right overhead,” Sambar said, adding that he was told afterward that the equipment “never moved” during the course of the storm. “It actually worked beautifully, and we were able to get in [to impacted areas] really quickly.”
The staging gamble paid off in the form of a faster network recovery for AT&T—and its network then became a lifeline not only for its own FirstNet and AT&T users, but for other network providers’ customers as those operators worked to get their networks back up and running. Within the first 24 hours of AT&T opening its network for roaming, the carrier reported that it handled more than 9.5 TBs of traffic from other network operators.
The information that helped AT&T make this and other climate-related emergency response and network resiliency decisions is drawn from a sophisticated model developed by scientists at the U.S. Department of Energy’s Argonne National Laboratory and its Center for Climate Resilience and Decision Science, as part of a partnership with AT&T. Around five years ago, AT&T turned to Argonne for the use of its supercomputing capabilities to fuel a climatological model that would not just be able to look out the usual few years, but decades—as long as 30 years into the future—with climate projections that have “a pretty good level of sophistication and high probability of accuracy,” as Sambar explained it.
“For a telco like us, or any infrastructure provider like us, we don’t want to predict out two or three years—we’ve got to predict out as far as possible, because I’m building things that are going to be there for a long time. I’m putting electronics in the ground for fiber that’s going to be there for 30 or 40 years, I’m putting cell towers up that again, are going to be there for 50 years plus,” said Sambar—who likely has a greater understanding of climatological models than most network execs. He was a meteorology major in college.
Argonne scientists put the model together and AT&T started using the information in both its network planning and resiliency efforts, and in disaster response—first on a regional basis for the Southeastern U.S., due to the sheer amount of data that had to be ingested, then in expanded geographies across the country as more local information was built out.
AT&T announced last week that, in partnership with the Federal Emergency Management Agency and Argonne National Lab, it is making the underlying data available to emergency managers and other local, tribal state and federal officials through the Climate Rick & Resilience (ClimRR) data portal so that they can use it for their own response and resiliency planning. ClimRR initially includes data on future hazards related to temperature, wind, precipitation and draught, with modeling on wildfire and flooding risks to be added in the coming months. AT&T provided “foundational funding for much of the data” used by ClimRR, while FEMA chipped in funding for development and maintenance of the portal itself. Argonne scientists conduct all of the modeling.
The publicly available alpha version enables users to look at future precipitation, temperature and wind conditions for local areas within the continental United States and build analysis that draws on climate, community and infrastructure datasets. Modeling on wildfire and coastal and inland flooding risks are planned to be added in the coming months to the public version.
Why make that type of data available to others? It’s an AT&T corporate social responsibility effort (the ClimRR website notes that “much of this data was made available through AT&T’s philanthropic and sustainability missions”), but it’s also sort of a force-multiplier on disaster response—which is a heightened priority for the company operating the FirstNet network for first responders across the country, as well as serving its regular commercial and consumer users. “When we go into an area where we know there’s a hurricane coming, or a wildfire, or tornados, the better information everybody has in the area, the better able we are to serve the citizens in the area,” Sambar says. “When the local officials know … the probability of flooding, flood zones, topography of the earth and what’s going to happen where—when they know where the biggest impact is going to be—they direct their resources there. It’s really not that useful if just [AT&T knows] that and we divert our resources there. We want everyone coming to the same areas.”
In the case of Hurricane Ian, the asset-staging piece “was very important to us,” he said. “Knowing where the hurricane was going to hit, what parts of the [cities] and the state were going to be the most impacted, based on the storm and based on flood plains, storm zones—understanding all that helped inform us where to stage our assets. … As soon as the storm came through, we all came back and all of our stuff was there, ready to go. We were able to get in much quicker and recover the area much quicker.”
It also helped that FEMA (which also uses the Argonne modeling) and the local first responders who were also using the same staging site were on the spot and ready to go as well. “Everybody knew what was coming [and] had the best information possible to be able to react,” Sambar said.
“There’s always a fine line of what you want to keep as your competitive advantage and what do you want to share,” he reflected. “When it comes to this … it results in a better outcome for everybody when everyone understands what’s coming and how to prepare.”
But disaster response and recovery is only part of the network resiliency puzzle—and AT&T is also using the climatological data to reexamine its existing network and make changes and upgrades based on evolving climate risks. Read more in Part 2.
