Googin’s Paradox of the
Perfect Network Solved

Western Coloradans Building Different Level
“LoopCos” Designed for Local Needs

The dynamic playing out in Colorado may be seen as a metaphor for the current state of the national economy. The center is trying to rule by command-and-control but at the same time in the provinces knowledgeable people are taking a grassroots and bottom up based attitude toward their telecommunication problems. When I started work on this issue by interviewing Corey for more than 4 hours in mid-June, I assumed the story was nothing more than one talented person figuring out how to build a statewide open access backbone in an open and commoditized and therefore extremely cost-effective methodology. For Corey Bryndal of PacketRail that turned out to be precisely the case.

In our interview Corey tells how from the late 1990's till the present he learned the most cost effective methods of building IP networks that exploit a routed & MPLS core and Gigabit Ethernet at the edge.  He then goes on to describe in detail how he has built PacketRail,  shares the goals he hopes PacketRail will achieve, and demonstrates how an open, engaged approach pays dividends and creates abundance for both PacketRail and the customers he serves.

Give your customer the most bandwidth possible per dollar spent and organize your network to be able to do just this.  PacketRail is a working model of his 90/10 Rule.  The fixed 90% is the transport cost of delivering IP packets at Gigabit speeds from the public internet to a rural consumer. Included in the variable ten percent is the cost of transit across the global internet via connection with Tier 1 providers.  By engaging local community stakeholders, and using an open-access model based on Carrier-Netural-Locations, a network purposely designed to fit the different requirements of each community is created.   The philosophy? Encourage your customers to share their needs & desires and everyone will benefit from the resulting collaboration.

Top Down

However it also turns out there is a lot more going on in Colorado than I knew at the time.  Consequently this issue has grown far beyond the concept of the initial interview to one that tries to paint a picture of the state caught in the crux of ferment between a top-down DC Beltway approach and a local rural Colorado citizen-based bottom-up effort.  I had become aware of the Eaglenet project in talking to the Internet2 folk in January 2011 and wrote about it in the combined seven months issue of the COOK Report published on April 25.  I assumed, falsely it turns out, that Eaglenet would be like the Kinber project in Pennsylvania and the 3 Ring Binder project in Maine also funded by NTIA stimulus grants. Fortunately in both of these later two case [Pennsylvania and Maine] those in charge appeared to be working very closely with the people in the communities they will serve and their proposals as submitted to NTIA were open with nothing redacted.

But much to my surprise, with Eaglenet, it turns out that NTIA has funded a closed and proprietary project submitted by a school technology purchasing consortium known as Centennial Boces.  The award was announced immediately after Labor Day year ago.  Last fall the first major thing the awardees did was to ask for and receive permission to transfer the award to a new “intergovernmental alliance” called Eaglenet. This was approved in February of this year.  The application funded was a Round B application.  The same folk were turned down by NTIA with their Round A request for 2009. Meanwhile, as best as I have been able to determine, the Round B was submitted to NTIA with apparently perfectly legal requests on the part of Centennial Boces to redact the most critical parts of the application as proprietary. Not surprisingly the project has been much discussed within Colorado. I am told that the Eaglenet grantees have been unwilling to answer publicly requests for information.  Nevertheless, the details of what they were doing would all come out in good time.

And most of the details indeed have now come out thanks to members from my Arch-econ mail lists who downloaded the redacted files, opened them with open source software, and removed the redactions.  Consequently we have seen that the NTIA funded EaglenetGrant is to three large corporations IBM, Adesta and Conterra that have never built networks in Colorado before.  Furthermore, from all the information I have been able to gather working full-time for the last six weeks, these folk have been having very minimal communication with the people in Colorado who will use the network that is to be built with $100.6 million of directly awarded NTIA stimulus money.

From my publisher's desk I have myself talked with or gathered information from the Region 9 Economic Development District and SCAN network, the Colorado Springs Independent, and key people in no less than five different Colorado rural telcos of various sizes all of whom have been asking Eaglenet to discuss their build plans only to have been greeted with silence.

Bottom Up

I was alerted to Corey's existence by Arch Econ list members who attended a meeting called MountainConnect that he put together in mid-June near Durango Colorado. This meeting gathered together approximately 130 people from rural areas of Colorado who are involved in Local Technology Planning Teams facilitated by an NTIA funded broadband outreach program located in the office of Gov. Hickenlooper.

These folk enabled by a program by the Governors Office Information Technology represent the other side of the unfolding Colorado story.  The governor has a program for economic development within Colorado called  Colorado Blueprint. In a very creative manner, the Colorado Blueprint program has been put together by a series of bottom-up meetings in every county of the state.  These are meetings designed to get people together and talk about local economic strengths and what is needed to make their local economies more resilient.

Operating under this overall umbrella is the Colorado Broadband Data and Development Program. This program operates under the Gov.'s office of Information Technology which has received a total of $5.4 million in funding from NTIA for broadband mapping assessment and outreach.  So far there are 6 local technology planning team efforts underway.  These folk are putting together a carrier neutral location points in each of their areas. These points discussed on pages 31 and 32 would save to the middle mile networks  “our community asks you to terminate traffic in a neutral exchange within our community where customers can establish their own competitive last mile providers.”

One of the six local technology planning teams, that is the Region 9 Team is a year into the building of what it calls the Southwest Colorado open Access Network or SCAN.  SCAN is documented on pages 41 through 46. SCAN is a local equivalent to the perfect network as a utility concept that introduces this issue and the concept on which I conclude that Corey Bryndal's PacketRail is also built.  I see both as “loop cos” built to enable their customers to build and own infrastructure rather that pay an absentee owner of local infrastructure via recurring fees over and over again on.  The model is put 90% of available funds into fixed price transport that you terminate as physically close to your customer as you possibly can.

The five other Local Technology Planning Team operations have the opportunity of working through the governor's bottom-up program to build SCAN-like networks in their own regions. It should be rather obvious to everyone outside the Washington Wall Street Beltway echo chamber that this kind of approach designed locally by the people of the community's that will own and operate their own perfect network utility should become the wave of the future.

Washington Closes the Barn Door

Apparently NTIA is legally required to keep secret what the grantee tells it to. Therefore once NTIA was informed that the Eaglenet award files had been unredacted, it took them down from its website buttoned them up securely this time and returned them to the website in late July. It was rather a bit late because the unredacted material is now all over Colorado and for the first time the people of the Colorado who take an interest can see the Eaglenet plans submitted in March of last year to come in and overbuild much of the existing infrastructure in the state without ever having bothered to talk to those on the ground in the areas of rural Colorado that they are claiming to serve. The maps that are now in public’s possession make it clear that North Carolina-based Conterra is building a 150 to 300 MB microwave network patched into a number of aggregation points by some new pieces of fiber.  The precise location of perhaps 200 microwave endpoints is known along with the Eaglenet's quite expensive pricing for bandwidth that, according to Conterra’s detailed but redacted network maps, uses end point radios that are capable of only 40 or 50 megabits each.  As far as I can determine, the Conterra network brings bandwidth to district headquarters leaving each district to connect its schools.

Now SCAN is in the midst of an RFP process to build its own community owned fiber network. It has awarded the two city hub and ten gigabit backbone transit to PacketRail.  But from my reading of both the Eaglenet unredacted documents and the SCAN documents, it appears that a stalemate exists.  Until Eaglenet steps forward and publicly says where it is prepared to spend money in Region 9 by means of describing exactly what infrastructure it plans to ask Adesta and Conterra to build, Region 9 has no further idea on how to design and release an RFP for the building of the already Colorado funded local fiber feeder networks that will go from the towns of the region into Cortez and Durango.  In other words Region Nine could benefit from an open and public response from Eaglenet that explains what Eaglenet intends to do.  I have quoted Eaglenet questions asked by SCAN in April and May of this year on pages 49 and 50 of this report.  As of August 4th these are unanswered.  For example a very critical asset is fiber from Durango to Silverton via the world famous railroad which is shown as being provided by Eaglenet on one of the maps retrieved from the NTIA website.  So far the fiber does not exist and Eaglenet has not committed publicly to build it.

Eaglenet’s top-down, command and control approach, is now being played out.  It has begun to awarding contracts for services that people in Colorado say they have seen no requests for.  See material added on August 3 to the Recovery.GOV Track the Money website.  Eaglenet seems to have decided to do what it intends to do on a take-it-or-leave-it basis.  It has apparently received its environmental approvals within the last few days and almost a year after its grant was awarded.  Since this is a grant that still has almost two and one half years to run, it will very likely begin to make announcements of what it is going to do and where. But since these announcements have apparently been based on no discussions with any existing network owners or with any Local Technology Planning Team members, the likelihood of a congenial outcome seems questionable.  Nevertheless, one would still hope that the Eaglenet folk will figure out how to have open discussions with the grass roots, bottom up Coloradans they claim to serve.

Given the perilous state of the economy, it seems that it would behoove everyone in Colorado to sit down and talk very openly with each other about how the $100.6 million in federal funding can most effectively be used for its stated purpose of bringing better broadband connectivity to the citizens of Colorado. It is my understanding that efforts to do this are underway and I very much hope that they will succeed. If the federal money is used to support the homegrown efforts of Colorado companies, then the people of the state and their children will have been well served.

But no matter what the outcome, Colorado has done well by demonstrating in the ways outlined in this issue that equipment commoditization combined with open-source technology know-how have matured to the point where a Googinesque Perfect Network can be built as a local utility in the service of, by, and for the local economy of its builders.



Introduction - The Perfect Network Lives at Last    p. 3

The Expectations Mismatch                                             p. 3
PacketRail and Googin’s Paradox                                    p. 5
Genesis of the Perfect Network Colorado Style                p. 5

Earlier Career of a Local Technology Leader        p. 9

Learning the Provisioning Process                                    p. 11
Cisco, Level 3, MPLS and Softswitches                           p. 12
Aggregating Access technologies                                     p. 13
From Extreme Networks Late 1999 to 2003 and Then
to the Hospitality Industry and Consulting                        p. 17
Keep your Builds Simple --  Banish PON                        p. 21
Point-to-point Ethernet Without any Proprietary
Distribution System                                                          p. 25
PacketRail: 2009 to Present                                              p. 26
Egress to the Rest of the World via Collaboration           p. 29
Broadband Local Technology Planning Teams Are
Investing their Own Communities                                    p. 31
The MountainConnect June 19 -20,   2011 Meeting           p. 34

PacketRail’s   90/10 Rule                            p. 37

Googin’s Paradox Revisited                                         p. 42
PacketRail Photo Album                                              p. 44

Southwest Colorado Access Network (Scan)

as an Example of Googin’s Perfect Network               p. 45

Executive Summary                                     p. 51