Adjusting to Life Under ANS

The meeting's attendees included representatives from six mid- level networks. One, Joel Maloff left CICnet to become an ANS vice president shortly after the meeting. A second, Guy Almes, left SESQUInet to become an ANS vice-president in the spring of 1991. Bill Yundt represented BARRnet, Glenn Ricart SURAnet, Gary Augustson PREPnet, and Richard Mandelbaum NYSERnet. Mike Roberts was there on behalf of Educom's National Telecommunications Task Force.²¹ Al Weis and Jordan Becker attended for ANS, Doug van Houwelling for MERIT and Ann O'Beay from MCI.

The meeting established four working groups designed to contribute to "successful ANS mid-level relationships:" Services; Qualifications Process for Regionals; Growth and Funding; Organizational Structure. All were established specifically to serve ANS purposes with the Services Group charged with developing a rating mechanism for the quality and kinds of services offered by mid-levels to their customers.

As Mandelbaum wrote: "ANS felt that for it to establish successful mutually beneficial relationships with the mid-level networks, some methods of assessing the relative capabilities of these mid- levels was thought to be required. This led to the formation of a Qualification Process work group to structure a fair qualification process for the regionals" [mid-levels]. What Mandelbaum was saying was that the regional networks did not all deliver the same quality of service in the areas identified by the Services working group and it would be necessary to develop a set of "criteria or metrics that could lead to the qualification of a regional network for the services it may provide to or receive from ANS."

The Growth and Funding Group would "focus on determining the technical, financial and process issues relating to reimbursements across Regional/ANS boundaries." The Organizational Group would "identify and make recommendations on organizational relationships between the regionals and ANS." ANS assigned a staffer to coordinate relations between the groups which would be between three and five members in size including an ANS member in each group. The groups would operate by email and face-to- face discussions and would finish their business by early December 1990 when a larger meeting of mid-levels was planned. The larger meeting did take place at the end of the first week in December of 1990. When ANS unveiled its Plan for Commercial Services for the first time at the FARnet meeting of August 1991, it would cite that plan as the result of a consultative process with the mid-levels that began at the December 1990 meeting.

The flavor of the events of the autumn of 1990 however was rather clear. ANS as the new landlord of the network was telling the mid-level tenets what it would expect of them under the regime that everyone expected to unfold beginning in November 1991. At that time it was widely anticipated that the NSF would announce that ANS would remain the backbone operator on the November 1 1992 expiration of the Cooperative Agreement with MERIT and that the NSF would continue to buy services from ANS.^u Since the NSF had been subsidizing mid-level backbone connectivity, it was widely assumed that when ANS became completely independent, mid-levels would have to find additional money to pay ANS for their backbone connections.²²

Thus the mid-levels publicly set out to see what would be required of them by their new backbone operator in a privatized environment. They took care in their public actions not to alienate ANS while in private conversations many expressed considerable fear that ANS was out either to put them out of business or take them over.²³ What ANS wanted became clear in a document sent by Al Weis to Jane Caviness on January 15, 1991. This document, released to the author under FOIA will be discussed below. However, in order to understand the document a look at Al Weis' infatuation with bandwidth is necessary.

A Gigabit Salesman Takes to the Road

Wherever Weis went in the autumn of 1990 he was selling gigabits as the new bandwidth for which he was aiming. In an interview with the Chronicle of Higher Education published on September 26th 1990, he was quoted as saying ANS, "within two years," had "plans to offer speeds of a billion bits per second."²⁶ It later turned out that Weis felt that he had been misquoted, and said that he meant OC12 or 622 megabits within two years. He was adamant that breaking a one gigabit barrier (OC24) would take Federal help.²⁷

Some weeks later at Harvard University on December 1, Weis gave an hour long presentation on ANS's plans as part of the Kennedy School of Government Conference on the NREN. The general theme of the talk focused on how ANS would bring the necessary resources to the network to increase network speed and membership dramatically.

There Weis said: "But we [also] want to push technology and want to push it very aggressively. Right now we are upgrading to T-3 and in fact the west coast loop of the network has been running at T-3 for a couple of days. We are going to start testing OC12 next year, hopefully deploy it the year after. And we've got plans for OC24 soon. We would like to deploy it as aggressively as possible. At the same time we want to push down and expand as aggressively as possible." And a few minutes later: "and whatever you do you have to keep in mind that T-3 is only going to last a year or two and then you're going to be in gigabits shortly there after. So what you do today has to be workable with gigabits tomorrow."²⁸

From hindsight some two years later, it is obvious that Weis' vision did not materialize. (The backbone - ostensibly T-3 - has by ANS's own description seen bursts in the 16 megabit range and averages closer to ten.)²⁹ and ^v Weis appears to have made the same mistake that many technology enthusiasts do: assume that customers will come for the sake of the technology alone. His assumption seems to have been: we will offer the fastest most "modern" network and the Fortune 1000 will rush to populate it. A dubious assumption, for eighteen months later, his own network map would show ANS attachments representing perhaps a dozen Fortune 1000 companies.³⁰ and ^x

Who Will Mind the Public Trust? - December 1, 1990

After Weis explained how he was planning billing procedures for commercial traffic that crossed his backbone, Kapor asked to be recognized and said:

"Let me make a suggestion. In the light of what we heard yesterday from the economists about the complexities of how you might go about pricing and these issues, it strikes me that not only are the technical problems very hard but that the policy issues are equally hard and that ANS would do well to hold some sort of a public forum to pick the brains of the smart people who have thought about this as far as the policy issues and the options go.

I have to say that it makes me nervous when these deliberations take place privately because I think the public interest is involved in this. I think it is commendable that you are aggressively trying to build the infrastructure, but I would really like to see public involvement on anything that is policy related. And I'd help you organize it. I'm sorry to slow up your process. But I think that the consequences of it have a quasi public character. It would be a tremendous mistake to pay any less attention to the policies than to the technologies."

Weis replied: "lest you think that we are paying less attention to the policy issues let me just say that at the meeting we are having in December we have got people coming from the government, from the regionals, from higher education, and from Educom. We have so much to do that we just can't afford to slow down."

Kapor: "But you also have the public trust. And you should ask yourself: does the institution [the network] have a peculiar character because of its quasi public nature? And you are breaking new ground here. It is not simply private enterprise. Private enterprise is used to moving ahead in this expeditious fashion. But I'm suggesting that if you do this in your position, it will cause problems."

Comment from the audience: "It's very much like the debate between President and the Congress on how much consultation there will be on policy making. What usually happens when the decision making is done in private by the executive, is that they get hammered by the public afterwards. Whereas, if there had been some consultation in advance, it wouldn't have happened."³²

This exchange proved to be prophetic. Mitch Kapor understood very well because it was the network that people used to network with each other. No matter what ANS would do behind closed doors, information would likely spill out and would be shared via the network. This was one technology that, because it spread power horizontally, was almost impossible to take over from above.

ANS and NSF: Diverging Directions

The following document, released to the author under FOIA, provides some insight into the tendency of ANS (or perhaps MERIT and ANS) to go their own way in the autumn of 1990. At 9:19 am on November 29, 1990 Steve Wolff wrote to Hans Werner Braun who was a co-principal investigator for MERIT's cooperative agreement:

"No problems here at all. However the deal was and is that ANS must compensate MERIT for the connection; I quote from our letter to Doug van Houwelling of September 10th:

NSF agrees that the new corporation may solicit and attach to the NSFNET Backbone new users, including commercial users, and may connect them to new or existing nodes on the Backbone, with the understandings that:

1. such users will reimburse the new corporation for at least the full average cost of the connection, the added traffic, and additional related support, and

2. the reimbursements will be used to enhance the network infrastructure and services, in order that the level of service provided by MERIT under its Cooperative Agreement with the NSF not be diminished.

NSF and MERIT will agree on the technical means of compliance with 2) above.

I should appreciate understanding the methodology MERIT plans to use to arrive at the reimbursable "pass-through" amount under 1) (and I'm willing to help develop that methodology if MERIT wants NSF involvement). Also it seems that we'll have to "agree on the technical means of compliance with 2) above" sooner rather than later.

We're available to begin talking at MERIT's convenience. If MERIT enters into any interim agreements with ANS pending the NSF- Merit discussion(s), ANS should understand that they have no force of precedent.

Cheers, -s"³⁴

Wolff also asked MERIT to agree on the technical means of compliance with point 2 "sooner rather than later." The fact that, according to other documents released to the author and discussed below, agreement did not take place for another two years seems to indicate that MERIT and ANS began to act increasingly in accord with their own desires.^aa One wonders whether Wolff had any recourse to bring them to heal, other than what would have been the exceedingly embarrassing public disclosure of their disagreements?³⁵ and ^bb

Meanwhile, on January 4, 1991 ANS Vice President Joel Maloff sent a lengthy list of ANS answers to questions that had been raised about ANS in the author's computer conference on the NREN sponsored by OTA. Maloff stated: "Regarding ANS's contribution to the backbone, ANS and the sponsors of ANS have contributed roughly 2/3 of the NSFnet backbone with only 1/3 coming from the NSF."³⁶

The author became aware of another disagreement between NSF and ANS when Steve Wolff told the author that he disagreed with Maloff's assertion and that the amount of investment on the part of the NSF was closer to half.^cc Wolff spoke of a need to have Weis screen comments from ANS before they were sent to the OTA Conference. Wolff also stated that the author should not give Division Program Director Steve Goldstein an account in the OTA conference, because the NSF would speak with one voice there and it would be his.³⁷ Wolff was clearly concerned with exerting centralized control over the story that was being told about the NSF's actions.^dd

In the above-cited January 4th statement Joel Maloff also said:

"when the original NSF T-3 expansion was first proposed NSF could raise only enough funds to support a point to point spine network with no redundancy for fault tolerance, or robustness required to absorb the traffic generated at the T-3 sites. This was deemed to constitute an unacceptable level of service and the decision was made by the sponsors of ANS to add additional infrastructure to provide backbone redundancy, and facilitate restoration of services in the event of a single point of failure. NSF is not funding any of these enhancements which constitute a considerable fraction of the total backbone expenditure."³⁸

Maloff was pointing out "cost sharing" on the part of ANS, MERIT, IBM and MCI. Seven months later, Steve Wolff, in a message to the author, stated: "IBM and MCI will not say what their investment [in the backbone] is, but I have never heard an estimate below $60 million if you include the T-3 upgrade. This does not include the MCI/IBM $10 million initial stake for ANS."³⁹ Time and again over the next two years the "fact" that the providers of the network had spent more on it than they received would be used as justification for what by the end of 1991 many would call the uneven playing field. It was said that: ANS and its founders were justified in having an advantage because they had contributed large amounts of their own money beyond that which they got from the federal government to the network effort.

Yet the first ANS Form 990 doesn't bear these "cost sharing" contentions out. Sent to the IRS by ANS in April of 1992, it covers the period from October 1, 1990 to September 30, 1991 - roughly the first year of ANS' existence. With the most liberal interpretation it shows ANS as having spent about 4% more on the network than the federal government paid for that purpose. A less liberal interpretation would show it making a profit.⁴⁰

On December 9, 1992 in a telephone conversation with the author, the New York City Public Affairs Office of the Internal Revenue Service stated that the intent of the Form 990 is to show all the income received for and spent on the tax exempt entity's program service activity. While the Public Affairs Officer obviously did not want to "rule" on a case where the details were not directly in front of him, he said that if ANS had spent significantly more than it received for providing the backbone service, it should be apparent from studying its Form 990 (assuming that the Form was filled out correctly and completly). (In his review of this report Steve Wolff asserted that the cost sharing had been in the form of "in kind" contributions from IBM and MCI.) The IRS Public Affairs office stated by telephone on December 30, 1992, that in kind contributions had to be tangible goods, could not therefore include services, and should be included on line 1a of page 1 of the Form 990. Consequently the author's analysis which includes the $6,060,680 shown on line 1a appears to withstand Wolff's objection.⁴¹ and ^ee

ANS as Decision Maker for the NSF

In a section called "Background" Weis sets out ANS' charter:

"ANS was formed to participate in enhancing the competitiveness of the US in high performance computer networking by privatizing and expanding the current national backbone network serving the research and education community. . . . . ANS expects to broaden the utilization of the network, make available vital commercial services to the research and education community, increase the speed and capability . . .[etc.] . . . . With the formation of ANS, a transition will be required and mechanics must be created to allow for the conversion from a research effort to a viable privatized entity to carry out the mission."

Weis goes on to cite Wolff's letter of September 10, 1990 to Doug van Houweling authorizing ANS "to solicit and attach to the NSFnet Backbone new users, including commercial users . . ." He continues by saying that the "body of this paper describes how ANS expects to implement these concepts."

First and foremost Weis needs revenue to push the network to the speeds that he has desired. He proceeds to list nine activities from which he anticipates gaining revenue.

1. "ANS anticipates that many of its potential clients will be qualified institutions who are currently attached through NSF funded mid-level networks. Direct attachment to ANS will provide them with many benefits not available to them currently. Characteristics such as performance, services available, quality, and others will be the determining factor in many cases." Weis then goes on to point out that since such institutions are already attached to the network, if he prys them away from a mid-level and attaches them directly to his backbone, they will not count as part of "the ANS incremental user population" for which ANS would be responsible for investment in new backbone infrastructure. Nevertheless, he assures the NSF in the next sentences that ANS will invest in the network to broaden the overall infrastructure.⁴³

2. "Direct funding . . . will come from several sources. The initial direct funding source is the originating members contributions to establish ANS. The initial payment of $2 million has been received by ANS and an additional $8 million has been committed."⁴⁴

3. "In addition to the direct funding mentioned, both IBM and MCI will continue to invest significant indirect funds in the form of equipment, people resources, and circuits above and beyond the funding made available by NSF. . ."⁴⁵ Hindsight appears to show that these were activities that both companies were involved in anyway. While IBM was developing the PARIS/Planet Switch that Weis could use, it was not doing the work solely for Weis. (If the work had been just for the benefit of ANS, we should expect to see some indication of its value listed in the ANS Form 990.) The same thing is true for MCI which was developing SONET, but obviously not solely for Weis' benefit.

4. "Another significant source of funding will be: the attachment revenues derived from research and education users".⁴⁶ It is not clear what Weis means here. Perhaps he is talking about the universities that he is going to take away from mid-level networks and attach to his own?

5. "the attachment revenues derived from not for profit mid-level networks;"⁴⁷ All mid-level networks were attached by the NSF to the backbone for free. If this were going to change and a mid- level were to have to pay ANS $50,000 a year for a backbone attachment at T-1 or $300,000 for a T-3 attachment, the mid- levels would have to try to recoup these charges by increasing what they charged their university or commercial customers. To the extent that these prices grew higher, the gap between the cost of connecting to a mid-level and connecting directly to the ANS backbone would decline. This would make it easier for ANS to achieve the purpose cited in point one above: detaching current clients from the mid-levels and attaching them directly to ANS. Weis viewed the matter somewhat differently in his words to the author: the very success of the mid-levels that could sell attachments without having to account for the cost of backbone service could lead to the downfall of the network when the backbone could no longer support the traffic they shoved on to it.⁴⁸

6. "revenues from attaching commercial providers of information to the research and education communities;"⁴⁹ In addition to the direct attachment fee, it appears that Weis was thinking of this as a means of increasing the attachment fee paid by the mid-level in order to receive the commercial traffic.

7. "attachment revenues from for-profit mid-level networks;" These revenues would fall into the same category as point 5 above. They would presumably be those networks who over the next year in reacting to ANS pressures would form the Commercial Internet Exchange (CIX).

8. "other billable service and consulting revenues;"

9. "grants from industry and government. All of these sources will be required for ANS to meet its goals."⁵⁰

But the ANS vision did not stop with the cost recoveries from pricing just described. The Proposal went on to say:

"For ANS to make the investments that will lead the network to its next plateau, SONET technology, funding above and beyond the recoveries derived from its pricing will be required. It is expected that such funding will come from investments by government, industry and the research and education communities."⁵¹

The very next paragraphs of the proposal shift focus back to the present:

"Qualified mid-level networks are expected to attach to ANS so that they can offer their users the advantage of a broader set of usage guidelines and the ability to access commercial providers on the national backbone. This concept was discussed at a recent meeting organized by the not-for-profit mid-level network operators and the proposal currently under consideration is attached as Appendix C. [A Mechanism for Capital Infusion into the Backbone Via Mid- level Network Fees and Settlements for Commercial Traffic.]

Clearly the "reallocation" of "premium fees" was the infrastructure pool in nascent form. However when FARnet wrote to Al Weis during the summer of 1990 and said: "NewNet should be flexible in making business arrangements with the mid-level networks and should endeavor to preserve not supplant them"; it is unlikely that the imposition of a means of "capital infusion into the backbone via mid-level network fees" was what FARnet had in mind.⁵³ The above two paragraphs summarize Appendix C of the ANS "Proposal" of January 15, 1991. This plan would continue to evolve until August of 1991 when ANS presented it at a major FARnet meeting as the ANS Plan for Commercial Services.

Appendix C shows that ANS was proposing to measure traffic sent to and received from the mid-levels in the form of "COMbit counts which would be a "'Combination' of network elements characterizing traffic load . . . used to calculate specific percentages" in the categories of research and education, commercial, and commercial to research and education. ANS gave an example of a fee calculation for a midlevel 85% of whose traffic was research and education only, with 10% commercial to research and education and 5% commercial only. The network's base fee for attachment to ANS would be $5000. However there would be three fees added on top of this. First a research and education traffic fee of $10,000, a research and education to commercial traffic fee of $12,000 and a commercial traffic fee of $20,000. Such a network would pay the following additional fees: 85% of 10,000 or $8,500; 10% of 12,000 or $1,200, and 5% of 20,000 or $1,000. The total fee for the mid-level would be $15,700.⁵⁴

The Appendix goes on to list a Basic Fee of $15,000 and the difference of $700 between basic fee and the full $15,700 as the Premium Commercial Fee. (Note that it was presumably out of this very small percentage difference that the money for infrastructure enhancement would come after other deductions were taken.) It does not describe how the basic fee was arrived at and it also does not say what the fee would be when applied to a large versus a small mid-level. Nor does it say how the fee for a specific mid-level would be arrived at. In section H of the Appendix it does say that "funds redistributed to specific mid- levels would be based on individual negotiations between ANS and each mid-level."⁵⁵ Presumably the fee for each mid-level would be arrived at in the same manner.

After saying in the body of its Proposal what it would require of the of the mid-levels, ANS went on to say what it would give back to the network:

"In working toward meeting the spirit an letter of its agreements, ANS will fund:

1. Physical attachment to the shared infrastructure . . .

2. Services associated with an incremental ANS attachment to the shared infrastructure . . .

3. ANS traffic load placed on the shared backbone . . .

4. At NSF's request, ANS will participate at no cost to NSF, within the limitations of ANS' available resources, in technology demonstrations and conferences that serve the national interest.

5. ANS is committed to develop new tools and applications . . ."⁵⁶

Some of the Difficulties of Determining Who Congests a Shared Resource

This section states that MERIT and ANS will monitor the performance of the network "to identify potential sources of congestion. ANS is committed to assuring that congestion sources which result from ANS attachments will be eliminated through backbone engineering or additional investments in circuits, network equipment, software or services."⁵⁸

Congestion can occur in the backbone at three points: routers, circuits, and the interface or boundary between the backbone and mid-level network. "It is recognized that congestion between the backbone network and mid-level network may be the result of capacity limitations in either the backbone or the mid-level network. It is recognized that ANS will have no control or responsibility over congestion resulting from inadequate infrastructure within the mid level network."⁵⁹

ANS was saying if its congestion at the interface, it might or might not be its fault. Consequently this leaves routers and circuits. But then we learn that: "there is no agreed to model for measuring congestion within an IP based network. At present even the model of congestion determination as described in the NSFNET solicitation of 1987 is not currently measurable by any tools that have been developed to date nor does there seem to be any current work in the open literature suggesting that this is feasible.

In the absence of any data on the actual performance of the T-3 backbone, MERIT and ANS propose to use the best measurable performance of the T-1 network for which historical records exist as a gauge for establishing a capacity ceiling below which shared infrastructure would not fall as new ANS traffic was added.

This performance measure utilizes such factors as:

1. Average and peak delay - ping time
2. Average and peak delay circuit - utilization
3. Packet switched performance

The documentation for these statistics is found in the monthly reports generated for the NSF by MERIT and in reports presented on a regular basis to the NSFnet Executive Committee."⁶⁰

What these remarkable paragraphs seem to say is that there is no way to measure what has been agreed upon as a definition of ANS commercial use of the network. Furthermore there is talk of measurement of a shared infrastructure rather than talk of measuring the traffic of either component. One wonders if separate measurements of traffic on ANSnet and NSFnet are possible. It looks as though such measurements are not possible - a strange conclusion in view of ANS assertions that which the networks share the same physical facilities, they are logically distinct.

A director of an internet services provider who reviewed this text had the following comment:

"Even worse it suggests that as long as ANS keeps traffic levels and response times better than the T-1 baseline they see the remainder of the circuits as theirs to play with. So rather than taking a bunch of federal monies and adding in private moneys and delivering a T3 net, they look to be promising to deliver something "not worse than" their old technology T-1.

Does that sound like a real good deal to you? It begs the question as to whether the T-1 was ever really evaluated. Let's examine the promises here:

1. MERIT builds a T-1.

2. It carries "a lot" of traffic.

3. MERIT/ ANS promises to build a T-3, which will carry "a lot" of traffic. It does carry "a lot" of traffic, more traffic in fact than the T-1.^ff

Some Concluding Thoughts on NSFnet Privatization

For Weis offered a zero sum solution. To the extent that he was successful with his revenue raising tariffs, the mid-levels would loose by having to bear the brunt of additional charges on top of an academic user base that generally could not afford them. Especially if Weis succeeded in detaching many of the mid-levels current clients, these networks would be hard hit and would either have to increase the fees that they charged their remaining clients, go hat in hand to the NSF for more money, or go out of business. The last alternative was something that everyone knew that the NSF in the context of NREN could ill afford to allow to happen. What was by far the most likely would be for the mid-levels to go to the NSF for further subsidies.

Privatizing the one part of the network, the backbone, that had no paying customers didn't make much sense while the NSF still supported (if one can believe the accuracy of its own database figures)⁶² the more than 30 mid-level networks with new grants for both new mid-level infrastructure and "connections program" grants in the amount of $7,257,349 during fiscal year 1991 the first year of ANS' existence. The purpose of these grants were to extend the reach of the network and attach new colleges and universities to it. For ANS's plans to work it would have to reach into the part of the network where almost all users were attached and both take customers from the mid-levels and more money from them in the form of attachment and commercial use fees. To the extent that ANS was successful, it would create a situation where someone would have to come up with money to offset the charges that ANS imposed in order to pursue its own ends.

This however was not the picture that was given out for public consumption. When presentations were made, the emphasis was that IBM and MCI had committed vast resources to see that their non profit spin-off ANS got what it took to do the job. ANS Vice President Joel Maloff put it the following way: "IBM and MCI are each providing five million dollars to fund the startup of ANS. In addition they are spending significantly more internally to ensure that they have the technology required to achieve ANS's aggressive schedules in moving to gigabit per second services."⁶³