A Practical Navigator for the Internet Economy

Fast Thinking    PART 2


The Emergence of  a Global

Collaborative “Operating System”

The second half of this book has laid out the foundation of the grid and illustrated a few of its more interesting application before closing with an interview with Dave Lambert the new CEO of Internet2 and with my own evaluation of the future. But let me invite my readers to step way back and think about what this all means.

Can you imagine a world established by means of a communication mechanism that connected all its knowledge makers, its teachers and its curious citizens who wanted to understand more about their own environment into a learning system for research and discovery? How about a communications environment that has made possible a grid on which its inhabitants could self-organize to accomplish their tasks? This is what has been happening over the past 15 years or so in such a quiet a way that most people are entirely unaware of the significant changes and developments outlined in this book.

Becoming somewhat dimly aware of what was happening, I wanted to understand it in its totality which, at the end of about two years of solid work, I now do. But I also believe that, outside the small circle of implementers, few people are aware of the significance of what has been done. It may have been the case in times of prosperity before 2008 that it served the cause well to fly under the radar. I would argue however that in the very different environment 2011 when the old bubble mania has been kept alive for Wall Street  while being replaced by an austerity mania for the rest of us, it becomes vitally important, if this good work is to be kept alive and benefit all of us, for a much broader public to understand it and talk about it and help in its implementation and support. These  goals are another reason (besides my general curiosity) for writing this book.

What’s Happened

Let’s recall what has happened. Physical highways still exist.   However the globe has been encircled many times over with glass highways. The small band of men and women honored in this book have taken what they have learned about packet networking and applied it to fiber and the light paths that replace electron streams. They’re doing so has banished the old world of bandwidth scarcity based on copper and dependent on all manner of electrically hungry equipment to shove bits down those relatively resistant copper pathways.

This book explains in the collaborative research ecosystem that is emerging. It shows how they have woven together an intelligent canvas called by various names of “grid” or “e-grids”.  This is a canvas that, right now from the individual user’s point-of-view, is more like a star network than a mesh network that the term “grid” calls to mind. It is however an environment that is unlike any environment on the commercial Internet today.  This uniqueness is because, from the ground floor to the penthouse, it has been designed for collaboration and for social networking be means of the formation of groups of people with like interests gathered into virtual organizations.  It offers such groups of people a portal into a global community numbering probably hundreds of thousands now if not millions and one would hope soon to be offered  by turning tens into into hundreds of millions.

The fortunate inhabitants of this new world can gain entry through authentication and authorization given by their affiliated community.  Once they  join, they will have the opportunity to see the disciplinary grids offered by the organization. In talking with their local experts they may select one or more of these grids. They will be given then the proper instructions for accessing them.  At the present moment, their access will be pointed to a physical server and computational infrastructure that normally is a part of the infrastructure established by the country in which they live. However this does not always have to be true. In any case software, in what you might call their home node, speaks to many other notes that are gathered together to assist researchers in this particular field of expertise and mainly number in the dozens or in the case of the worldwide large hadron collider grid in a figure totaling well into the thousands.  Furthermore, in the case of the LHC offering likely well into the hundreds of thousands of computing cores spread all over the world and linked by optical fiber at Bitstream speeds so high that for most practical purposes they could all be in the same room.

The grid fabric could be thought of as a potential department store like organization or perhaps better to say library like organization for the real-time evolution of the world’s knowledge. The idea is to take the grid software stack that is discussed in its various components in chapters throughout this book and apply it to this stack so that individual users can have versions tailored to their needs.  But, at the same time the network technical folk must do more work in order to make these intelligent systems scale. 

Right now this global eco-system is in its beginning stages and requires a lot of manpower to make it work. Work is ongoing to develop a user interface that is more intelligent and needs much less mediation by people who are both computer and subject matter specialists. Work is ongoing to refine what is essentially a user interface in such a way that it is much more amenable to questions from its users and can guide them to connecting the resources they need in the most expeditious way. Achieving this is the 21st century equivalent of the duties of a reference librarian in the 1980s.

The overall purpose of all this is to give members of the involved user communities the means of organizing and networking on a social and intellectual level while being connected or connectable to the scientific instruments they need.  These range from those that focus on the study of the atomic and molecular level all the way up to the ecological level of the surrounding world and the global level of tectonic plates and weather patterns to the universal intergalactic level of radio telescopes. With access to instruments, they also have access to data collection into huge data stores measured in terabytes and petabytes. The system will enable them to catalogue these data stores, find them, retrieve some, send them where desired for the application of various forms of high-performance computing, retrieve the results, visualize the results, and finally discuss them in videoconferences – locally, regionally or globally.

In Whose Interest?

The ability to do all of this exists right now. It is being worked out and refined in a self-organizing way that is fascinating to observe. At this building level the movers and shakers not surprisingly have been the computer and network people. Scientists have been working alongside them to explain the abilities of the tools which they desire. But by now these abilities are reasonably well understood and a second and absolutely critical phase is beginning. This is one that is designed to reach out to entire scientific communities and show these communities what is available and how it is in their interest to begin to adopt them.

Of course this will involve some problems of training and serious fundamental issues of getting these research disciplines to adopt toolsets that will in all likelihood be mandatory both for enabling them to do what they need to do, and for solving the problems they need to solve. It is necessary to understand however that these new tools are as important and significant for our continued understanding of what we are doing within the global and universal ecosystem in which we find ourselves. The problems of adoption in transition are not small. In fact they are on the same level of importance as those encountered in the transition from hand written books to the printing press and all the other ways in which the human mind over the last five centuries has developed to be able to use the information acquired and make it into meaningful stores of knowledge in ways that have been chronicled most recently by James Glieck in his new book called Information.

There are a few things that are probably less well understood about what is happening.   In order to give them appropriate evaluation and measurement in setting priorities for the use of capital and resources it is necessary to think about this in terms somewhat different from those of 20th century capitalism. This is not a system and these are not tools that exist for enhancing the development and sale of commercial goods at a general level of so-called consumer products that have been marketed and advertised death in a way that, before the spread of the Internet was much more amenable to use by large top-down hierarchical organizations. This is not the development of an entertainment system. It is not the development of enhanced marketing or manufacturing of goods to be briefly used and deposited soon thereafter in landfills.

It is the development of a system - a system which is an infrastructure.   We’ve all come to understand the infrastructure of highways, water and sewage services, and electric grids. Although there has been much discussion in the last 5 to 10 years of Internet as infrastructure, understanding the new and significantly different operational ecosystem described in this book as an enabling system for the continued creation of knowledge infrastructure is critical in enabling people to understand why this mechanism for building knowledge is a system that is so basic and so pretty much uniform in its basic constructs that it makes absolutely no sense to think of it ever becoming commercial. Lights, road systems, water systems, electric systems -- you name them.  They are all parts of a single global system of basic civilized infrastructure that cannot be isolated into the ownership of one company or group of people because basically they are natural monopolies. It is so basic and so uniform that it makes absolutely no sense to even try to define let alone think of a competing system.  It would mean building something huge and expensive twice when doing it once is quite adequate.

We shall have globally developed collaborative eco-systems of people that are working within their communities and within their colleagues to define and redefine the community by community basis what is necessary. It is imperative that we understand these most basic fundamental differences

And in the United States it is imperative that we seize the opportunity of US UCAN to bring these tools into their respective communities at a level where they are understood and developed from the point of view of infrastructure and not as a new package of soon to be commercialized Internet services.  They are the glue that makes the community a community and they will be in their own way unique to the people and the values of their respective communities because they are open and run on the uniform standards.  Namely standards that are interoperable but can be tailored to the preferences and needs of their local users.

What this book is talking about and what it is seeking to articulate is an understanding of the development and emergence of something much better thought of as a global collaborative operating system.  This is an OS that can be embedded in the values and points of view and interests of each of its communities and in no way is defined as something that can be packaged and merchandised and sold and delivered top-down from any commercial organization to purchasers at the edge.


Part 1

How Human Ingenuity, DIY Technology,

and Global

R&E Networks Are Remaking the World

Abstract                                            p. 11

Introduction: The American Political Context

The Quiet Coup                                             p.12

Becoming a Banana Republic                                 p.13
The Wall Street–Washington Corridor                       p.14
Susceptibility to Systemic Shocks                             p.14

Outcome Will Be the Hollow State                             p.15

The Network and Local Resilience                             p.17

Chapter 1
Collaboration Networks: the New Glue
in a World of Failing Nation States?

Consider Then What Lies Ahead                                p. 18
What about the Rest of “Us”?                                p. 19
Absolute Necessity of Grasping
Transformative Impact of Technology on National Economies        p. 21
Role of U. S. UCAN Network in Uniting
Research & Education with the Rest of America                    p. 24
Thumbnail Sketch of the European & American Situation            p. 25
A Future Scenario of What is Possible                        p. 28
The Power of the 20th Century “Center”
Leaks to the 21st Century “Edge”                            p. 29
Open Everything Brings Major Change                        p. 30
Civilizing the Economy: A New Economics of Provision            p. 32
Economic Disconnect Addressed by Two Stories                p. 33
Creation of Economic Value Moves from Proprietary Silos
to Collaboration Infrastructures                                p. 34
Getting Better Faster by Working Together and By Creating
New Architectures                                        p. 35
The Economy Must be Freed from Dependence on
Scarcity Communications                                     p. 37
Collective Intelligence                                     p. 37
Bridging the Gap Between High End and Local Community            p. 38

Chapter 2
The R&E Networks in the Rest of
World: National & Task Oriented

Europe, Asia- Pacific, China, India, Canada                        p. 41
The United States
Early History of I2 and NLR                                p. 42
National Lambda Rail                                        p. 43
The Merger Imbroglio                                        p. 45
US UCAN – a New Beginning                                p. 46
Internet2 K20 Initiative                                    p. 52
The Civic Applications                                    p. 54
The NSF and Cyber Infrastructure Office                        p. 58
Significant American Researchers                            p. 60
The 10th Annual Global LambdaGrid Workshop                    p. 65
Modern Research Needs an Integrated ICT
Infrastructure:  an Eco-system                                p. 66
The American Model Versus the Netherlands                    p. 67
The Future                                            p. 68
A Cautionary Note                                        p. 69
Survey of Internet2 Services and Activities                         p. 71
Collaboration with DICE, GLIF and Joint Engineering Team            p. 74
Federated Identity Management                                p. 75
Scientific Outreach                                        p. 77
Gratia Accounting                                        p. 77
Remote Instrumentation & Data Visualization Tools Working Group      p. 78
An Example of a State BTOP Grant -- Michigan                       p. 79

Chapter 3

GLIF and SURFnet – a Parallel History
of Infrastructure Development

Interviews with Erik - Jan Bos, CTO of SURFnet                    p. 85
Fiber Acquisition in the Netherlands                            p. 88
Cross Border Fiber in  Europe                                p. 91

GLIF -- The Beginnings and
Organizational Structure                            p. 91
Open Protocol Issues and GLIF Inclusive Philosophy                p. 96
GLORIAD                                                p.105
Automation of Lightpaths for the Integrated Collaboration Facility         p.106
The Collaboration Infrastructure - SURFconext                    p.106
Collaboration Infrastructure Software: Science Without Borders        p.107
How GLIF Provisions its Bandwidth                            p.109
How SURFnet Enables Innovation                            p.110
K-12 Education in The Netherlands                            p.114

Chapter 4

The Collaboration Infrastructure
in Detail

OpenDrac                                                p.115
Enablers of Cyber Infrastructure                                p.117
Increasing Collaboration                                    p.124
Trusted Identity Management                                p.129
Defining the Online Collaboration Environment                    p.137

Chapter 5
Supercomputing -
the Eco-System Pinnacle

The Role of SARA, The Dutch National Supercomputer Center         p.143

The Ecosystem Pyramid                                     p.144
Figuring out What You Need for Resources                        p.147
Tuning the Machine                                        p.150
Multi Disciplinary Teams and Blue Brain                        p.154
Need for Petascale Computing                                p.155
Why No Supernode and SARA Merger and Split                    p.159
Commercial Spinoff and Green Energy Concerns                    p.162
Impact of Collaboration infrastructure on SARA                    p.166
Where the Global Supercomputing
Eco System is Headed 
Sustained Petascale computing in US and Switzerland                  p.168
Switzerland                                            p.171

Executive Summary                             p.176


The Emergence of  a Global

Collaborative “Operating System”

Introduction                                            p.181
The Collaborative Architecture of Grids                    p.183

Chapter 6
The Grid Ecosystem an
Architectural Overview
Maurice Bouwhuis

Computational Eco-systems                                p.185
The User and Software Perspective: Interfacing with the Grids        p.189
Communication and Collaboration is Transitioning
from Point-to-Point to Fabrics of Grids                        p.191
The Eco System Enabling the Data intensive Scientist’s Work        p.192
The Physical Architecture of the CPU and
Data Intensive Grid in Amsterdam                            p.193

Chapter 7
Understanding the Workings of the Global Ecosystem Architecture
Steven Newhouse

Looking Under the Hood of the Grid Infrastructure                p.196
The Grid Services Software Stack
Decides What Happens Where and When                        p.201
Jumping into the GLIF Presentation on EGI                p.202
Grids in North America?                                     p.206

Chapter 8
BiG Grid as the Dutch
National e-Science Infrastructure
Arjen van Rijn
The Dutch Eco-system and its Evolution                        p.211
Organizing Large Scale Collaboration                            p.213
Why Collaboration?                                        p.215
A Business Model?                                        P.216
Summing Up -- Big Grid Information, Infrastructure,
and User Communities                                    p.220
Alexandre Bonvin’s Netherlands - Italian e-NMR GRid Project        p.224

Chapter 9
Distributed ASCI Supercomputer (DAS)
Henri Bal and the Computer Science Testbed/Grid

Editor’s Introduction                                         p.226
DAS-3 Runs Independently from Big Grid                        p.229
IBIS - a Language for Real World Distributed Computing            p.232
Demonstration – Distributed Hierarchical Heterogeneous Computing    p.234

Chapter 10
Life Sciences as an Immature Field
An Introduction from Bob Hertzberger

Genomics Is Relatively New in the Netherlands             p.246
Life Sciences – a Young and Not Yet Stabile Field                p.249
Lacking Enough Understanding to Enable Predictive Models        p.252

Chapter 11
Looking for Structure in the Life Science Wars
Timo Breit’s Micro-array Transcriptomics

The Many Nuances of Eco System Thinking                p.255
Bringing Computational Science to Bear
on a Problem of Unbearable Complexity                        p.260
Onward to e-Science                                        p.263

Chapter 12

The Gap
The Hard Place Between Biology, Systems
Biology and Economic Benefit  - Brian Hanley

Editor’s Introduction                                       p.274
A Different Point of View                                         p.275
The Computing Environment of Biology                             p.276
Genomics as a focus of Systems Biology                            p. 277
The Medical-Industrial Complex and Economic Payoff to Society         p. 281
A Very Short Biography and Comments on My Doctoral Program         p. 284
My Point of View on the Preceding Two Chapters                        p. 287

Chapter 13
Ecologies of Birds, Plants and Animals
Willem Bouten Computational Geo Ecology
Computational Geo-Ecology                                p.293
The Fly Safe Bird Avoidance Model                            p.295
The Military Radars                                        p.299
Tracking with GPS Backpacks Mounted on Birds                    p.302
Bird Migration                                            p.304
Our Specific Bird Tracking Projects                            p.306
Data-driven Models - Migration of Storks                          p.308
Visualization of Our Data                                    p.315
Let’s come back to Our Other Virtual Laboratories                  p.316
Exploring the Economic Implications                            p.320

Chapter 14
Virtual Collaboration for Research in
Humanities and Science
The Virtual Research Environments Program at
JISC For Universities in the United Kingdom - F. Van Till
The Role of JISC in UK Higher Education                        p.326
VRE - Presenting the Three Phases                              p.328
What is a Virtual Research Environment?                          p.329
First phases, VRE1 and VRE2                                p.330
Documenting the Projects as a Teaching Tool                      p.334
VRE2 - Focusing on Four Pilots                              p.335
The Four Stage Two Pilots                                  p.336
Acquiring Users via Delivery of
Productive Easy-to-use Tools                                     p.338
SDM VRE - VRE for the Study of Documents and Manuscripts          p.340
My Experiment:   well-structured science                         p.342
CREW - Collaborative Research Events on the Web                  p.344
Small Tools                                               p.349

Chapter 15
Designing the Network for the
Mother of All Grids
David Foster on the World Wide LHC Computing Grid OPN
A Star Network Where its Strength Lies in its Simplicity                  p.355
The Complexity of the Tier-2s                                    p.363
Federated Flexibility                                            p.365
How the New Architecture Was Nurtured                              p.366
An Open-Ended Model                                          p.370
LHC Open Network Environment                                 p.371
Access methods                                            p.371

Chapter 16
At a Critical Moment in American History New R&E Leadership at Internet2
Interview with I2 CEO Dave Lambert

Early Career                                            p.372
Owning and Lighting Fiber                                    p.375
The Way Forward                                        p.377
Collaboration Tools and Grid Middleware                          p.379
Technology and a New Culture of Learning                      p.381
The US UCAN Opportunity                                  p.383
Extending a Helping Hand                                   p.385
Role of the Incumbents in U.S. UCAN                            p.386

Chapter 17

Conclusion -- At a very Difficult Time, the
Necessity of Informed Choice

Communities will Need the Network Resources                             p.390
R&E Networks Married to Political Reality                             p.398
Ask Yourself Where You Stand                                     p.399

Chapter 18
Fast Thinking: US  UCAN’s
Investment in Our Own Communities

A National Strategic narrative                                p.402
A Non Partisan Blueprint for
Understanding Changes of 21st Century                         p.403
At Stake is the Future of Local
Higher Education, Libraries and Schools                        p.408
Wanted: a US UCAN Strategy                                p.409
Fast Strategic Thinking Needed                                p.410

Chapter 19
Internet 2 Spring Meeting Shows the Challenge Ahead
The Utah-MCNC October 2010 Strategy Meeting                         p.415
Thanks to SURF and SURFnet for their Support                         p.417

Executive Summary                                 p.418

What’s Happened
In Whose Interest?                                        p.421

Postscript from Thailand                        p.423

(A) Some Data  on StarPlane and Das-3 Projects             p. 427
Das-4 Overview                              p. 429
Relevant URLs from Late March to Early 
April 2011                                     p. 433