What do you think of e-ScienceCity? Click here!

Building a grid


"Middleware" is the software that organizes and integrates the resources in a grid.  

Middleware is made up of many software programs, containing hundreds of thousands of lines of computer code. Together, this code automates all the "machine to machine" (M2M) interactions that create a single, seamless computational grid.


Middleware automatically negotiate deals in which resources are exchanged, passing from a grid resource provider to a grid user. In these deals, some middleware programs act as "agents" and others as "brokers".

Agent programs present "metadata" (data about data) that describes users, data and resources. Broker programs undertake the M2M negotiations required for user authentication and authorization, and then strike the "deals" for access to, and payment for, specific data and resources.

Once a deal is set, the broker schedules the necessary computational activities and oversees the data transfers . At the same time, special "housekeeping" agents optimize network routings and monitor quality of service.

And all this occurs automatically, in a fraction of the time that it would take humans at their computers to do manually.


There are many other layers within the middleware layer. For example, middleware includes a layer of "resource and connectivity protocols", and a higher layer of "collective services".

Resource and connectivity protocols handle all grid-specific network transactions between different computers and grid resources. For example, computers contributing to a particular grid must recognize grid-relevant messages and ignore the rest. This is done with communication protocols, which allow the resources to communicate with each other, enabling exchange of data, and authentication protocols, which provide secure mechanisms for verifying the identity of both users and resources.

The collective services are also based on protocols: information protocols, which obtain information about the structure and state of the resources on a grid, and management protocols, which negotiate uniform access to the resources. Collective services include:

  • updating directories of available resources
  • brokering resources (which like stockbroking, is about negotiating between those who want to "buy" resources and those who want to "sell")
  • monitoring and diagnosing problems
  • replicating data so that multiple copies are available at different locations for ease of use
  • providing membership/policy services for tracking who is allowed to do what and when.