"Middleware" is the software that organizes and integrates the resources in a grid. Middleware automates all the "machine to machine" (M2M) interactions that create a single, seamless computational grid.

A key ingredient of middleware is metadata: data that describes data. Metadata includes information about how and when data was collected, how the data is formatted, and where it is stored.

Middleware includes many software programs. For example, the European Data Grid project involves 150 software engineers and over 300,'000 lines of computer code.

AGENTS, BROKERS AND STRIKING DEALS

Some middleware programs act as "agents" and others as "brokers". Together, they automatically negotiate deals in which resources are exchanged, passing from a grid resource provider to a grid user.

Agent programs present metadata about users, data and resources. Broker programs undertake the M2M negotiations required for user authentication and authorization, and then strike the 'deals' for access to, and payment for, specific data and resources.

When a deal is set, the broker will schedule the computational activities and oversee the data transfers required. At the same time, special 'housekeeping' agents optimize network routings and monitor the quality of service.

And all this occurs automatically, in a fraction of the time that it would take humans at their computers to do manually.

DELVING INSIDE MIDDLEWARE

There are many other layers within the middleware layer. For example, middleware includes a layer of resource and connectivity protocols, and a higher layer of collective services.

Resource and connectivity protocols handle all "grid-specific" network transactions between different computers and grid resources. For example, computers contributing to a particular grid must recognize grid-relevant messages and ignore the rest. This is done with communication protocols, which allow the resources to communicate with each other, enabling exchange of data, and authentication protocols, which provide secure mechanisms for verifying the identity of both users and resources.

The collective services are also based on protocols: information protocols, which obtain information about the structure and state of the resources on the grid, and management protocols which negotiate access to resources in a uniform way. Collective services include:

• updating directories of available resources,
• brokering resources (which like stockbroking, is about negotiating between those who want to "buy" resources and those who want to "sell")
• monitoring and diagnosing problems
• replicating data so that multiple copies are available at different locations for ease of use
• providing membership/policy services for tracking who is allowed to do what and when.