Introduction to grid technology

Contents:

   1. What is a grid?
   2. Why use a grid?
   3. Who should use a grid?

1. What is a grid?

The grid is a service for sharing computer power and data storage capacity over the Internet. It goes well beyond simple communication between computers, and ultimately aims to turn networks of compute and storage devices into one vast resource for solving large-scale, compute- and data-intensive applications.

The compute and storage resources in a grid are distributed, but the users are able to access these resources in a transparent way without having to know the physical location of the resources. Special software, so-called middleware, is required for the operation of the grid and to provide a consistent, dependable service across the Internet. Grids are arguably known as the next evolution in distributed computing and are used to optimize resource utilization and boost the aggregate capacity for data-intensive and loosely coupled parallel computations, sometimes by several orders of magnitude.

There is no single grid. There exist many grids and many types of grids. A network of compute resources form what is known as a computational grid. Users can submit computational jobs from their local desktop that are automatically and transparently started at the most suitable resource, irrespective of where it is located geographically and organizationally. Computational grids are usually for high-throughput computation to quickly process large numbers of loosely-coupled or independent serial computations. This in contrast to e.g., national HPC resources with a tight coupling between the compute elements which are designed to run large multi-processor (parallel) jobs. A data grid aims to combine multiple heterogeneous storage resources into one big storage resource that enables users to share and manage data irrespective of the location of the data. And obviously there exist grids that are a combination of a computational and data grid or that target a specific class of applications.

  • More introductions to grid computing can be found here
  • Grid computing according to Wikipedia
  • A long index of acronyms used in the grid world: Grid Acronym Soup

2. Why use a grid?

Grids combine the capacities of multiple computers to create a massively powerful and fully comprehensive compute resource. For scientists trying to solve extremely complex problems, such aggregated resource provides the capacity to help solve their questions. Grids also accelerate research activity: a simulation that might take weeks on a single compute resource may complete in hours on a grid. Grids facilitate quickly changing needs and provide a resource for time-critical simulations or peak-demands. As such, grids are not only of interest to users (researchers), but also to resource providers as grids help in optimizing and managing the workload across multiple resources.

Grids virtualize storage resources and data. Grids provide advanced services for data management and for sharing data collections across the Internet. Researchers need not be bothered with complex data flows to and from compute resources and need not keep track of the physical location of data sets. Data sets may be partitioned over multiple locations and several storage media. In grid environment, researchers can access files and other data objects with logical names. The grid middleware takes care of mapping these logical names onto the actual physical locations where the data is stored.

Another key motivation for using grids is to enable collaboration between communities which may be geographically dispersed, but which are organized into so-called Virtual Organizations (VOs). Simply stated, a VO is a group of individuals or institutions who share resources of the grid for a common purpose, e.g., a research project. VOs are becoming fundamental to much of modern computing. VOs enable disparate groups of organizations and/or individuals to share resources in a controlled fashion, so that members may collaborate to achieve a shared goal.

3. Who should use a grid?

Grid environments can be used to enable distributed applications with complex workflow or data management. You should consider using a grid environment in case

  • you have a large computational problem that can be partitioned into smaller independent tasks that can be distributed across multiple compute resources
  • you have temporary peak-demands for compute or storage resources that cannot be handled by a single resource alone
  • you have a complex application where users, applications, input/output data, compute and storage resources and other relevant services are at different locations
  • you want to have a working environment where you need not be bothered with scanning resources to find unused compute or storage capacity; you would like the grid middleware to take care of this and minimize job turn-around time