Virtual Clusters of Mobile and Limited Devices
(extending the Grid towards the lower end of the performance spectrum)

Note: This is only a temporary page until the Virtual Clusters Wiki site becomes operational

The Virtual Clusters Project

The Virtual Clusters platform provides an efficient way for integrating mobile and resource-limited devices into Grid systems as providers and not just consumers. A fully functional prototype has been developed and tested under a variety of conditions and scenarios. Find out more by following the links:

- Objectives & Challenges

- Architecture

- Lightweight Server-side Container Framework

- Tools

- Application Domains

- Publications

- Presentations

- Related Research and Resources



Objectives
The objectives and motivation of the Virtual Clusters project is to:
  • Make future Grid systems truly ubiquitous and pervasive
  • Exploit functionality not available in traditional Grid nodes (such as context aware applications, multimedia equipment, sensor networks, Global Positioning Systems and more)
  • Take advantage of the huge abundance of mobile resources to create ad hoc instant high performance computing facilities, for instance in conference rooms or Wi-Fi hot spots
  • Reach geographical areas where traditional infrastructure cannot be deployed
  • Provide Grid resources as a backbone for organizations and enterprises that cannot deploy standard Grids (for field operations, personnel and fleet management, and more)
Challenges and Requirements
The challenges that arise when trying to “merge” two such different domains (in terms of reliability, availability, performance, and flexibility) evolve around the resource limitations of mobile devices, and the inherent dynamicity. A bit more specifically:
  • Current Grid middleware cannot be deployed in resource constrained devices -too many resources required, many dependencies, complex installation. A more lightweight service container solution is required
  • Even with a suitable container in place, we cannot just incorporate each mobile device as a single full member of the Grid: too much dynamicity, many heterogeneity issues, wide range of possible failures (connectivity, battery dependence, etc). As more diverse and unreliable devices join the Grid, the average reliability and availability decreases.
  • Programming in such a dynamic environment can be a daunting task: abstractions are required, as well as a subsystem to handle leveraging of resources towards higher level programming models
  • The nature of the mobile domain mandates techniques to handle dynamic addressing, mobility, and high availability
  • Energy issues and resource limitations require special, optimized communication models
Top
Architecture
The Virtual Clusters architectural design focuses on providing the means to overcome the resource limitations, dynamicity, and mobility that humble the mobile domain. The main conceptual idea is to hide the big cardinality of the mobile domain by exposing all mobile devices as a single virtual entity, thus enabling a smooth integration and reduce the administrative and performance overhead. In order to present a group of devices as a single entity, a representative proxy node is required. Hence, the VC approach is a proxy-based, partially centralized design, where the grid system is merely extended to include a single extra node: the proxy. The latter acts as the gateway for the underlying group of mobile devices, which is now coined a Virtual Cluster.

Architecture

The main entity in the realization of a Virtual Cluster is the aggregator service (or simply ‘Aggregator’). The Aggregator resides at the proxy node, and is responsible for a group of mobile devices that provide a similar resource or service (where similarity is determined by the implemented interfaces). Aggregators expose single, consistent, and permanent interface points to functionality available in the fabric service layer.

Proxy

The Virtual Clusters platform focuses solely on overcoming the identified challenges. More specifically:
  • Special communication models ensure the optimized and resource efficient communication between the platform services and the fabric layer
  • The Virtual Cluster group abstraction encapsulates the unwanted properties of the mobile domain, and exposes consistent and reliable interfaces to potentially unreliable and mobile fabric services. Failure masking and recovery techniques within the Virtual Cluster present a virtually stable and highly available entity to the higher layers (Application, User Tools etc.)
  • The Aggregators are dynamically reconfigured at run-time in response to events such as service failure, device mobility, resource depletion and more, to provide full state awareness towards a “lookahead” status that supports the high-availability requirements for the Virtual Cluster. Supporting the dynamic reconfigurability of Aggregators, there are a number of components handling dynamic fabric service discovery and activation, dynamic, private, or NAT-based addressing, and more
  • Mobility and Communication Loss prediction mechanisms ensure that pro-active action will be taken whenever necessary in order to ensure the sustainability of application execution and the high-availability properties of the Virtual Cluster
  • The introduced programming abstractions, a set of value-adding collective interfaces, and the platform API, provide enhanced functionality and simplifcations towards rapid development of high-level applications, service composition, and mashup-like entities
Aggregator

Top
Lightweight Server-side Container (LSC) Framework
Accompanying the Virtual Clusters platform, the Lightweight Server-side Container (LSC) framework, provides a lightweight service hosting environment, where service providers can easily expose their Java-based components/applications/beans. Through the J2ME-compliant graphical user interface and a straightforward one-off process, a provider selects which services he wishes to share.

LSC Gui

Apart from proprietary solutions, which of course are not suitable for hybrid service oriented systems, the only other approaches are RMI for limited devices, and Mobile Web Service frameworks. The latter is very heavy for resource constrained devices, and is humbled by performance issues, while the RMI-OP requires complex installation procedures, a running local RMI registry, and has a number of restrictions. The LSC has no software dependencies, no complex installation procedures and no heavy resource requirements, thus making it suitable for limited devices such as mobile phones. Further, as it is based on pure low level TCP sockets, it is very fast compared to other service frameworks.

lsc

The drawbacks of low level socket implementations are counter balanced in the context of a Virtual Cluster, since functionality available in mobile devices is accessed through standards-based, high-level Aggregators.

In summary, the LSC framework manages to marry the high performance of low-level TCP sockets with the interoperability of the standards-based Web Services paradigm through the high-level interfaces of Aggregators.
Top
Administrator Tools
A number of administrator tools have been developed to support the configuration, troubleshooting, and management of Virtual Clusters, the most significant of which are the Management Console, and the Configuration Panel. The latter, is a graphical user interface to configuring various parameters of the Virtual Cluster platform upon installation and deployment.

GUI 1

The Management Console provides an invaluable tool that enables the monitoring of active fabric services, currently deployed Aggregators, views of the resource and service index repositories, views and testing of the service interfaces (including the collective and asynchronous interfaces) and more.

GUI 2

Top
Application Domains and Use Cases
Applications that can benefit from this integrated model are those that operate near the blurry boundaries of the mobile and grid domains. A grid infrastructure at the back can provide a vast amount of resources such as databases, high performance computing facilities, specialized hardware and software, and more. At the same time, the mobile devices with their unique functionality can extend the grid system to previously unreachable geographical areas, provide flexibility, and enable a variety of application perspectives. Concrete examples may include (but are of course not limited to):
  • Hospital management systems: A grid backbone at the main hospital site hosts the data repositories, the patient records, and the master control system. Hospital personnel carry with them highly personalized personal devices such as PDAs. The network accessibility of the mobile devices enables the increased efficiency and automation of operations such as record updating; emergency response and staff allocation can take full advantage of an integrated environment.
  • Distributed Data collection from Mobile Sources: Police patrol units, emergency response units, industrial site monitoring units all have a common characteristic: the need for accurate and up-to-minute information flows from multiple sources. An integrated model can allow the automatic binding and utilization of nearby mobile data sources such as sensory units, monitoring equipment such as cameras and audio capturing, motion detectors and more. Collective and transparent management of all sources is simplified with the utilization of a hybrid platform
  • Fleet management: Companies and organizations that have outdoors operations, for example delivery companies, large warehouse control companies, hospital ambulance fleet, and more; need to manage efficiently their personnel and mobile entities. An integrated platform can provide support for the easy and efficient monitoring, management, event response situations, dynamic and automatic staff allocation and scheduling, better team cohesion and cooperation and more.
  • Integration of sensors and sensor farms: Sensors and sensor networks are extensively used in a wide variety of applications ranging from the medical (e.g. patient monitoring) and the security domain (e.g. industrial site monitoring, alarm notifications) to traffic management and geophysical applications (e.g. earth and weather monitoring). The integration with a grid system will provide the storage and computational analysis resources required for sensor data processing.
  • Context aware mobile applications: Mobile users can form communities where they can exchange information and content in a context aware but location and address transparent way. For instance, members can publish information about potential traffic congestion problems, or recommend events and places to go in a location and time sensitive manner.
Top
Publications (See also related research)
  • S. Isaiadis, V. Getov, I. Kelley, I. Taylor, “Dynamic Service-based Integration of Mobile Clusters in Grids“, in Grid Computing: Achievements and Prospects, Springer, pp. 159-171, 2008
    [Bibtex]
  • S. Isaiadis and V. Getov, “Evaluation of Dynamic Clustering Architecture for Utilising Mobile Resources“, in Proceedings of the IASTED International Conference on Parallel and Distributed Computing and Networks (PDCN 2008), Insbruck, Austria, pp. 117-125, 14 Feb 2008
    [Bibtex]
  • S. Isaiadis and V. Getov, “Dynamic Service Aggregation in Heterogeneous Grids“, in CoreGRID Workshop on Grid Programming Model, Grid and P2P Systems Architecture, Grid Systems, Tools and Environments, Heraklion, Greece, TR-0080, Jul 2007
    [Bibtex]
  • S. Isaiadis, “Going Mobile: Extending the Grid to the Lower End“, in International Science Grid this Week, Article by Cristy Burne, 6th June 2007
    [Download]
    [Bibtex]
  • S. Isaiadis and V. Getov, “Dependability in Hybrid Grid Systems: a Virtual Clusters Approach“, in Proceedings of the IEEE JVA International Symposium on Modern Computing 2006, Sofia, Bulgaria, pp. 52-57, Oct 2006
    [Bibtex]
  • S. Isaiadis and V. Getov, “A Lightweight Platform for Integration of Mobile Devices into Pervasive Grids“, in Proceedings of High Performance Computing and Communications, Sorrento, Italy, pp. 1058-1063, Sep 2005
    [Bibtex]
  • S. Isaiadis and V. Getov, “Integrating Mobile Devices into the Grid: Design Considerations and Evaluation“, in Proceedings of the 11th International Euro-Par Conference on Parallel Processing 2005, Lisbon, Portugal, pp. 1080-1088, Aug 2005
    [Bibtex]
Top
Presentations
  • 12 Feb 2008, “Evaluation of Dynamic Clustering Architecture for Utilising Mobile Resources“, in IASTED International Conference on Parallel and Distributed Computing and Networks, Innsbruck, Austria
  • 13 Jun 2007, “Dynamic Service Aggregation in Heterogeneous Grids“, in CoreGRID Workshop, Heraklion, Crete, Greece
  • 15 Oct 2006, “Dependability in Hybrid Grid Systems: the Virtual Clusters Approach“, in IEEE JVA International Symposium on Modern Computing, Sophia, Bulgaria
  • 23 Sep 2005, “A Lightweight Platform for Integration of Mobile Devices into Pervasive Grids“, in High Performance Computing and Communications Conference, Sorrento, Italy
  • 31 Aug 2005, “Virtual Clusters for Integration of Resource Limited Devices into the Grid“, in EuroPar Conference on Parallel Computing, Lisboa, Portugal
  • 13 Jun 2005, “A Lightweight Platform for Integration of Resource Limited Devices into the Grids“, in 2nd WP7 CoreGRID Workshop, Universitat Polytechnica Catalonia, Barcelona, Spain
Top