GARUDA Data Management
Data management in Grid is crucial considering the large quantities of data generated from scientific instruments and long running computational simulations. Data grids are federated collection of distributed and heterogeneous storage resources. They provide a Global name space for storage entities. The Global Namespace is a hierarchical organization of each and every entity present as data on grid, independent of their physical storage infrastructure. The storage requirements for such a dynamic environment are highly challenging. Data solutions for grid should address Virtualization, Data availability, Security, Performance & Latency and QoS.
To support data oriented applications, Data management in GARUDA was handled through Storage Resource Broker (SRB) from Nirvana.
In PoC phase of GARUDA, SRB was deployed across four GARUDA sites Bangalore, Pune, Chennai and Hyderabad, with total storage capacity if 2TB. Initiatives have been taken for an alternate data solution for GARUDA. Various public domain data grid solutions like iRODS, Grid File Systems, WS-DAI, SRM has been surveyed and their suitability for GARUDA applications was evaluated. SRM has been found to be the most suitable and optimal data solution for GARUDA.
We are working on a SRM based Storage Resource Manager, GARUDA SRM (GSRM). GSRM adheres to OGF standards and is based on Disk Pool Manager (DPM) SRM implementation. The GSRM services provide Web Service interfaces supporting dynamic space management, security, data transfer, directory functions and file management and supports interoperability between various SRM implementations.
Features supported by GSRM are as follows:
- Unified Namespace: Provides single point storage access for users by providing global namespace
- Security: Supports GSI, Kerberos 5, Access Control Lists, and VOMS Security mechanism and supports secure data transfers.
- User Quota Allocation: Each user is allocated 10GB of default, which can be increased based on the user request.
- Space Management: GSRM supports for dynamic space management by Reservation, release and setting lifetime for files.
- File and Directory management: It supports upload /download of files in spaces, pinning & release of files.
- Access remote sites for files :
- Bring files from other sites and SRMs as requested
- Use existing transport services (GridFTP, https, rfio)
- Transfer protocol negotiation
- File systems support: GSRM can be integrated with high performance parallel file systems (e.g, PVFS, GPFS) and standard POSIX file system (e.g, ext2) as well.
- Interoperability: GSRM interoperates with other SRM implementations like StoRM, Bestman, dcache
GSRM User Interfaces:
GSRM can be accessed using the three basic interfaces: Command Line, APIs, and Web Interface. GSRM is interoperable with other SRM implementations, as it adheres to standard specifications. Hence it supports the client request from any of the SRM implementation. Users can access GSRM from any of the GARUDA head nodes using the command line and API interfaces.
The pre-requisites for a user to access GSRM are:
- GARUDA head node login.
- A valid GARUDA grid certificate
- Authorization for GARUDA storage resources (given by the GSRM administrator)
- SRM client interface installation
Support & Queries:
For any queries send a mail to: firstname.lastname@example.org