About Data Science Storage

Highlights

  • Economical TCO
  • Access from LRZ HPC Ecosystem
  • Uses High Speed HPC Backbone
  • High Speed access to LRZ Backup
  • Managed by LRZ
  • Modular Design
  • Scales to tens of PBs and GB/s
  • LRZ to Customer UID conversion
  • Optional data mgmt. features

Over the last few years the amount of data generated in various science areas exploded and with it satisfying the requirements to store, analyze, manage, protect and retain these data oceans became more and more challenging. This trend is often referred to as Big Data and can be seen as transforming data into information then to knowledge and finally to wisdom. What the term Big Data hides, is that each of these transformation steps requires computing. So ultimately, the process of generating new insights from Big Data requires also Big Computing.

While LRZ as an HPC provider has a decent amount of compute resources, today the data is often stored locally in the science departments. This arises the problem that we have to bridge the gap between Big Data and Big Compute by the use of Wide Area Networks, which is an approach that is not likely going to scale. Therefore, we propose to consolidate your Big Data and LRZs Big Compute resources under one umbrella by using our Big Data Storage Solution called Data Science Storage (DSS).

Economical Total Cost of Ownership (TCO) 

The prices for DSS include all required hard- and software as well as 5 years of vendor maintenance, LRZ operations, data center space, energy and cooling. Even the starting price of a minimal DSS System is nearly half of the costs per TB and year of comparable Cloud offerings for Disk Storage.

Accessibility

Access to DSS can be made available as direct file system mount on the following LRZ HPC Systems:

  • Linux Cluster (Login Nodes)
  • SuperMUC (Login Nodes)
  • LRZ Hosted Clusters
  • LRZ Compute Cloud
  • LRZ Visualization System (coming soon)

Accesses to DSS from outside of the LRZ is possible via a GridFTP gateway.

Managed by LRZ

DSS is fully managed by LRZ. You do not have to worry about: 

  • Tender
  • Shipment and physical installation
  • Data Center Infrastructure
  • Hardware Maintenance
  • Software Setup and Maintenance
  • Tuning
  • Monitoring
  • Backup

Optional Data Management and Protection Features

Optionally you can leverage advanced data management and protection features like:

  • Hierarchical Storage Management
  • High Performance TSM Backup
  • Snapshots
  • Quotas
  • Replication 
  • WAN Caching
  • SSD Acceleration Cache

Architecture

DSS was designed with modularity and scalability in mind. The minimal DSS configuration consists of a redundant pair of File System Modules, which are connected to a Storage Module in the backend and a Network Module in the frontend.

HPCbackbone

Depending on the drive type, this configuration delivers between 165TB and 247TB usable capacity at 2GB/s sustained sequential processing throughput. Depending on your needs you can add more Storage Modules to increase capacity by 165TB or 247TB increments or add more Network and/or File System Modules to increase performance. LRZ will help you to mix and match the modules to fit your needs.

HPCbackboneerweitert

In order to glue all modules together into a single big data file system namespace and to provide optional advanced data management features like ILM, a Software Defined Storage (SDS) Product is leveraged.

Get in touch

If you are interested in this offering please don't hesitate to reach out to us via the LRZ Servicedesk.

Library

LRZ-Gebäude