From 9656ccdd20a14a1e6d207aa1e32545c689815ba0 Mon Sep 17 00:00:00 2001 From: Thomas Date: Mon, 2 Dec 2024 09:57:07 +0100 Subject: [PATCH] Update storage-locations.md --- bih-cluster/docs/storage/storage-locations.md | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) diff --git a/bih-cluster/docs/storage/storage-locations.md b/bih-cluster/docs/storage/storage-locations.md index 26a90e611..e73ad8556 100644 --- a/bih-cluster/docs/storage/storage-locations.md +++ b/bih-cluster/docs/storage/storage-locations.md @@ -2,9 +2,6 @@ This document describes the forth iteration of the file system structure on the BIH HPC cluster. It was made necessary because the previous file system was no longer supported by the manufacturer and we since switched to distributed [Ceph](https://ceph.io/en/) storage. -!!! warning "Important" - For now, the old, third-generation file system is still mounted at `/fast`. **It will be decommissioned soon, please consult [this document describing the migration process](storage-migration.md)!** - ## Organizational Entities There are the following three entities on the cluster: @@ -128,7 +125,7 @@ $ cp important_file.txt /data/cephfs-2/unmirrored/groups/cubi/ ## Technical Implementation ### Tier 1 -- Fast & expensive +- Fast & sparse - mounted on `/data/cephfs-1` - Currently 12 Nodes with 10 × 14 TB NVME SSD each - 1.68 PB raw storage @@ -137,16 +134,16 @@ $ cp important_file.txt /data/cephfs-2/unmirrored/groups/cubi/ - For typical CUBI use case 3 to 5 times faster I/O then the old DDN - Two more nodes in purchasing process - Hardware costs: - - One node/chunk: 45.000 € (150 TB) - - ca. 300 €/TB + - One node/chunk: 50.000 € (150 TB) + - ca. 350 €/TB ### Tier 2 -- Slower but more affordable +- Slower but much more affordable - mounted on `/data/cephfs-2` - Currently 10 nodes with 52 HDDs slots and SSD cache (~40 HDDs per node with 16–18 TB capacity) - - 6.6 PB raw storage - - 5.3 PB erasure coded (EC 8:2) - - 4.5 PB usable (85 %; Ceph performance limit) + - 8.7 PB raw storage + - 7.0 PB erasure coded (EC 8:2) + - 5.9 PB usable (85 %; Ceph performance limit) - More nodes in purchasing process - Hardware costs: - ca. 50 € per TB, 100 € mirrored