Skip to content

Commit

Permalink
Update storage-locations.md
Browse files Browse the repository at this point in the history
  • Loading branch information
sellth authored Dec 2, 2024
1 parent c8c025b commit 9656ccd
Showing 1 changed file with 7 additions and 10 deletions.
17 changes: 7 additions & 10 deletions bih-cluster/docs/storage/storage-locations.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,6 @@
This document describes the forth iteration of the file system structure on the BIH HPC cluster.
It was made necessary because the previous file system was no longer supported by the manufacturer and we since switched to distributed [Ceph](https://ceph.io/en/) storage.

!!! warning "Important"
For now, the old, third-generation file system is still mounted at `/fast`. **It will be decommissioned soon, please consult [this document describing the migration process](storage-migration.md)!**

## Organizational Entities
There are the following three entities on the cluster:

Expand Down Expand Up @@ -128,7 +125,7 @@ $ cp important_file.txt /data/cephfs-2/unmirrored/groups/cubi/

## Technical Implementation
### Tier 1
- Fast & expensive
- Fast & sparse
- mounted on `/data/cephfs-1`
- Currently 12 Nodes with 10 × 14 TB NVME SSD each
- 1.68 PB raw storage
Expand All @@ -137,16 +134,16 @@ $ cp important_file.txt /data/cephfs-2/unmirrored/groups/cubi/
- For typical CUBI use case 3 to 5 times faster I/O then the old DDN
- Two more nodes in purchasing process
- Hardware costs:
- One node/chunk: 45.000 € (150 TB)
- ca. 300 €/TB
- One node/chunk: 50.000 € (150 TB)
- ca. 350 €/TB

### Tier 2
- Slower but more affordable
- Slower but much more affordable
- mounted on `/data/cephfs-2`
- Currently 10 nodes with 52 HDDs slots and SSD cache (~40 HDDs per node with 16–18 TB capacity)
- 6.6 PB raw storage
- 5.3 PB erasure coded (EC 8:2)
- 4.5 PB usable (85 %; Ceph performance limit)
- 8.7 PB raw storage
- 7.0 PB erasure coded (EC 8:2)
- 5.9 PB usable (85 %; Ceph performance limit)
- More nodes in purchasing process
- Hardware costs:
- ca. 50 € per TB, 100 € mirrored
Expand Down

0 comments on commit 9656ccd

Please sign in to comment.