Home/Platform
Storage, network & managed operationsThe platform that keeps expensive GPUs busy
Accelerators are the easy part. Velyrix builds the storage, fabric, scheduling and observability layers that decide whether your cluster runs at 30% utilisation or 55% — and operates all of it 24×7.
Feed the GPUs, checkpoint without stalling
A 512-GPU job writing a multi-terabyte checkpoint will expose any weakness in the storage path. Velyrix sizes storage against your checkpoint interval and dataset read pattern, not against a capacity number on a purchase order.
| Tier | Technology | Typical performance | Use case | From (per TB/mo) |
|---|---|---|---|---|
| Scratch | Local NVMe Gen4/Gen5 | Up to 60 GB/s per node | Shuffle space, dataloader cache, temp | Included |
| Parallel FS — Performance | WEKA or VAST on NVMe | 1–10 TB/s aggregate read | Training datasets, checkpoints | $42 |
| Parallel FS — Capacity | Hybrid NVMe + HDD tier | 200–800 GB/s | Cold epochs, dataset archive | $22 |
| Block | NVMe-oF replicated | Up to 1M IOPS per volume | Databases, vector stores, boot volumes | $101 |
| Object | S3-compatible, erasure coded | Multi-GB/s, 11 nines durability design | Data lake, model registry, artefacts | $20 |
| Archive | Cold object / tape gateway | Retrieval in minutes to hours | Retention, compliance, raw corpora | $6 |
Indicative list pricing, exclusive of tax. Parallel file systems are sized and quoted per cluster; performance figures depend on the deployed configuration.
Three planes, never sharing a bad day
- Compute plane: NVIDIA Quantum-X800 XDR 800G or Quantum-2 NDR 400G InfiniBand, rail-optimised, 1:1 non-blocking, SHARP enabled
- Ethernet alternative: NVIDIA Spectrum-X with RoCEv2, adaptive routing, congestion control and BlueField-3 DPU offload
- Storage plane: dedicated 200/400G so checkpoint writes never collide with gradient all-reduce
- Management plane: isolated out-of-band BMC network with jump-host access and full audit
- External: dual-homed 100G transit, DDoS mitigation, BYOIP and BGP sessions on request
- Interconnect: private links to AWS, Azure, Google Cloud and Oracle, plus dark fibre between Velyrix campuses
Reference topology for a 512-GPU pod. Larger fabrics add a third tier; the rail-optimised principle is unchanged.
A NOC that understands NCCL, not just ping
24×7 NOC
A 24×7 on-call rotation with a contractual 15-minute P1 response, GPU-literate engineers and direct escalation to the team that built your cluster — not a first-line script.
Observability
DCGM, node exporter, fabric counters and job telemetry in Prometheus and Grafana, with per-job attribution and export to your SIEM.
Proactive health
Automated node health checks between jobs, XID and ECC trend analysis, and pre-emptive drain and replace before a run is lost.
Lifecycle management
Firmware, driver and engine upgrades validated in a staging pod first, then rolled through production in agreed windows.
Scheduler operations
Slurm and Kubernetes tuning, queue and quota policy, fair-share configuration, and utilisation reporting per team or cost centre.
SLA reporting
Monthly availability, incident and capacity reports measured against contract, with credits applied automatically when targets are missed.
Isolation you can evidence
- Tenancy: single-tenant physical hosts, dedicated VLAN/VRF and dedicated InfiniBand partitions
- Identity: SAML/OIDC single sign-on, SCIM provisioning, hardware MFA, scoped API keys and role-based access
- Keys: customer-managed encryption keys with HSM backing; Velyrix cannot read tenant volumes
- Data at rest: AES-256 on block, object and parallel storage tiers
- Data in transit: TLS 1.3 externally, optional MACsec and IPsec on private links
- Firmware integrity: secure boot, signed firmware baselines and attestation on every provision
- Decommission: NIST SP 800-88 purge with certificate of sanitisation, or customer-witnessed destruction
- Logging: immutable audit trail of every console, API and physical access event
provider "velyrix" { region = "us-east-1" } resource "velyrix_cluster" "training" { name = "acme-pretrain" node_type = "hgx-h200-8g" node_count = 64 fabric = "infiniband-ndr" scheduler = "slurm" storage { parallel_fs_tb = 800 object_tb = 2000 } # 512 GPUs, one apply, validated on delivery }
Own the GPUs. Let us run them.
Buy your NVIDIA servers from any OEM or distributor you like, ship them to a Velyrix hall, and we handle the rest — deployment, fabric, cooling, monitoring and support. Or rent ours. Either way, you get a plan in one business day.
Frequently asked questions
What storage should we use for large training runs?
A performance parallel file system - WEKA or VAST on NVMe - for active datasets and checkpoints, backed by S3-compatible object storage for the wider data lake, with local NVMe scratch on each node for shuffle space. Velyrix sizes the performance tier from your checkpoint size and interval so that checkpointing never dominates step time.
Do you support Ethernet instead of InfiniBand?
Yes. NVIDIA Spectrum-X with RoCEv2, adaptive routing and congestion control is available where a single Ethernet operating model is preferred. InfiniBand remains the default for the largest training fabrics because of SHARP in-network reduction and mature subnet management.
Can we connect a Velyrix cluster to our existing cloud environment?
Yes, through private interconnects to AWS, Azure, Google Cloud and Oracle Cloud, IPsec or MACsec-protected links to your own data centres, and cross-connects in our carrier-neutral meet-me rooms.
Who operates the cluster day to day?
You can operate it yourself with full root and BMC access, or hand operations to Velyrix under a managed service covering monitoring, incident response, firmware and driver lifecycle, scheduler policy and SLA reporting.