Storage

Overview

The HPC-AI.com platform provides two main types of storage: local storage and remote storage. Each type is optimized for different use cases and persistence levels.

• Local Storage: Automatically created when an instance starts and tied to its lifecycle.
• Remote Storage: A separate service that needs to be ordered and can persist independently of instances.

Storage Types

Local Storage

Local storage is directly attached to the instance and automatically created and managed with the instance's lifecycle. There are two types of local storage:

1. System Disk: Stores the operating system and environment files. Persistent even if the instance is stopped.
2. Data Disk: Temporary storage for runtime data. Not persistent if the instance is stopped.

Storage Type	Path	Speed	Size
System Disk	Includes: /bin, /home, /lib, /lib64, /media, /mnt, /opt, /root, /sbin, /srv, /usr, /var, /etc	Normal	50 GB NVMe per CPU
Data Disk	/root/dataDisk	Fast	2000 GB NVMe per GPU or 50 GB per CPU

What do the paths indicate?

The paths listed in the table correspond to directories within the instance file system, providing storage locations for different purposes:

• System Disk: Includes critical directories like /bin, /usr, and /var that store essential operating system and software files.
• Data Disk: /root/dataDisk is a dedicated location for temporary files and datasets that are specific to the current instance runtime.

How should you use the Data Disk?

To optimize the usage of the data disk, store files that are frequently used or generated during the instance runtime, such as temporary data or intermediate results. Avoid saving important or persistent data, as the data disk is not persistent—its contents are lost when the instance is stopped. Use shell commands or programs to directly access /root/dataDisk for fast read and write operations.

Example Data Disk Usage:

# Save a temporary file to the data disk
echo "temp data" > /root/dataDisk/temp_file.txt

# List all files in the data disk
ls /root/dataDisk

Tip: Regularly back up any important data from the data disk to a persistent storage solution, such as remote storage, to prevent accidental data loss.

Remote Storage

Remote storage provides persistent storage independent of the instance lifecycle, making it suitable for long-term or shared storage requirements. It need to be explicitly ordered and is managed separately from the instance.

• File System: A reliable storage solution offering normal I/O performance. It provides persistent storage that remains intact even if the instance is terminated. The File System can also be shared across multiple instances, making it ideal for collaborative or multi-instance workloads.

• High-Speed Storage: Optimized for high-performance read and write operations, this storage type is suitable for distributed workloads and tasks with high I/O requirements. It is preferred for scenarios such as large-scale data processing or multi-instance training.

Storage Type	Path (Custom Mount Point)	Speed	Size
File System	Defined during creation, see steps below for instruction	Normal	On demand
High-Speed Storage	Defined during creation, see steps below for instruction	Fast	On demand

How to Mount Remote Storage to a Path?

To use remote storage effectively, follow these steps to mount it to a directory on your instance:

Steps:

1. Order Remote Storage: If no remote storage service has been started, create a remote storage (File System or High-Speed Storage) through the platform.
2. Configure Mounting Options: When creating new storage, select "Attach a New Storage", then:
   • Specify a storage name.
   • Enter the storage path on your instance where the storage will be mounted.

Illustration:

The image above demonstrates how to create and attach remote storage.