How to Manually Format, Mount, and Partition Volumes

DigitalOcean volumes are represented within the Droplet’s operating system as a raw storage device. This means that you can use conventional tooling to prepare the volume for use. However, there are some platform-specific details that are useful to know when working with DigitalOcean volumes.

The volume name has implications on how the device is represented by the operating system. In Linux, hardware devices are typically represented by special files within the /dev directory with names like sda and sdb. Subdirectories in the /dev/disk directory contain convenient symbolic links with more descriptive names in order to facilitate device identification.

Links within the /dev/disk/by-id directory are conventionally associated with hardware serial numbers, but on Droplets the link names are created by appending the volume’s name to a scsi-0DO_Volume_ prefix. Partitions on volumes are composed of this volume identifier and a further -part# suffix. This means that links within the /dev/disk/by-id directory are both predictable and reliable across boots. Because of these benefits, the links within /dev/disk/by-id are the recommended identifiers for DigitalOcean volumes and their partitions.

Most tools can take these identifiers as input, but will use the traditional /dev/sd* names in their output. To see how the /dev/disk/by-id links are mapped to their associated names, you can read the link targets. The file utility can display this cleanly:

file /dev/disk/by-id/*
/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01:       symbolic link to ../../sda
/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part1: symbolic link to ../../sda1

Because the /dev/sd* names can change between boots, you will need to check the links again after each boot.

Processes Required for Volumes

The processes required to prepare a volume for use on a Linux server depend on whether the volume has been partitioned and formatted previously.

Preparing New Volumes for Use within a Linux Server

Before you can begin using new DigitalOcean volumes, you need to:

  • partition the volume,
  • format the partitions,
  • create mount points,
  • mount the filesystems, and
  • adjust /etc/fstab to automatically apply the mounts during subsequent boots.

When running the following commands, change the highlighted volume-nyc1-01 references to the name you gave your volume. Use the /dev/disk/by-id identifiers because they are predictable based on the volume name and will always point to the correct volume, even after reboots.

First, create a single GPT partition that spans the entire volume (certain tools expect some type of partition table), substituting in the name of your volume.

sudo parted /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01 mklabel gpt
sudo parted -a opt /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01 mkpart primary ext4 0% 100%

Then format the partition with the Ext4 filesystem.

sudo mkfs.ext4 /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part1

Then create the mount point in /mnt.

sudo mkdir -p /mnt/volume-nyc1-01-part1

Adjust the /etc/fstab file to define a persistent mount.

echo '/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part1 /mnt/volume-nyc1-01-part1 ext4 defaults,nofail,discard 0 2' | sudo tee -a /etc/fstab

And finally, mount the filesystem.

sudo mount -a

Preparing Previously Formatted Volumes for Use within a Linux Server

If your volume was previously used and has already been partitioned and formatted, you will need to perform a subset of the above actions. To set your volume up on a different Droplet, you will need to:

  • create mount points,
  • mount the filesystems, and
  • adjust /etc/fstab to automatically apply the mounts during subsequent boots.

Partitioning and formatting are both destructive actions, so you should not attempt these steps on existing volumes unless you wish to erase their contents entirely.

After attaching an existing volume that has previously been partitioned and formatted, you will need to identify the filesystems available, mount them, and likely adjust the /etc/fstab file for persistence.

The identification process involves:

  • discovering the partitions and filesystems available
  • mapping the partition names to stable identifiers, so that they can be mounted reliably

To discover the partitions and filesystems on a volume, pass the volume identifier to the lsblk command. The /dev/disk/by-id identifier is a combination of the string /dev/disk/by-id/scsi-0DO_Volume_ (a static prefix) and the name of your volume. Request specific output fields with the -o option:

lsblk -o NAME,FSTYPE,SIZE,TYPE,MOUNTPOINT /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01
NAME   FSTYPE  SIZE TYPE MOUNTPOINT
sda            100G disk 
└─sda1 ext4    100G part

To map the /dev/sd* naming scheme used in the lsblk output to the /dev/disk/by-id naming scheme used to reliably identify DigitalOcean volumes and partitions, use the file command:

file /dev/disk/by-id/*
/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01:       symbolic link to ../../sda
/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part1: symbolic link to ../../sda1

In this example, the disk consists of a single 100G partition that spans the entire volume. It is formatted with the Ext4 filesystem. Looking at the mapping, we can see that the stable identifier for this Ext4 partition is /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part1. With this information, we can mount the filesystem reliably.

Create a mount point to attach the volume:

sudo mkdir /mnt/volume-nyc1-01-part1

Add the entry to the /etc/fstab file with the appropriate options for the filesystem type you discovered:

sudo nano /etc/fstab

In this example, the partition we found is using the Ext4 filesystem:

. . .
/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part1 /mnt/volume-nyc1-01-part1 ext4 defaults 0 2

Save and close the file when you are finished.

You can mount the volume by typing:

sudo mount -a

Partitioning the Volume

Partitioning lets you divide a single storage device into smaller units that can be managed independently. Multiple filesystems can be written to a single device and space can be segmented along functional lines.

If you have already partitioned and formatted your volume, performing these actions again will likely result in data loss. Do not perform the partitioning and formatting steps for existing volumes unless you want to erase the contents of the volume. Skip to the section on mounting filesystems if the volume has already been formatted.

Some tools expect disk space to be organized within a partition table, regardless of whether you need the segmentation capabilities. It is generally a good idea to write a partition table to most volumes, especially when you might need to move them between Droplets in the future. An existing partition table makes it more apparent at a glance that there is likely data written to the volume. If you do not need storage segmentation, it is easy to write a single partition that spans the entire volume.

The two most common partitioning systems are the traditional Master Boot Record, or MBR, format, and the more modern GUID Partition Table, or GPT. The MBR format has some inherent limitations, especially regarding the number and sizes of partitions that can be created. If you have no specialized needs, GPT is the suggested modern partitioning format.

There are quite a few partitioning tools available for Linux. Some are specific to MBR or GPT, while others handle both formats. The versions of these tools can also impact the available features significantly. For GPT partitions, parted and gdisk will be likely be installed by default. Generally, parted will be a better option for non-interactive partitioning, while gdisk has a more robust menu-driven interface for interactive sessions.

Partitioning with parted

If you want to partition non-interactively or from a script, parted is usually your best bet. Pass in the -s option if operating from a script to turn off confirmation prompts.

You can write a bare GPT partition table to your volume by passing the device to the parted command with the mklabel gpt subcommand:

sudo parted /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01 mklabel gpt

Next, you can begin writing your partitions.

Although parted is very capable of working with the GPT partitioning format, the argument structure still reflects its MBR origins. As such, you still need to pass in that you are writing “primary” partitions, even though GPT doesn’t use these designations.

The other arguments you need to pass in are the starting and ending position of the partition. These can take a variety of units, like GB, sectors, or percentages. Using percentages and passing in the -a opt option will allow parted to automatically align the partitions to the underlying sectors, which is important for proper performance.

To create a single partition that spans the entire volume, use the following command, substituting in the name of your volume.

sudo parted -a opt /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01 mkpart primary 0% 100%

To instead create two equally sized partitions, you could use these commands:

sudo parted -a opt /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01 mkpart primary 0% 50%
sudo parted -a opt /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01 mkpart primary 50% 100%

Unfortunately, parted can only handle absolute positioning by providing a definite start and end point. For example, you cannot give a starting position and the total size of the partition to automatically calculate the end point. You instead have to manually calculate and provide the ending position by combining the starting position offset with the total size you want.

Partitioning with gdisk

The gdisk utility is probably the best option for menu-driven GPT partitioning.

To start, use the volume identifier as an argument to gdisk:

sudo gdisk /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01

The utility will scan the device to try to locate existing structures. It will then drop you into an interactive prompt:

GPT fdisk (gdisk) version 1.0.1

Partition table scan:
  MBR: not present
  BSD: not present
  APM: not present
  GPT: not present

Creating new GPT entries.

Command (? for help):

To write a new partition table to the disk, use the o option:

o

You will be prompted to confirm the operation. Take this opportunity to double check that you are working with the correct volume:

This option deletes all partitions and creates a new protective MBR.
Proceed? (Y/N):

Next, create partitions by using the n option:

n

You will be taken through a series of prompts asking you to select the partition number, the first sector, the last sector or size, and the GUID for the partition type:

Partition number (1-128, default 1): 
First sector (34-209715166, default = 2048) or {+-}size{KMGTP}: 
Last sector (2048-209715166, default = 209715166) or {+-}size{KMGTP}: +10G
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): 
Changed type of partition to 'Linux filesystem'

You can just hit ENTER to accept the suggested default values. This usually makes sense for the prompts about the partition number, the first sector, and the current GUID type. For the last sector or size prompt, note that you can use the “+” sign to indicate relative sizing. This means that you can pass the partition size directly instead of having to calculate the end position as was necessary with parted.

You can display the partitions by using the p option:

p
[custom_prefix Output]
Disk /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01: 209715200 sectors, 100.0 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 19252774-25E2-4899-96CD-DCFE3B846DCC
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 209715166
Partitions will be aligned on 2048-sector boundaries
Total free space is 188743613 sectors (90.0 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048        20973567   10.0 GiB    8300  Linux filesystem

As you can see, passing in the total partition size worked exactly as it should.

If you go back to create another partition, the default values will change to reflect the current best recommended options:

n
Partition number (2-128, default 2): 
First sector (34-209715166, default = 20973568) or {+-}size{KMGTP}: 
Last sector (20973568-209715166, default = 209715166) or {+-}size{KMGTP}: 
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): 
Changed type of partition to 'Linux filesystem'

To write the table to the volume and exit the utility, use the w option:

w

Once again, you are prompted to review the changes:

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): 

When you are certain of your changes, type Y to write the partitions to the volume.

Formatting the Partitions

If you have already partitioned and formatted your volume, performing these actions again will likely result in data loss. Do not perform the partitioning and formatting steps for existing volumes unless you want to erase the contents of the volume. Skip to the section on mounting filesystems if the volume has already been formatted.

After partitioning, you must format the partitions by writing a filesystem to each storage segment. The filesystem is responsible for managing file-level interactions and providing reliable methods of writing and retrieving information on the disk. There are many different types of filesystems that can be used with Linux systems, each with their own trade-offs.

Some of the more popular filesystems for Linux are:

  • Ext4: The most popular default filesystem is Ext4, or the fourth version of the extended filesystem. The Ext4 filesystem is journaled, backwards compatible with legacy systems, incredibly stable, and has mature support and tooling. It is a good choice if you have no specialized needs.
  • XFS: XFS specializes in performance and large data files. It formats quickly and has good throughput characteristics when handling large files and when working with large disks. It also has live snapshotting features. XFS uses metadata journaling as opposed to journaling both the metadata and data. This provides fast performance, but can potentially lead to data corruption in the event of an abrupt power loss.
  • Btrfs: Btrfs is modern, feature-rich copy-on-write filesystem. This architecture allows for some volume management functionality to be integrated within the filesystem layer, including snapshots, cloning, volumes, etc. Btrfs still runs into some problems when dealing with full disks. There is some debate over its readiness for production workloads and many system administrators are waiting for the filesystem to reach greater maturity.
  • ZFS: ZFS is a copy-on-write filesystem and volume manager with a robust and mature feature set. It has great data integrity features, can handle large filesystem sizes, has typical volume features like snapshotting and cloning, and can organize volumes into RAID and RAID-like arrays for redundancy and performance purposes. In terms of use on Linux, ZFS has a controversial history due to licensing concerns. Ubuntu is now shipping a binary kernel module for it however, and Debian includes the source code in its repositories. Support across other distributions is yet to be determined.

Distributions usually promote Ext4 or XFS as the default choice for general purpose computing. The tooling for these two filesystems is usually installed by default, which means that formatting and managing these filesystems is straightforward.

Most Linux filesystems are formatted using tools that begin with mkfs. followed by the name of the filesystem in lowercase. For instance, Ext4 filesystems can be created with mkfs.ext4 and XFS filesystems can be made with mkfs.xfs.

While many options are available for these formatting tools, the default settings usually are all you need. By default, it will create a filesystem that fills the entire block device passed to it. One option you might want to add is the -L option, that allows you to label the partition with a persistent name. This can be used as a persistent alternative to the /dev/disk/by-id names and can be easier to remember.

Be sure that you pass in the partitions of your volume and not the whole volume itself. Formatting the whole volume will overwrite the partitioning scheme you created in the last section. Remember, when using the /dev/disk/by-id identifiers, partitions end in -part#.

To format your first partition with a Ext4 filesystem, type:

sudo mkfs.ext4 /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part1
mke2fs 1.42.13 (17-May-2015)
Discarding device blocks: done
Creating filesystem with 2621440 4k blocks and 655360 inodes
Filesystem UUID: 37858ba9-c2f3-4afe-9013-83111111e862
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

To format your second partition as XFS, you could use a command like this:

sudo mkfs.xfs /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part2
meta-data=/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part2 isize=512    agcount=4, agsize=5898175 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=0
data     =                       bsize=4096   blocks=23592699, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=11519, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

Mounting the Filesystems

Once a partition is formatted, it can be mounted. Mounting is the process of hooking a filesystem into the existing file hierarchy so that it can be used by users and system processes.

You can check the filesystems written to your partitions by using the lsblk utility. Most modern variants have a --fs option which displays filesystem-related information:

sudo lsblk --fs
NAME   FSTYPE LABEL  UUID                                 MOUNTPOINT
sda                                                       
├sda1 ext4          37858ba9-c2f3-4afe-9013-83111111e862 
└sda2 xfs           5d141215-7473-4100-a0e2-e69d23d89ec5 
vda                                                       
└vda1 ext4   DOROOT bc8ce2bd-8c5f-430d-9ae3-0b874736e7e4 /

If you are using an older version, you can manually replicate the output by using -o NAME,FSTYPE,LABEL,UUID,MOUNTPOINT instead:

sudo lsblk -o NAME,FSTYPE,LABEL,UUID,MOUNTPOINT
NAME   FSTYPE LABEL  UUID                                 MOUNTPOINT
sda                                                       
├sda1 ext4          37858ba9-c2f3-4afe-9013-83111111e862 
└sda2 xfs           5d141215-7473-4100-a0e2-e69d23d89ec5 
vda                                                       
└vda1 ext4   DOROOT bc8ce2bd-8c5f-430d-9ae3-0b874736e7e4 /

The bare minimum needed to mount a filesystem is:

  • the filesystem device to be mounted
  • a mount point

A mount point is the directory where the partition’s filesystem is attached so that its data can be accessed. This should almost always be an empty directory, but otherwise is a fairly arbitrary choice.

The Filesystem Hierarchy Standard recommends using /mnt or a subdirectory under it for temporarily mounted filesystems. If this matches your use case, this is probably the best place to mount it. It makes no recommendations on where to mount more permanent storage, so you can choose whichever scheme you’d like. In many cases, /mnt or /mnt subdirectories are used for more permanent storage as well.

For ease of identification, it’s often best to create a subdirectory under /mnt that can be easily tied to the filesystem it is meant to house. For instance, if you have two partitions on a volume named volume-nyc1-01, you might want to create these two mount points:

sudo mkdir /mnt/volume-nyc1-01-part1
sudo mkdir /mnt/volume-nyc1-01-part2

There are many options that can be enabled when mounting. General options can be found in the FILESYSTEM-INDEPENDENT MOUNT OPTIONS section of the man mount manual. Options targeting a specific filesystem type can be found organized by filesystem type in the FILESYSTEM-SPECIFIC MOUNT OPTIONS section.

The aptly named defaults option is a good starting point. It translates to:

  • rw: Mount the volume with read and write access enabled
  • suid: Allows the use of the set user identifier and set group identifier bits, which can be used to allow a program to run as the application owner instead of the calling user or group.
  • dev: Allows special device files to be interpreted on the filesystem.
  • exec: Allows programs to be executed from the filesystem.
  • auto: The filesystem will be mounted automatically when the mount -a command is given or when the system boots.
  • nouser: Only allow superusers to mount the filesystem.
  • async: Disk I/O will be completed asynchronously.

DigitalOcean volumes are backed by SSDs. This makes the discard option, which enables continuous TRIM operations on the device, a possible good addition to this list. There is some debate over the performance and integrity implications of recommending continuous TRIM rather than setting up an automated task to periodically perform TRIM. Distributions like Ubuntu have a cron job enabled by default that provides periodic TRIM operations, which makes the discard option slightly redundant.

When you’ve decided on appropriate mount options, you can mount the filesystems using this syntax. Make sure to adjust the options to match your preferences:

sudo mount -o defaults,discard /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part1 /mnt/volume-nyc1-01-part1
sudo mount -o defaults,discard /dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part2 /mnt/volume-nyc1-01-part2

Check that the server sees the additional space with the df command:

df -h -x tmpfs -x devtmpfs
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        20G  1.2G   18G   6% /
/dev/sda1       9.8G   23M  9.2G   1% /mnt/volume-nyc1-01-part1
/dev/sda2        90G   33M   90G   1% /mnt/volume-nyc1-01-part2

You can also check write and read capabilities with simple tests:

echo "success" | sudo tee /mnt/volume-nyc1-01-part1/test_file
cat /mnt/volume-nyc1-01-part1/test_file
success

If files can be written to and read from the filesystem, you can be fairly confident that it is working as expected. Feel free to remove the test file to clean up:

sudo rm /mnt/volume-nyc1-01-part1/test_file

Setting Up Persistent Mounting

The operating system uses entries in the /etc/fstab file to decide which filesystems to mount automatically at boot.

Adjust the /etc/fstab file to configure this functionality:

sudo nano /etc/fstab

The entries in this file are composed of space-separated fields representing:

  • filesystem to mount: This is the partition that should be mounted.
  • mount point: This is the location in the file hierarchy where the filesystem should be attached.
  • filesystem type: This defines the type of filesystem, in lower case. For example, ext4, or xfs.
  • mount options: This defines the mount options that should be used. As mentioned in the previous section of this guide, check the general and filesystem-specific mount options sections in man mount for available choices.
  • filesystem dump: Used by the dump utility to know whether to operate on the filesystem. Usually, this should be set to “0” to disable this functionality.
  • fsck ordering: Tells the fsck utility which order the filesystems should be checked. The root filesystem should be set to “1”. Other filesystems can be set to “2” to enable checking or “0” to disable checking.

One particularly useful option to add to the /etc/fstab file when working with volumes is nofail. This will allow the operating system to continue the boot sequence if the filesystem cannot be found, which is important in case the volume is detached in the future.

Assuming that our first partition is formatted as Ext4 and our second is XFS, the entries might look something like this:

. . .

/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part1 /mnt/volume-nyc1-01-part1 ext4 defaults,nofail,discard 0 2
/dev/disk/by-id/scsi-0DO_Volume_volume-nyc1-01-part2 /mnt/volume-nyc1-01-part2 xfs defaults,nofail,discard 0 2

Save and close the file when you are finished.

Now, we can test that the entries can be interpreted correctly. If your filesystems are already mounted, you can unmount them with the umount command:

cd ~
sudo umount /mnt/volume-nyc1-01-part1
sudo umount /mnt/volume-nyc1-01-part2

Check that the volume filesystems are no longer usable:

df -h -x tmpfs -x devtmpfs
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        20G  1.2G   18G   6% /

Now, check that automatic mounting works correctly by typing:

sudo mount -a

Check that the filesystems were mounted again:

df -h -x tmpfs -x devtmpfs
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        20G  1.2G   18G   6% /
/dev/sda1       9.8G   23M  9.2G   1% /mnt/volume-nyc1-01-part1
/dev/sda2        90G   33M   90G   1% /mnt/volume-nyc1-01-part2

After you attach a volume to your Droplet, you must prepare the volume for use as you would with any new disk in Linux. The steps needed will depend on whether you are working with a newly created volume or setting up an existing volume on a new server.

Testing the Mounted Filesystem

Check that the mount was successful by passing the mount point to findmnt:

findmnt /mnt/volume-nyc1-01-part1

You should see output indicating that it is currently mounted:

TARGET                    SOURCE    FSTYPE OPTIONS
/mnt/volume-nyc1-01-part1 /dev/sda1 ext4   rw,relatime,data=ordered

If you list the contents of the directory, you should see the lost+found directory if you are using an Ext4 filesystem:

ls /mnt/volume-nyc1-01-part1
lost+found

Double check that the new storage space is writable:

echo 'success' | sudo tee /mnt/volume-nyc1-01-part1/test_file

Now read the file back:

cat /mnt/volume-nyc1-01-part1/test_file
success

If the mount is working as expected, delete the test file:

sudo rm /mnt/volume-nyc1-01-part1/test_file