How To Use SSHFS to Mount Remote File Systems Over SSH

• SSH Access: Two Linux servers (or one local machine and one remote server) configured to allow SSH access between them. You can accomplish this by following our Initial Server Setup Guide.

• User Permissions: Appropriate permissions to install software and mount file systems on the local machine.

• Network Connectivity: Stable network connection between local and remote systems. For AI/ML workflows, consider bandwidth requirements for large dataset access.

• SSH Key Authentication: For production use and automation, set up SSH key-based authentication to avoid password prompts. Learn more about SSH essentials and working with SSH servers.

• FUSE Support: Ensure FUSE (Filesystem in Userspace) is available on your system. Most modern Linux distributions include FUSE by default.

SSHFS is available for most Linux distributions and can be installed using standard package managers. The installation process varies slightly between operating systems, but the core functionality remains consistent.

Linux Installation

Ubuntu/Debian Systems

First, update your package sources:

sudo apt update

Install SSHFS and FUSE:

sudo apt install sshfs fuse3

For older systems, you may need fuse instead of fuse3:

sudo apt install sshfs fuse

RHEL/CentOS/Fedora Systems

For RHEL-based systems:

sudo dnf install sshfs fuse-sshfs

Or for older systems:

sudo yum install sshfs fuse-sshfs

Arch Linux

sudo pacman -S sshfs

macOS Installation

On macOS, SSHFS requires FUSE support. Install using Homebrew:

brew install --cask macfuse
brew install gromgull/fuse/sshfs-mac

Alternatively, you can use the macFUSE Project directly.

Windows Installation

Windows users can install SSHFS through third-party implementations:

1. Install WinFsp: Download and install WinFsp from the official repository.

2. Install SSHFS-Win: Download and install SSHFS-Win from the project’s GitHub repository.

Verifying Installation

After installation, verify that SSHFS is working correctly:

sshfs --version

You should see output similar to:

SSHFS version 3.7.3
FUSE library version: 3.10.5
fusermount3 version: 3.10.5

Mounting a remote filesystem with SSHFS requires creating a local mount point and using the sshfs command with appropriate options. This section covers both basic and advanced mounting techniques optimized for different use cases.

Basic Mounting

Creating a Mount Point

First, create a directory to serve as the mount point. For AI/ML workflows, consider using descriptive names that indicate the purpose:

# For general use
sudo mkdir /mnt/remote_data

# For AI/ML datasets
sudo mkdir /mnt/ml_datasets

# For collaborative development
sudo mkdir /mnt/shared_code

Basic Mount Command

Mount a remote directory using the basic sshfs command:

sudo sshfs -o allow_other,default_permissions sammy@your_other_server:~/ /mnt/remote_data

Command Breakdown:

• -o allow_other,default_permissions: Allows other users to access the mount and uses standard filesystem permissions
• sammy@your_other_server:~/: Remote user, server, and directory path (using SSH syntax)
• /mnt/remote_data: Local mount point

Advanced Mounting Options

Performance-Optimized Mounting for AI/ML Workflows

For data-intensive applications like machine learning, use these optimized options:

sudo sshfs -o allow_other,default_permissions,compression=yes,cache=yes,auto_cache,reconnect,ServerAliveInterval=15,ServerAliveCountMax=3 sammy@your_other_server:/datasets /mnt/ml_datasets

Advanced Options Explained:

• compression=yes: Enables SSH compression to reduce bandwidth usage
• cache=yes: Enables local caching for better performance
• auto_cache: Automatically manages cache invalidation
• reconnect: Automatically reconnects on connection drops
• ServerAliveInterval=15: Sends keep-alive packets every 15 seconds
• ServerAliveCountMax=3: Maximum failed keep-alive attempts before disconnecting

Security-Enhanced Mounting

For sensitive data or production environments:

sudo sshfs -o allow_other,default_permissions,idmap=user,uid=1000,gid=1000,umask=0022,IdentityFile=/home/sammy/.ssh/id_rsa sammy@your_other_server:/secure_data /mnt/secure_data

Security Options:

• idmap=user: Maps remote user IDs to local user IDs
• uid=1000,gid=1000: Sets specific user and group IDs
• umask=0022: Sets file permissions mask
• IdentityFile: Specifies SSH private key for authentication

AI/ML-Specific Mounting Strategies

Mounting Large Datasets

For machine learning datasets that are too large for local storage:

# Mount with read-only access for large datasets
sudo sshfs -o ro,allow_other,default_permissions,compression=yes,cache=yes sammy@gpu_server:/datasets/imagenet /mnt/imagenet

# Mount with write access for model checkpoints
sudo sshfs -o allow_other,default_permissions,compression=yes sammy@gpu_server:/models /mnt/model_checkpoints

Multi-Server Mounting for Distributed Training

Mount multiple remote directories for distributed machine learning:

# Mount training data from primary server
sudo sshfs -o allow_other,default_permissions,compression=yes sammy@data_server:/training_data /mnt/training_data

# Mount validation data from secondary server
sudo sshfs -o allow_other,default_permissions,compression=yes sammy@backup_server:/validation_data /mnt/validation_data

# Mount shared model repository
sudo sshfs -o allow_other,default_permissions,compression=yes sammy@model_server:/models /mnt/shared_models

Troubleshooting Common Issues

Connection Reset by Peer

If you encounter a “Connection reset by peer” error:

1. Verify SSH Key Authentication:

   ssh sammy@your_other_server
   
   

   If you need to set up SSH keys, follow our guide on how to set up SSH keys on Ubuntu 22.04.

2. Check SSH Configuration:

   ssh -v sammy@your_other_server
   
   

   For advanced SSH configuration, see our SSH essentials guide.

3. Test with Verbose SSHFS:

   sudo sshfs -o debug,allow_other,default_permissions sammy@your_other_server:~/ /mnt/remote_data
   
   

Permission Issues

For non-root mounting, add your user to the fuse group:

sudo groupadd fuse
sudo usermod -a -G fuse sammy

Then log out and back in, or use:

newgrp fuse

Verifying the Mount

Check that the remote filesystem is mounted correctly:

# List mounted filesystems
mount | grep sshfs

# Check mount point contents
ls -la /mnt/remote_data

# Test file operations
touch /mnt/remote_data/test_file
ls -la /mnt/remote_data/test_file

Unmounting

To unmount the remote filesystem:

# Standard unmount
sudo umount /mnt/remote_data

# Force unmount if needed
sudo fusermount -u /mnt/remote_data

# Check if unmounted
mount | grep sshfs

For production environments and AI/ML workflows that require persistent access to remote data, configuring permanent SSHFS mounts is essential. This section covers both traditional /etc/fstab configuration and modern systemd-based approaches.

Traditional fstab Configuration

Basic fstab Entry

Open /etc/fstab with your preferred editor:

sudo nano /etc/fstab

Add a basic SSHFS entry at the end of the file:

# SSHFS mount for remote data
sammy@your_other_server:~/ /mnt/remote_data fuse.sshfs noauto,x-systemd.automount,_netdev,reconnect,identityfile=/home/sammy/.ssh/id_rsa,allow_other,default_permissions 0 0

Advanced fstab Configuration for AI/ML Workflows

For data-intensive applications, use this optimized configuration:

# AI/ML Dataset Mount - Optimized for Performance
sammy@gpu_server:/datasets /mnt/ml_datasets fuse.sshfs noauto,x-systemd.automount,_netdev,reconnect,identityfile=/home/sammy/.ssh/id_rsa,allow_other,default_permissions,compression=yes,cache=yes,auto_cache,ServerAliveInterval=15,ServerAliveCountMax=3 0 0

# Model Checkpoints Mount - Read/Write Access
sammy@model_server:/models /mnt/model_checkpoints fuse.sshfs noauto,x-systemd.automount,_netdev,reconnect,identityfile=/home/sammy/.ssh/id_rsa,allow_other,default_permissions,compression=yes 0 0

# Shared Code Repository Mount
sammy@git_server:/repos /mnt/shared_code fuse.sshfs noauto,x-systemd.automount,_netdev,reconnect,identityfile=/home/sammy/.ssh/id_rsa,allow_other,default_permissions 0 0

Configuration Options Explained:

• noauto: Prevents automatic mounting at boot
• x-systemd.automount: Enables systemd automounting (mounts on first access)
• _netdev: Indicates network dependency
• reconnect: Automatically reconnects on connection drops
• identityfile: Path to SSH private key for authentication
• compression=yes: Enables SSH compression
• cache=yes,auto_cache: Enables local caching
• ServerAliveInterval=15: Keep-alive interval
• ServerAliveCountMax=3: Maximum failed keep-alive attempts

Modern systemd-based Configuration

Creating a systemd Mount Unit

Create a systemd mount unit for better control:

sudo nano /etc/systemd/system/mnt-remote_data.mount

Add the following content:

[Unit]
Description=SSHFS mount for remote data
After=network-online.target
Wants=network-online.target
Before=remote-fs.target

[Mount]
What=sammy@your_other_server:~
Where=/mnt/remote_data
Type=fuse.sshfs
Options=allow_other,default_permissions,compression=yes,cache=yes,auto_cache,reconnect,IdentityFile=/home/sammy/.ssh/id_rsa

[Install]
WantedBy=multi-user.target

Creating a systemd Automount Unit

For on-demand mounting, create an automount unit:

sudo nano /etc/systemd/system/mnt-remote_data.automount

[Unit]
Description=SSHFS automount for remote data
After=network-online.target
Wants=network-online.target

[Automount]
Where=/mnt/remote_data
TimeoutIdleSec=300

[Install]
WantedBy=multi-user.target

Enabling and Managing systemd Mounts

# Enable and start the automount
sudo systemctl enable mnt-remote_data.automount
sudo systemctl start mnt-remote_data.automount

# Check mount status
sudo systemctl status mnt-remote_data.automount

# Manually mount/unmount
sudo systemctl start mnt-remote_data.mount
sudo systemctl stop mnt-remote_data.mount

Testing Permanent Mounts

Test fstab Configuration

# Test fstab entries without rebooting
sudo mount -a

# Check if mounts are active
mount | grep sshfs

# Test automount functionality
ls /mnt/remote_data

Verify systemd Mounts

# Check systemd mount status
sudo systemctl status mnt-remote_data.mount

# View mount logs
sudo journalctl -u mnt-remote_data.mount

# Test automount
sudo systemctl status mnt-remote_data.automount

Security Considerations for Permanent Mounts

SSH Key Management

Ensure SSH keys are properly secured:

# Set correct permissions on SSH keys
chmod 600 /home/sammy/.ssh/id_rsa
chmod 644 /home/sammy/.ssh/id_rsa.pub

# Use SSH agent for key management
ssh-add /home/sammy/.ssh/id_rsa

Network Security

Configure SSH for optimal security:

# Edit SSH client config
nano ~/.ssh/config

Add the following configuration:

Host your_other_server
    HostName your_other_server
    User sammy
    Port 22
    IdentityFile /home/sammy/.ssh/id_rsa
    ServerAliveInterval 15
    ServerAliveCountMax 3
    Compression yes
    ForwardAgent no
    ForwardX11 no

Troubleshooting Permanent Mounts

Common Issues and Solutions

When setting up permanent SSHFS mounts, you might encounter several issues. Here’s a breakdown of common problems and how to troubleshoot them:

1. Mount fails at boot: This often occurs if the network is not fully initialized when systemd attempts to mount the filesystem, if there are errors in the /etc/fstab entry, or if the systemd automount unit is misconfigured.

   # Check systemd logs for the mount unit
   sudo journalctl -u mnt-remote_data.mount
   
   # Test manual mount to isolate fstab/systemd issues from SSHFS command issues
   sudo mount /mnt/remote_data
   
   

2. Network connectivity issues: Problems connecting to the remote server can stem from incorrect server addresses, firewall restrictions (on either local or remote machine), or general network instability.

   # Test the underlying SSH connection independently
   ssh sammy@your_other_server
   
   # Check the status of your local network manager
   systemctl status NetworkManager
   
   

3. Permission problems: These usually arise when the local user doesn’t have the necessary permissions to access the mounted directory, if allow_other is missing, or if uid/gid mapping is incorrect, or if the IdentityFile has incorrect permissions.

   # Check the permissions of the local mount point
   ls -la /mnt/remote_data
   
   # Verify the user and group IDs of the local user
   id sammy
   
   

SSHFS Performance Optimization

For AI/ML workflows and high-performance applications, consider these optimization strategies.

Network latency and available bandwidth are often the biggest bottlenecks for SSHFS performance, especially in AI/ML workflows involving large datasets. Optimizing the SSH connection itself can significantly reduce transfer times and improve responsiveness. This involves enabling compression and configuring connection keep-alives to prevent disconnections.

1. Network Optimization

# Optimize SSH connection for SSHFS
sshfs -o compression=yes,compression_level=6,cache=yes,auto_cache,reconnect,ServerAliveInterval=15,ServerAliveCountMax=3,sshfs_debug sammy@your_other_server:/data /mnt/optimized_data

This command optimizes the SSH connection specifically for SSHFS to maximize throughput and reliability for AI/ML workflows.

Key Options Explained:

• compression=yes: Enables SSH compression to reduce bandwidth usage, crucial for large dataset transfers
• compression_level=6: Sets compression level (1-9). Level 6 provides optimal balance between compression ratio and CPU usage
• cache=yes: Enables local caching of file metadata and directory listings, reducing network round-trips
• auto_cache: Automatically manages cache invalidation, ensuring data consistency while maintaining performance
• reconnect: Automatically reconnects if SSH connection drops, essential for long-running AI training jobs
• ServerAliveInterval=15: Sends keep-alive packets every 15 seconds to detect connection issues quickly
• ServerAliveCountMax=3: Allows up to 3 failed keep-alive attempts before considering connection dead (45 seconds total)
• sshfs_debug: Enables debug logging to help troubleshoot performance issues

Best For: High-bandwidth applications, AI/ML data processing, and environments with stable network connections.

2. Caching Strategies

# Enable aggressive caching for read-heavy workloads
sshfs -o cache=yes,auto_cache,entry_timeout=7200,attr_timeout=7200,ac_attr_timeout=7200 sammy@your_other_server:/datasets /mnt/cached_datasets

This command implements aggressive caching for read-heavy workloads, particularly useful for large datasets accessed repeatedly in AI/ML training.

Key Options Explained:

• cache=yes: Enables local caching of file data and metadata
• auto_cache: Automatically manages cache invalidation based on file modification times
• entry_timeout=7200: Caches directory entries for 2 hours (7200 seconds), dramatically reducing time needed to list large directories
• attr_timeout=7200: Caches file attributes (permissions, size, timestamps) for 2 hours, reducing metadata lookups
• ac_attr_timeout=7200: Caches access control attributes for 2 hours, reducing permission checks

Performance Impact:

• First access: Normal speed (data fetched from remote)
• Subsequent access: Near-local speed (data served from cache)
• Memory usage: Higher (cached data stored in RAM)
• Network usage: Significantly reduced for repeated access

Best For: Machine learning datasets read multiple times, large directory listings, development environments with frequent file access, and read-only or mostly-read workloads.

3. Bandwidth Management

# Limit bandwidth usage for shared connections
sshfs -o compression=yes,compression_level=9,sshfs_debug,debug sammy@your_other_server:/data /mnt/bandwidth_limited

This command maximizes compression and provides detailed debugging for bandwidth-constrained environments, ideal for expensive or slow network connections.

Key Options Explained:

• compression=yes: Enables SSH compression
• compression_level=9: Maximum compression level. Uses more CPU but achieves best compression ratio, ideal for slow or expensive network connections
• sshfs_debug: Enables SSHFS-specific debug logging
• debug: Enables general FUSE debug logging for detailed troubleshooting

Performance Trade-offs:

• CPU Usage: High (due to maximum compression)
• Bandwidth Usage: Minimal (maximum compression)
• Latency: Slightly higher (compression overhead)
• Debugging: Excellent (comprehensive logging)

Best For: Slow network connections (mobile, satellite), expensive bandwidth (cloud data transfer costs), debugging performance issues, and environments where bandwidth is more expensive than CPU.

Performance Comparison Table

AI/ML Workflow Integration

TensorFlow/PyTorch Integration

Modern AI/ML workflows frequently involve working with massive datasets that often reside on remote storage systems, specialized data servers, or GPU-accelerated machines. Copying these large datasets locally for every experiment or training run is inefficient and time-consuming. SSHFS provides an elegant solution by allowing you to mount these remote datasets directly onto your local development or training environment.

This enables AI/ML frameworks like TensorFlow and PyTorch to access the data seamlessly, as if it were stored on a local disk, without the overhead of manual transfers. This integration streamlines data access, accelerates development cycles, and facilitates distributed training setups.

# Example: Mounting remote datasets for TensorFlow
import os
import tensorflow as tf

# Mount remote dataset
os.system("sshfs -o compression=yes,cache=yes sammy@gpu_server:/datasets/imagenet /mnt/imagenet")

# Load dataset from mounted location
dataset = tf.keras.preprocessing.image_dataset_from_directory(
    '/mnt/imagenet/train',
    image_size=(224, 224),
    batch_size=32
)

This example demonstrates how to mount a remote dataset using SSHFS and then integrate it directly into a TensorFlow data loading pipeline. The key benefit is that your AI/ML code can treat the remote data as if it were local, simplifying data management and access.

Distributed Training Setup

#!/bin/bash
# Mount multiple remote directories for distributed training

# Mount training data
sshfs -o compression=yes,cache=yes sammy@data_server:/training_data /mnt/training_data

# Mount validation data
sshfs -o compression=yes,cache=yes sammy@validation_server:/validation_data /mnt/validation_data

# Mount model checkpoints
sshfs -o compression=yes sammy@model_server:/checkpoints /mnt/checkpoints

# Start distributed training
python train_distributed.py \
    --train_data /mnt/training_data \
    --val_data /mnt/validation_data \
    --checkpoint_dir /mnt/checkpoints

1. What is SSHFS and why should I use it instead of other file transfer methods?

SSHFS (SSH Filesystem) is a FUSE-based filesystem that allows you to mount remote directories over SSH connections. Unlike traditional file transfer methods like SCP or SFTP, SSHFS provides seamless, real-time access to remote files as if they were local. This makes it ideal for:

• AI/ML Workflows: Access large datasets without local storage constraints
• Development: Edit remote files directly with local tools
• Collaboration: Share code and data across different environments
• Security: Leverages SSH’s robust encryption without additional configuration

The key advantage is that you can use any local application (editors, IDEs, data analysis tools) to work with remote files transparently, without the overhead of constantly uploading/downloading files.

2. How do I mount a remote file system over SSH in Linux?

The basic process involves three steps:

1. Install SSHFS: sudo apt install sshfs fuse3 (Ubuntu/Debian)
2. Create mount point: sudo mkdir /mnt/remote_data
3. Mount remote directory: sudo sshfs sammy@your_server:/remote/path /mnt/remote_data

For AI/ML workflows, use optimized options:

sudo sshfs -o compression=yes,cache=yes,reconnect sammy@gpu_server:/datasets /mnt/ml_datasets

Always ensure SSH key authentication is set up to avoid password prompts, especially for automated workflows.

3. Can I use SSHFS on macOS or Windows?

Yes, SSHFS is available on all major platforms:

macOS:

brew install --cask macfuse
brew install gromgull/fuse/sshfs-mac

Windows:

1. Install WinFsp
2. Install SSHFS-Win

Cross-Platform Considerations:

• Linux typically offers the best performance and compatibility
• macOS and Windows implementations may have different performance characteristics
• For AI/ML workflows, Linux is generally recommended for optimal performance

4. How do I unmount an SSHFS mount and what happens if I don’t?

To unmount an SSHFS mount:

# Standard unmount
sudo umount /mnt/remote_data

# Force unmount if needed
sudo fusermount -u /mnt/remote_data

Always unmount SSHFS filesystems before:

• Shutting down or rebooting your system
• Disconnecting from the network
• Switching to a different network

What are the consequences of not unmounting:

• Data corruption risk
• Pending file operations may be lost
• System may hang during shutdown
• Network connections may remain open unnecessarily

5. How do I set up SSHFS to reconnect automatically and handle network interruptions?

For automatic reconnection and robust network handling, use these options:

Basic Auto-Reconnect:

sshfs -o reconnect,ServerAliveInterval=15,ServerAliveCountMax=3 sammy@server:/data /mnt/data

Advanced Configuration for Production:

# In /etc/fstab
sammy@server:/data /mnt/data fuse.sshfs noauto,x-systemd.automount,_netdev,reconnect,identityfile=/home/sammy/.ssh/id_rsa,allow_other,default_permissions,compression=yes,cache=yes,ServerAliveInterval=15,ServerAliveCountMax=3 0 0

systemd Automount (Recommended):

# Enable automount
sudo systemctl enable mnt-data.automount
sudo systemctl start mnt-data.automount

This configuration ensures SSHFS automatically reconnects on network recovery and mounts on-demand, providing both reliability and efficiency.

SSHFS has evolved from a simple remote filesystem tool into a critical component of modern AI/ML workflows and collaborative development environments. This comprehensive guide has covered not only basic SSHFS usage but also advanced configuration techniques, performance optimization strategies, and real-world applications that demonstrate its continued relevance.

Next, you may want to learn about working with object storage which can be mounted concurrently across multiple servers. You might also find these related SSH tutorials helpful:

• SSH Essentials: Working with SSH Servers, Clients, and Keys: Learn the fundamental concepts of SSH, including how to set up and manage SSH servers and clients, and how to use SSH keys for secure authentication.
• How To Use SSH to Connect to a Remote Server: A practical guide on establishing an SSH connection from your local machine to a remote server, which is the foundational step for using SSHFS.
• Understanding the SSH Encryption and Connection Process: Dive deeper into the technical details of how SSH secures your connections, explaining the encryption protocols and the various stages of the connection handshake.