Question
SCP from Windows destroys SSH connections
- Posted on November 27, 2023
- DigitalOcean DropletsUbuntu
Asked by d1b7968b962046a0b02da4e5573c8e
Hi, I’m trying to write a publish script that copies a bunch of files over from a Windows machine to a Linux droplet (Ubuntu). I can successfully SSH into the remote Linux machine and can also run small (single file) scp commands to copy files over.
When I run a recursive copy scp.exe -r [source dir] [remote dir] of a few megabytes, all SSH connections into my droplet stall. Eventually the original scp command times out with the error message:
ssh: connect to host IPADDRESS port 22: Connection timed out
I’m unable to reconnect to the droplet over ssh for about 5 minutes.
Some things I’ve noticed and tried:
- Websites running on the droplet continue to be accessible during the freeze, so the droplet is still alive,.
- I’ve tried changing the MTU on the remote droplet and it didn’t make a discernible difference.
- I tried to look at the ssh logs on the remote server and noticed that the scp commands create connections to the server, but never close them. When I issue an ssh command, these connections are closed after use.
- I have adjusted the MaxSessions and MaxStartups number in the
/etc/ssh../sshd_configfile (and restarted usingsudo systemctl restart ssh.service)
Anyone know anything else i can try?
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!
These answers are provided by our Community. If you find them useful, show some love by clicking the heart. If you run into issues leave a comment, or add your own answer to help others.
Get our biweekly newsletter
Sign up for Infrastructure as a Newsletter.
Hollie's Hub for Good
Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.
Become a contributor
Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.
Hi there,
There are a few things that I could suggest here to try out:
Keep an Eye on Server Resources: Before initiating the transfer, use
htopto monitor your Droplet’s performance in real-time. If you see CPU or memory usage spike during the transfer, consider upgrading your Droplet to have some extra buffer.This little tool gives you a dynamic overview of your Droplet’s health. If you don’t have it installed, you can get it with:
Verbose SCP for Transparency: Use the
-vflag with your SCP command to get a verbose output. This will give you a play-by-play of what SCP is doing behind the scenes.The verbose output can be quite extensive, but look for lines that begin with “debug1” or “debug2” — they often contain the clues you need.
Optimize SCP with Rsync: Rsync is a more robust tool for file transfers, especially for larger datasets. It only transfers what’s changed and can resume interrupted transfers without starting over.
The
-aoption preserves permissions and times,-vgives you verbose output,-zcompresses file data during the transfer, and--progressshows you the transfer progress.Let me know how it goes!
Best,
Bobby
Heya, @d1b7968b962046a0b02da4e5573c8e
I would also recommend using
rsyncinstead ofscp. Bobby already mentioned why this would be beneficial for your setup.On top of what’s already mentioned, you can add the following lines to your SSH client configuration file (usually
~/.ssh/configon the Windows machine) to send keep-alive packets, preventing the connection from timing out.Adjust the
ServerAliveIntervalvalue as needed.Regards
Heya @d1b7968b962046a0b02da4e5573c8e,
It sounds to me like your Droplet just can’t handle the command as there might be a lot of files there.
Apart from what already is suggested, you can confirm this by being in the Droplet when you run the command and seeing if it’s responsive. I’m sure it won’t be.
To resolve the issue shortterm, you might want to add some SWAP:
https://www.digitalocean.com/community/tutorial-collections/how-to-add-swap-space
wap is a portion of hard drive storage that has been set aside for the operating system to temporarily store data that it can no longer hold in RAM. This lets you increase the amount of information that your server can keep in its working memory, with some caveats. The swap space on the hard drive will be used mainly when there is no longer sufficient space in RAM to hold in-use application data.
The information written to disk will be significantly slower than information kept in RAM, but the operating system will prefer to keep running application data in memory and use swap for the older data. Overall, having swap space as a fallback for when your system’s RAM is depleted can be a good safety net against out-of-memory exceptions on systems with non-SSD storage available.