IOError: [Errno 32] Broken pipe in Paramiko SFTP transfers occurs when the SSH channel is closed unexpectedly—often due to idle timeouts, large file uploads, or server-side limits. This guide covers keepalive, chunked transfers, retry logic, and SSHD configuration to eliminate broken-pipe errors.
1. Enable SSH Keepalive
Send periodic keepalive packets to prevent server closing idle connections:
transport = ssh.get_transport() transport.set_keepalive(30) # send keepalive every 30 seconds sftp = paramiko.SFTPClient.from_transport(transport)
Tip: Adjust interval below server’s ClientAliveInterval/ClientAliveCountMax.
2. Use Chunked File Transfers
Break large files into smaller chunks to avoid buffer overflow:
def sftp_put_chunked(sftp, local_path, remote_path, chunk_size=32768): with open(local_path, 'rb') as f: with sftp.file(remote_path, 'wb') as remote_file: while True: data = f.read(chunk_size) if not data: break remote_file.write(data) remote_file.flush()
3. Implement Retry Logic
Wrap transfers in retries with exponential backoff:
import time import socket def safe_sftp_put(sftp, local, remote, retries=3): for i in range(retries): try: sftp_put_chunked(sftp, local, remote) return except (IOError, socket.error) as e: if i < retries - 1: time.sleep(2 ** i) else: raise
4. Adjust Server-Side SSHD Timeout Settings
Increase server idle timeouts in /etc/ssh/sshd_config
:
ClientAliveInterval 60 ClientAliveCountMax 10
Restart SSH service:
sudo systemctl restart sshd
.5. Use SFTP’s putfo
for File-Like Objects
Stream via file object with default buffering:
with open('large.bin','rb') as f: sftp.putfo(f, 'remote_large.bin', file_size=os.path.getsize('large.bin'), callback=None)
6. Monitor Transfer Progress
Use callback to detect stalls:
def print_progress(transferred, total): print(f"Transferred {transferred}/{total} bytes") sftp.put('large.bin', 'remote.bin', callback=print_progress)
7. Summary Checklist
- Enable
transport.set_keepalive()
on SSH transport. - Perform chunked writes to remote file.
- Wrap file transfers in retry/backoff logic.
- Increase SSHD
ClientAliveInterval
andClientAliveCountMax
. - Use
putfo
for streaming file-like uploads. - Implement progress callbacks to detect stalls early.