How to Fix IOError: [Errno 32] Broken pipe during SFTP transfers in Paramiko

IOError: [Errno 32] Broken pipe in Paramiko SFTP transfers occurs when the SSH channel is closed unexpectedly—often due to idle timeouts, large file uploads, or server-side limits. This guide covers keepalive, chunked transfers, retry logic, and SSHD configuration to eliminate broken-pipe errors.

1. Enable SSH Keepalive

Send periodic keepalive packets to prevent server closing idle connections:

transport = ssh.get_transport()
transport.set_keepalive(30)  # send keepalive every 30 seconds
sftp = paramiko.SFTPClient.from_transport(transport)
    

2. Use Chunked File Transfers

Break large files into smaller chunks to avoid buffer overflow:

def sftp_put_chunked(sftp, local_path, remote_path, chunk_size=32768):
    with open(local_path, 'rb') as f:
        with sftp.file(remote_path, 'wb') as remote_file:
            while True:
                data = f.read(chunk_size)
                if not data:
                    break
                remote_file.write(data)
    remote_file.flush()
    

3. Implement Retry Logic

Wrap transfers in retries with exponential backoff:

import time
import socket

def safe_sftp_put(sftp, local, remote, retries=3):
    for i in range(retries):
        try:
            sftp_put_chunked(sftp, local, remote)
            return
        except (IOError, socket.error) as e:
            if i < retries - 1:
                time.sleep(2 ** i)
            else:
                raise
    

4. Adjust Server-Side SSHD Timeout Settings

Increase server idle timeouts in /etc/ssh/sshd_config:

ClientAliveInterval 60
ClientAliveCountMax 10
  

5. Use SFTP’s putfo for File-Like Objects

Stream via file object with default buffering:

with open('large.bin','rb') as f:
    sftp.putfo(f, 'remote_large.bin', file_size=os.path.getsize('large.bin'), callback=None)
    

6. Monitor Transfer Progress

Use callback to detect stalls:

def print_progress(transferred, total):
    print(f"Transferred {transferred}/{total} bytes")

sftp.put('large.bin', 'remote.bin', callback=print_progress)
    

7. Summary Checklist

1. Enable transport.set_keepalive() on SSH transport.
2. Perform chunked writes to remote file.
4. Wrap file transfers in retry/backoff logic.
5. Increase SSHD ClientAliveInterval and ClientAliveCountMax.
6. Use putfo for streaming file-like uploads.
7. Implement progress callbacks to detect stalls early.