Linux File Transfer with scp, rsync, and Command-Line Email
I moved files between hosts with scp for quick copies and rsync for resumable, verified, delta transfers, then wired CLI email for job notifications. Resumable rsync transfers survived interruptions that would have forced scp to restart from zero.
Objective & Context
Automation pipelines must move artifacts reliably and report results. This lab compares scp, sftp, and rsync transports and adds curl/mail-based notifications, the transfer layer for backups and deployments.
Environment & Prerequisites
- Two SSH-reachable Linux hosts with rsync installed.
- curl and a mail relay or API for notifications.
- A large file to demonstrate resume.
flowchart LR
A[Source] -->|rsync delta| B[Destination]
B --> V{checksum ok?}
V -->|yes| N[notify via curl/mail]
V -->|no| R[retry]
Step-by-Step Execution
1. Quick copy with scp
scp report.tar.gz taki@nas:/backups/2. Resumable, verified transfer with rsync
rsync -avP --checksum big.img taki@nas:/backups/3. Notify on completion
curl -s -X POST $WEBHOOK -d 'text=backup transfer complete'big.img 1,073,741,824 100% 118.20MB/s resumed at 62%Validation & Testing
Interrupt an rsync transfer and restart it to confirm it resumes rather than restarts, then verify the checksum matches. Pass criteria: rsync resumes from the breakpoint, checksums match, and the notification fires on success.
Advanced: Troubleshooting
- Transfer restarts fully: use
-P(partial + progress) so rsync resumes. - Permission denied: confirm SSH key auth and destination write perms.
- Silent corruption: add
--checksumfor content verification, not just size/time.
Key Results
- Resumed interrupted transfers instead of restarting from zero.
- Verified transfer integrity with content checksums.
- Automated completion notifications via webhook/mail.
- Selected the right transport (scp vs rsync) per task.