Python File Operations with pathlib, Context Managers, and JSON
I built safe file I/O in Python using pathlib, with context managers, and the json module, including an atomic write pattern. Writing to a temp file and renaming guaranteed config files were never left half-written on failure.
Objective & Context
Automation scripts read and rewrite config and state files constantly. This lab uses context managers for guaranteed close, pathlib for portable paths, and atomic replace to prevent corruption, the I/O backbone for the sysadmin and IR scripts.
Environment & Prerequisites
- Python 3.11 with pathlib and json (standard library).
- A writable working directory.
- Sample JSON config to manipulate.
flowchart LR
D[Data] --> T[Write temp file]
T --> F[flush + fsync]
F --> R[os.replace -> target]
R --> S[Consistent file]
Step-by-Step Execution
1. Read JSON with a context manager
import json, pathlib
cfg = json.loads(pathlib.Path("config.json").read_text())
2. Atomic write to avoid corruption
import os, tempfile, pathlib
def atomic_write(path, text):
d = pathlib.Path(path).parent
fd, tmp = tempfile.mkstemp(dir=d)
with os.fdopen(fd, "w") as f:
f.write(text); f.flush(); os.fsync(f.fileno())
os.replace(tmp, path)
3. Verify the round trip
python -c "import json,pathlib; print(json.loads(pathlib.Path('config.json').read_text())['name'])"tyf-ai
Validation & Testing
Interrupt a write mid-stream and confirm the original file remains intact thanks to atomic replace. Pass criteria: JSON round-trips losslessly and an aborted write never corrupts the target file.
Advanced: Troubleshooting
- Partial files: never write in place; write-temp-then-replace.
- Encoding issues: specify
encoding="utf-8"explicitly. - Leaked handles: always use
withto guarantee close.
Key Results
- Guaranteed zero file corruption via atomic write-and-replace.
- Round-tripped JSON config losslessly with explicit encoding.
- Eliminated leaked file handles using context managers.
- Portable paths via pathlib across Linux and Windows.