Taki Sadik
Cybersecurity & IT Professional, North York, ON, Canada
Contact

Python File Operations with pathlib, Context Managers, and JSON

BeginnerPython~2 hours
pathlibcontext managersJSONatomic write

I built safe file I/O in Python using pathlib, with context managers, and the json module, including an atomic write pattern. Writing to a temp file and renaming guaranteed config files were never left half-written on failure.

Objective & Context

Automation scripts read and rewrite config and state files constantly. This lab uses context managers for guaranteed close, pathlib for portable paths, and atomic replace to prevent corruption, the I/O backbone for the sysadmin and IR scripts.

Learning Objectives

  • Navigate the filesystem with pathlib instead of string path juggling.
  • Guarantee file closure with with context managers.
  • Read and write JSON configs with the json module.
  • Implement atomic writes via temp file plus rename.
  • Verify a write round trip by reading back and comparing.

Environment & Prerequisites

Prerequisites

  • Python 3.11 with pathlib and json (standard library)
  • A writable working directory and a sample JSON config to manipulate
  • Assumed skill: basic Python syntax and dicts

Step-by-Step Execution

1. Read JSON with a context manager

echo '{"name": "tyf-ai", "version": 1}' > config.json

Expected output:

(no output; sample config.json written)
import json
with open("config.json", encoding="utf-8") as f:
    cfg = json.load(f)

Expected output:

(no output; cfg holds the parsed dict)

What this teaches: The with block guarantees the file is closed even if json.load raises, and explicit encoding="utf-8" avoids platform-default surprises on Windows. For one-shot reads, pathlib.Path("config.json").read_text() is the shorter equivalent.

2. Atomic write to avoid corruption

import os, tempfile, pathlib
def atomic_write(path, text):
    d = pathlib.Path(path).parent
    fd, tmp = tempfile.mkstemp(dir=d)
    with os.fdopen(fd, "w") as f:
        f.write(text); f.flush(); os.fsync(f.fileno())
    os.replace(tmp, path)

Expected output:

(no output; atomic_write is defined)

What this teaches: Writing to a temp file in the same directory and then renaming makes the update atomic: readers see the old file or the new file, never a half-written one.

3. Verify the round trip

python -c "import json,pathlib; print(json.loads(pathlib.Path('config.json').read_text())['name'])"

Expected output:

tyf-ai

What this teaches: Reading back the written file and extracting a known value closes the loop and proves the round trip preserved the data. This reads the config.json created in step 1, so run both steps in the same directory.

How It Works

How It Works

pathlib treats paths as objects with methods instead of strings you concatenate. The / operator joins segments portably, and methods like read_text(), write_text(), exists(), and glob() cover most file tasks without ever importing os.

Context managers exist because cleanup must happen even when code raises. The with block calls __exit__ on the way out no matter what, which closes the file handle; without it, an exception mid-write can leak handles or leave buffers unflushed.

Atomic writes exploit one filesystem guarantee: renaming a file within the same directory is atomic. By writing complete content to a temp file and then os.replace()-ing it over the target, concurrent readers and crash recovery only ever observe whole versions.

Validation & Testing

Interrupt a write mid-stream and confirm the original file remains intact thanks to atomic replace. Pass criteria: JSON round-trips losslessly and an aborted write never corrupts the target file.

Advanced: Troubleshooting
  • Partial files: never write in place; write-temp-then-replace.
  • Encoding issues: specify encoding="utf-8" explicitly.
  • Leaked handles: always use with to guarantee close.

Troubleshooting

SymptomLikely causeFix
FileNotFoundError on readrelative path resolved against the wrong working directorybuild paths from pathlib.Path(__file__).parent instead of relying on cwd
JSONDecodeErrorthe file is empty, truncated, or not JSONvalidate with json.tool on the command line before debugging code
PermissionError on renametemp file created in a different filesystem or directorycreate the temp file in the target's own directory with dir=parent
data lost after a crashcontent sat in the OS buffer and never reached diskcall f.flush() plus os.fsync() before the rename on critical writes
pathlib glob finds nothingpattern relative to the wrong base or wrong case on case-sensitive filesystemsprint the resolved base path and list the directory first

Key Results

  • Guaranteed zero file corruption via atomic write-and-replace.
  • Round-tripped JSON config losslessly with explicit encoding.
  • Eliminated leaked file handles using context managers.
  • Portable paths via pathlib across Linux and Windows.

Key Takeaways

  • pathlib replaces string path juggling with readable, portable path objects.
  • with blocks guarantee cleanup even when exceptions strike mid-operation.
  • Atomic writes (temp file plus rename) protect configs from partial updates.
  • Keep temp files in the target directory so the rename stays atomic.
  • Always verify writes with a read-back round trip in scripts that matter.

Practice Challenges

Challenge 1: Config updater

Goal: Write a script that loads a JSON config, bumps a version field, and saves it atomically, then proves the old file is untouched on simulated failure.

Hint: Raise an exception between write and rename to simulate the crash; check the original survives.

Challenge 2: Safe rotator

Goal: Implement log rotation: rename app.log to app.log.1 only if it exists, using pathlib, without race-prone exists-then-act patterns.

Hint: try/except around the rename handles the missing-file case atomically.