Text Processing with sed Stream Editing and awk Fields
I transformed text non-interactively with sed substitutions for find-replace and awk for field-based reporting and arithmetic. These tools edited config files in place and produced column summaries from logs without ever opening an editor.
Objective & Context
sed and awk are the scripting backbone of text automation. This lab uses sed for line-oriented edits and awk's record/field model for computation, the manual engine behind config templating and log reporting.
Environment & Prerequisites
- Linux with GNU sed and gawk.
- A config file and a delimited log.
- Regex familiarity from the grep lab.
flowchart LR
In[Stream] --> SED[sed: line edits]
In --> AWK[awk: fields + math]
SED --> O1[Edited file]
AWK --> O2[Column report]
Step-by-Step Execution
1. In-place substitution with sed
sed -i.bak 's/^#Port 22/Port 2222/' /etc/ssh/sshd_config2. Sum a column with awk
awk -F, '{sum+=$3} END{printf "total: %.2f\n", sum}' sales.csv3. Conditional field report
awk '$9 >= 500 {print $1, $9}' access.logtotal: 18423.50
203.0.113.45 503Validation & Testing
Confirm sed created a .bak backup and changed only the intended line, and that awk's totals match a manual sum. Pass criteria: targeted in-place edit with backup, correct field math, and accurate conditional filtering.
Advanced: Troubleshooting
- sed changed too much: anchor the pattern; unanchored matches hit every line.
- No in-place backup: use
-i.bakto keep the original. - awk wrong field: set the field separator with
-Fto match the data.
Key Results
- Edited config files in place with safe backups and anchored patterns.
- Computed column totals and reports directly from delimited data.
- Filtered records by field conditions without a full script.
- Replaced manual editing for repeatable text transforms.