Skip to content

Regex Substitution

Regular expressions (regex) are indispensable for text processing in Linux sysadmin workflows. Tools like sed (stream editor) and tr (translate) enable powerful substitution capabilities, from modifying configuration files to sanitizing log data. This section explores how to leverage regex for substitution using these utilities.

sed Substitution: Regex-Powered Text Manipulation

sed is the workhorse for regex-based substitution. Its s (substitute) command allows replacing patterns in text, with optional flags for control.

Basic Substitution

Replace the first occurrence of a pattern in a line: 

echo "Hello, world!" | sed 's/world/Universe/'
Output: 
Hello, Universe!

Global Substitution

Use the g flag to replace all matches in a line: 

echo "apple, banana, apple" | sed 's/apple/pear/g'
Output: 
pear, banana, pear

Case-Insensitive Replacement

Add the I flag for case-insensitive matching: 

echo "Password123" | sed 's/password/secret/I'
Output: 
Secret123

In-Place File Editing

Edit files directly with -i: 

sed -i 's/old_pattern/new_pattern/g' /path/to/file

Advanced: Line Replacement

Use c to replace an entire line matching a pattern: 

sed '/^old_line/c\new_line' /path/to/file

tr Translation: Character-Level Substitution

tr (translate) replaces characters in input, but it operates on a character-by-character basis rather than regex. It’s ideal for simple mappings.

Basic Character Translation

Replace all a with A and b with B: 

echo "abba" | tr 'ab' 'AB'
Output: 
ABBA

Removing Characters

Delete all digits from input: 

echo "abc123" | tr -d '0-9'
Output: 
abc

Case Conversion

Convert lowercase to uppercase: 

echo "Hello" | tr '[:lower:]' '[:upper:]'
Output: 
HELLO

Advanced Techniques

Combining sed with Pipes

Process multiline logs or files in pipelines: 

tail -f /var/log/syslog | sed '/ERROR/s/^/⚠ /'
Prefixes lines containing "ERROR" with a warning emoji.

Regex Anchors and Ranges

Use anchors like ^ (start of line) and $ (end of line) in sed: 

sed '/^#/d' /path/to/file  # Delete lines starting with a comment

Escaping Special Characters

Escape regex metacharacters in sed: 

echo "file[123]" | sed 's/\[/\[/g'  # Replace [ with [

Key takeaways

  • sed is essential for regex-based substitution, supporting flags like g (global) and I (case-insensitive).
  • tr excels at character-level translation but lacks regex capabilities.
  • Use sed -i for in-place file edits, and combine sed with pipelines for real-time data processing.
  • Always test substitutions with echo before applying them to critical files.