sed is the non-interactive stream editor of Linux and other Unix-like systems: it reads text one line at a time, applies a script of editing commands to each line, and writes the result to standard output. Within the Shell & terminal toolset it occupies the space between grep, which only selects lines, and awk, which is a full record-processing language.
Because sed never opens a file for interactive editing, it fits naturally into pipelines, shell scripts and automation. Administrators use it to rewrite configuration values, clean log output and transform data streams without loading entire files into memory.
How sed processes a stream of text
sed works through a read, execute, print cycle that repeats for every input line. It copies the current line into a buffer called the pattern space, removes the trailing newline, runs every command of the script against that buffer, and then prints the buffer unless printing was suppressed.
A second buffer, the hold space, survives between cycles. Commands such as h, g and x copy or exchange data between the two buffers, which allows multi-line operations such as joining lines or reversing their order. Most everyday scripts never touch the hold space, but it is the reason sed can solve problems that look impossible for a line editor.
Each command may carry an address that limits where it applies. Without an address, a command runs on every line; with one, it runs only on matching lines or ranges. The following table summarizes the address forms that sed accepts.
| Address form | Example | Lines selected |
|---|---|---|
| Line number | 3d | Only line 3 |
| Last line | $p | The final line of input |
| Regular expression | /error/d | Every line that matches the pattern |
| Range | 10,20s/a/b/ | Lines 10 through 20 inclusive |
| Pattern range | /start/,/end/d | From a matching line to the next closing match |
| Step (GNU) | 0~2d | Every second line |
| Negation | /^#/!p | Every line that does not match |
Core sed commands and options
The substitute command s/regex/replacement/flags accounts for most real-world sed usage. The g flag replaces every occurrence on a line instead of only the first, a number such as 2 targets a specific occurrence, and I in GNU sed makes matching case-insensitive. An ampersand in the replacement inserts the whole matched text, while \1 to \9 insert captured groups.
The table below lists the commands that cover the majority of administrative tasks.
| Command | Purpose | Typical use |
|---|---|---|
| s | Substitute text matched by a regex | Change a configuration value |
| d | Delete the pattern space | Remove comments or blank lines |
| p | Print the pattern space | Extract lines together with -n |
| a, i, c | Append, insert or change lines | Add a directive after a header |
| y | Transliterate characters | Convert separators one to one |
| q | Quit after the current line | Stop early on large files |
| N, D, P | Work on multi-line buffers | Join wrapped lines |
Command-line options change how the whole script behaves. The option -n disables automatic printing, so only lines explicitly printed with p appear. The option -e adds a script fragment, -f reads the script from a file, and -E switches to extended regular expressions, which removes the need to escape parentheses and the plus sign.
The option -i performs in-place editing: sed writes output to a temporary file and renames it over the original. Supplying a suffix, as in -i.bak, keeps a backup copy of the unmodified file, which is the safest habit when editing system configuration.
sed compared with grep, awk and other tools
sed is the right choice when the job is line-oriented transformation driven by patterns. It is weaker when the job requires arithmetic, field-aware logic or structured data, where other tools in the shell are more suitable. The comparison below helps select the correct tool.
| Tool | Primary strength | Best suited for |
|---|---|---|
| grep | Fast line selection by pattern | Finding matching lines without changing them |
| sed | Pattern-driven editing of a stream | Substitutions, deletions and in-place config edits |
| awk | Field splitting and computation | Column reports, sums and conditional logic |
| tr | Character-level translation | Changing case or squeezing repeated characters |
| perl | Full programming language | Complex multi-line or Unicode-heavy rewrites |
| jq | JSON-aware querying | Editing structured JSON safely |
In practice these tools are combined through a pipe. A command such as grep -v '^#' app.conf | sed 's/ *= */=/' first strips comments and then normalizes spacing, each program doing one task well in keeping with the Unix philosophy.
GNU sed, BSD sed and portability
Most Linux distributions ship GNU sed, while macOS and the BSD systems ship BSD sed. Both implement the POSIX specification, but their extensions differ, and a script written on one platform can fail silently or loudly on the other.
The most frequent trap is in-place editing syntax. GNU sed accepts -i alone, whereas BSD sed requires an explicit suffix argument, so portable scripts write -i.bak or avoid -i entirely by redirecting to a temporary file and moving it.
| Feature | GNU sed | BSD sed |
|---|---|---|
| In-place edit without backup | -i | -i '' |
| Extended regex flag | -E or -r | -E |
| Case-insensitive flag | Supported with I | Not supported |
| Newline escape in replacement | \n works | Requires a literal newline |
| Step addresses such as 0~2 | Supported | Not supported |
For scripts that must run on many systems, POSIX-only syntax and basic regular expressions are the most reliable baseline. Testing on each target platform remains the only definitive check.
Common mistakes and how to avoid them
Most sed failures come from quoting, delimiters and regex dialects, not from the commands themselves. The following table pairs typical symptoms with their causes and fixes.
| Symptom | Cause | Fix |
|---|---|---|
| Unknown option to s | Slash inside a path breaks the delimiter | Use another delimiter, such as s|/old|/new| |
| Variable not expanded | Script wrapped in single quotes | Use double quotes around the variable part |
| Only first match replaced | Missing g flag | Append g to the substitution |
| Parentheses matched as plain characters | Basic regex syntax in use | Add -E or escape the parentheses |
| Original file lost | Unchecked -i edit | Use -i.bak and review a dry run first |
| Output printed twice | p without -n | Add -n to suppress automatic printing |
A dependable workflow is to run the script without -i first, inspect the output or a diff, and only then apply the change in place. When an expression grows beyond a few commands, moving it into a script file with comments improves readability and review.
sed is not a parser for structured formats. Editing XML, YAML or JSON with regular expressions works for trivial cases but breaks on nesting, multi-line values and quoting, so format-aware tools are the safer choice for those files.
In simple terms
sed is like a find-and-replace robot for text files. Instead of opening a document and changing words by hand, a person writes a short instruction, and sed applies it to every line automatically.
It reads text from a file or from another program, changes what the instruction says, and passes the result along. Because it needs no screen and no mouse, it can edit hundreds of files in seconds and is a favorite tool for anyone who automates work on Linux.
A practical example
A systems administrator manages forty web servers whose configuration files still point to an old database host. Logging in to each machine and editing the file manually would take hours and invite typing errors.
Instead, the administrator writes a single command, sed -i.bak 's/db-old.internal/db-new.internal/g' /etc/app/app.conf, tests it on one server, compares the result with the backup using diff, and then runs it on all forty machines through a remote execution tool.
The whole change completes in under five minutes, every server keeps a backup of its original file, and a rollback would require only renaming the backup. The same approach later becomes part of the team’s deployment script, so future host changes need no manual editing at all.