awk, sed and grep in linux

awk, sed, and grep are powerful command-line utilities in Unix-like operating systems, each designed for different text processing tasks. Here’s an overview of each tool along with some examples:

awk

awk is a scripting language that is particularly good for pattern scanning and text processing. It’s designed to handle structured data, such as columns in a table.

Basic Syntax:

awk 'pattern { action }' input-file

Examples:

  • Print the second field of each line in a file:
  awk '{print $2}' input.txt
  • Sum all numbers in the first column and print the total at the end:
  awk '{sum += $1} END {print sum}' input.txt
  • Print lines that contain the word “error”:
  awk '/error/' input.txt

sed

sed (Stream Editor) is a stream editor for filtering and transforming text. It’s often used for in-place editing of files.

Basic Syntax:

sed 's/old/new/g' input-file

Examples:

  • Replace all occurrences of “old” with “new” in a file:
  sed 's/old/new/g' input.txt
  • Delete lines containing the word “error”:
  sed '/error/d' input.txt
  • Print lines 10 to 20 from a file:
  sed -n '10,20p' input.txt

grep

grep is used for searching text or searching the output of other commands for lines that match a regular expression.

Basic Syntax:

grep 'pattern' [file...]

Examples:

  • Find lines containing the word “error”:
  grep 'error' input.txt
  • Use regular expressions to find lines starting with “Error”:
  grep '^Error' input.txt
  • Recursively search for the word “error” in all files of a directory:
  grep -r 'error' /path/to/directory

Combined Usage

These tools can be combined in a pipeline to perform more complex text processing tasks.

Example:

  • Use grep to filter lines, then awk to process them, and finally sed to make replacements:
  grep 'error' input.txt | awk '{print $2}' | sed 's/old/new/g'

Summary

  • awk is powerful for complex text processing and handling structured data.
  • sed is ideal for stream editing and simple text transformations.
  • grep is best for searching text based on patterns.
  • By combining these tools, one can perform a wide range of text manipulations efficiently.

Each of these utilities has a wide range of options and can be used in many different ways to suit specific needs in text processing tasks.

This entry was posted in Linux awk, Linux sed, Linux Text Processing and tagged , , . Bookmark the permalink.