How to fast search for specified content in large files in linux

When we use linux, we often encounter the situation of searching for specified content in large files.

In this case, we directly use the grep command will be very, very slow, and will affect other normal running programs.

Here, it is recommended to use tail command or head command.

Of course, the premise is to know the approximate location of the search content in the file, the head or the tail of the file.

Syntax

tail -f -c 1G file.log | grep "test"

head -c 100m file.log | grep "test"

tail:

  • -c number
    The location is number bytes.
  • -f      
    The -f option causes tail to not stop when end of file is reached, but rather to wait for additional data to be appended to the input.

head:

  • -c number
    The location is number bytes.

Examples

If we confirm that the content to be searched is at the end of the file, then we can use the following command to search for the specified content in the large file.

tail -f -c 1G file.log | grep "test"

If we confirm that the content to be searched is at the head of the file, then we can use the following command to search for the specified content in the large file.

head -c 1m  file.log | grep "test"

Of course, in some cases, in order to better understand the situation, we need to output the context information of the specified content. At this time, the following commands can be used.

head -c 1m  file.log | grep -C 20 "test"

grep:

  • -A num, –after-context=num
    Print num lines of trailing context after each match.
  • -B num, –before-context=num
    Print num lines of leading context before each match.
  • -C[num, –context=num]
    Print num lines of leading and trailing context surrounding each match.
  • –colour=[when, –color=[when]]
    Mark up the matching text with the expression stored in GREP_COLOR environment variable.

Add a Comment

Your email address will not be published. Required fields are marked *