Practical Guide to Using Find Command for Time-Based File Searches in Linux and Unix Systems

In Linux and Unix systems, the find command is a very useful tool for searching files and directories based on various conditions. Among its functionalities, finding files based on time is a common use case. By specifying different time conditions, we can locate files that meet specific time-related criteria, such as modification time, access time, and status change time.

Syntax

The basic syntax for using the find command to search for files based on time is as follows:

find <path> -<time_condition> <time_parameter>

Where represents the directory path to search, denotes the time condition (e.g., -mtime for modification time, -atime for access time), and specifies the time parameter (e.g., +n for greater than n days, -n for less than n days).

Examples

1. Find files modified within the last 7 days:

find /path/to/directory -mtime -7

In this example, the -mtime parameter is used to search for files based on modification time, and -7 indicates files modified within the last 7 days.

2. Find files accessed within the last 30 days:

find /path/to/directory -atime -30

Here, the -atime parameter is used to search for files based on access time, and -30 indicates files accessed within the last 30 days.

3. Find files with status changes within the last 1 hour:

find /path/to/directory -cmin -60

In this example, the -cmin parameter is used to search for files based on status change time, and -60 indicates files with status changes within the last 1 hour.

Using the find command to search for files based on time allows for the convenient location of files and directories that meet specific time-related criteria. By understanding the syntax and examples of the find command, users can perform file searches and management more effectively and flexibly.

Posted in File & Directory, Find Command | Tagged , , | Comments Off on Practical Guide to Using Find Command for Time-Based File Searches in Linux and Unix Systems

Awk – F parameter tutorial

The awk command is a powerful text processing tool that allows for effective manipulation of structured text data. It is commonly used to read data from files or pipelines, process the data, and then output the results. One of the key features of awk is its ability to use field separators to process data in columns.

Syntax:

The basic syntax for using the -F parameter in awk is:

awk -F 'separator' 'pattern { action }' filename

Examples:

1. Using comma as a field separator Assuming we have a file named data.csv containing the following content: John,Doe,25 Jane,Smith,30. We can use the awk command with the -F parameter to process the data using comma as the field separator

awk -F ',' '{print $1,$3}' data.csv

2. Using tab as a field separator Assuming we have a file named data.tsv containing the following content: Alice 100 Bob 150. We can use the awk command with the -F parameter to process the data using tab as the field separator:

awk -F '\t' '{print $2}' data.csv

3. Using multiple characters as a field separator Assuming we have a file named data.txt containing the following content: 2021-10-01;apple;10 2021-10-02;banana;20. We can use the awk command with the -F parameter to process the data using semicolon as the field separator:

awk -F '; ' '{print $2}' data.csv

Posted in Awk Command, Text Processing | Tagged , | Comments Off on Awk – F parameter tutorial

Awk BEGIN/END tutorial

In AWK, the BEGIN and END blocks are special code blocks that are executed before and after processing the input, respectively. These blocks are useful for initializing variables, performing cleanup operations, or generating summary reports.

Syntax:

The syntax for BEGIN and END blocks in AWK is as follows:

BEGIN {
# Code to be executed before processing the input
}

END {
# Code to be executed after processing the input
}


Examples:

1. Counting the number of words in a file

BEGIN {
count = 0
}

{
count += NF
}

END {
print "Total words: " count
}
awk 'BEGIN { count = 0 } { count += NF } END { print "Total words: " count}' input.txt

In this example, the BEGIN block initializes the variable count, and then for each line, the number of fields (i.e., words) is added to count. Finally, the END block prints the total number of words.

2. Calculating the average of numbers in a file

BEGIN {
    sum = 0
    count = 0
}

{
    for (i = 1; i <= NF; i++) {
        if ($i ~ /^[0-9]+$/) {
            sum += $i
            count++
        }
    }
}

END {
    print "Average: " sum / count
}
awk 'BEGIN {sum = 0;count = 0;}{ for (i = 1; i <= NF; i++) {if ($i ~ /^[0-9]+$/) {sum += $i;count++;}}} END { print "Average: " sum / count }' input-1.txt

In this example, the BEGIN block initializes the sum and count variables. For each line, it iterates through the fields, checks if they are numbers, and adds them to the sum while incrementing the count. The END block then prints the average value.

3. Reversing all lines in a file

BEGIN {
    RS = "\n"
    FS = ""
    ORS = "\n"
}

{
    for (i = NF; i > 0; i--) {
        printf "%s ", $i
    }
    printf "\n"
}
awk 'BEGIN {RS = "\n";FS = "";ORS = "\n";}{ for (i = NF; i > 0; i--) { printf "%s ", $i; } printf "\n"}' input.txt

In this example, the BEGIN block sets the input record separator (RS), field separator (FS), and output record separator (ORS). For each line, it iterates through the fields in reverse order and prints them, effectively reversing the lines in the file.

In summary, BEGIN and END blocks in AWK are powerful tools for performing operations before and after processing input, and they can be used for a wide range of tasks such as initialization, cleanup, and generating summary reports.

Posted in Awk Command, Text Processing | Tagged , | Comments Off on Awk BEGIN/END tutorial

Awk time functions: systime, mktime, strftime

Today, I would like to share with you how to use the awk time function:

  • systime
    Get the timestamp and return the whole seconds from January 1, 1970 to the current time (excluding leap years)
  • mktime
    Create the specified time and convert it to timestamp
  • strftime
    Format the time output and convert the timestamp to a time string

Syntax

awk 'pattern {  action }'

Example

In the following example, we use the awk systime function to get the current timestamp.

awk 'BEGIN{nowTimestamp=systime();print nowTimestamp;}'

In the following example, we use the awk mktime function to create a timestamp at the specified time.

awk 'BEGIN{print "Number of seconds since the Epoch = " mktime("2022 10 08 20 20 10")}'

In the following example, we use the awk strftime function to output the timestamp formatted time string.

awk 'BEGIN {print strftime("Time = %m/%d/%Y %H:%M:%S", 1665140876)}'

Posted in Awk Command, Text Processing | Tagged , , | Comments Off on Awk time functions: systime, mktime, strftime

Awk built-in functions: split, tolower, toupper, sprintf

Today we will share four commonly used awk built-in string functions:

  • split(String, A, [Ere])
    Splits the parameter specified by the String parameter into the array elements A[1], A[2], . . ., A[n], and returns the value of the n variable. This separation can be done with the extended regular expression specified by the Ere parameter, or with the current field separator (FS special variable) (if no Ere parameter is given)
  • tolower( String )
    Returns the string specified by the String parameter, each uppercase character in the string will be changed to lowercase
  • toupper( String )
    Returns the string specified by the String parameter, each lowercase character in the string will be changed to uppercase
  • sprintf( Format, Expr, …)
    Formats the expression specified by the Expr parameter according to the printf subroutine format string specified by the Format parameter and returns the resulting string.

Syntax

awk 'pattern {  action }'

Example

In the following example, we use the awk split function to split the string str into an array according to the fixed delimiter “,”, and print the third item of the array.

awk 'BEGIN { 
    str = "a b c d";
    split(str, arr, " ");
    print arr[3];
}'

In the following example, we use the awk tolower function to convert all strings to lowercase.

awk 'BEGIN { 
    str = "Linux Command Site!";
    newStr = tolower(str);
    print newStr;
}'

In the following example, we use the awk toupper function to convert all strings to uppercase.

awk 'BEGIN { 
    str = "Linux Command Site!";
    newStr = toupper(str);
    print newStr;
}'

In the following example, we use the awk sprintf function to convert the string according to the format.

awk 'BEGIN { 
    str = "%s Command Site!";
    newStr = sprintf(str, "Linux");
    print newStr;
}'

Posted in Awk Command, Text Processing | Tagged , , | Comments Off on Awk built-in functions: split, tolower, toupper, sprintf

Batch change file extension in Linux

In the daily development process, we sometimes need to do some batch operations, as described in the previous article.


Today, we will share how to use Linux commands to change file extensions in batches.

First, we create files with txt extensions in batch under the current directory.

➜ awk 'BEGIN {do {++i; system("touch file_num_" i ".txt") } while (i<9) }'

Next, we will introduce two change methods:

Method 1: use the for loop

Use the for loop to change the txt extension of all files in the current directory to json.

➜ for i in *.txt;do mv "$i" "${i%.txt}.json" ;done

Method 2: use the awk command

Use the find command and awk command to batch change the json file extension in the current directory to txt.

➜ find . -type f -name "*.json" | awk -F"[./]" '{system("mv "$0" " $3".txt")}'

ok.
Is there any better way? Comments are welcome.

Posted in Internet Technology | Tagged , , | Comments Off on Batch change file extension in Linux

Awk built-in functions: index, match, length

Awk’s built-in string processing function is the one we use most.

Today, we share with you: index, match, length

  • index(s, t)
                  the position in s where the string t occurs, or 0 if it does not.
  • match(s, r)
                  the position in s where the regular expression r occurs, or 0 if it does not. 
  • length(s) 
          the length of its argument taken as a string, number of elements in an array for an array argument, or length of $0 if no argument.

Syntax

	awk 'pattern {  action }'

Example

In the following example, we use the awk index function to return the position where the string t appears in the string s. if the string t does not appear in the string s, it returns 0.

➜  ~ awk 'BEGIN{str="linux command. linux is the best";position=index(str, "the");print position;}'
25

In the following example, we use the awk match function to return the position where the string t appears in the string s and the length of the character T. The variables RSTART and RLENGTH are set to the position and length of the matched string. If not found, the variable rstart is 0 and rlength is – 1;

➜  ~  awk 'BEGIN{str="linux command. linux is the best";match(str, "the");print RSTART, RLENGTH;}'
25 3

➜  ~  awk 'BEGIN{str="linux command. linux is the best";match(str, "th1e");print RSTART, RLENGTH;}'
0 -1

In the following example, we use the awk length function to return the string length.

➜  ~  awk 'BEGIN{str="linux command. linux is the best";len=length(str);print len;}'
32

Posted in Awk Command, Text Processing | Tagged , , | Comments Off on Awk built-in functions: index, match, length

Awk built-in functions: gsub, sub, substr

Awk’s built-in string processing function is the one we use most.

Today, we share with you: gsub, sub, substr

  • substr(s, m [, n])
    the n-character substring of s that begins at position m counted from 1.  If no n, use the rest of the string.
  • sub(r, t [, s])
    substitutes t for the first occurrence of the regular expression r in the string s.  If s is not given, $0 is used.
  • gsub(r, t [, s])
    same as sub except that all occurrences of the regular expression are replaced; sub and gsub return the number of replacements.

Syntax

awk 'pattern {  action }'

Example

In the following example, we use the awk sub built-in function to perform a substring replacement, which replaces the first substring with t. The third parameter is optional and defaults to $0.

➜  ~ awk 'BEGIN{ str="linux command"; sub("linux", "unix", str);print str; }'
unix command

➜  ~ awk 'BEGIN{ str="linux command. linux is the best"; sub("linux", "unix", str);print str; }'
unix command. linux is the best

In the following example, we use the awk substr function to return a substring of length n from the Nth character in the string.

➜  ~ awk 'BEGIN{ str="linux command"; subs=substr(str,1,5);print subs; }'
linux

In the following example, we use the awk gsub built-in function for global replacement. It replaces all substrings that appear with t.

➜  ~ awk 'BEGIN{ str="linux command. linux is the best"; gsub("linux", "unix", str);print str; }'

unix command. unix is the best

Posted in Awk Command, Text Processing | Tagged , , , , | Comments Off on Awk built-in functions: gsub, sub, substr

awk print function in linux

Awk print function is a built-in function of awk, used for prints its arguments on the standard output (or on a file if > file or >> file is present or on a pipe if | cmd is present).

Syntax

➜  ~ awk 'pattern {  print [ expression-list ] [ > expression ] }'

Example

First, we create a test file: test.log, which is as follows:

➜  ~ cat test.log
line 1 one
line 2 two
line 3 three
line 4 four
line 5 five

Example 1, use the awk print function to print each line of the file.

➜  ~ awk '{print $0}' test.log
line 1 one
line 2 two
line 3 three
line 4 four
line 5 five

Example 2, use the awk print function to print the second field of each line of the file.

Awk uses space as delimiters by default. You can use the -F option here to specify the delimiter or not.

➜  ~ awk '{print $2}' test.log
1
2
3
4
5
➜  ~ awk -F" " '{print $2}' test.log
1
2
3
4
5

Example 3, use the awk print function to print the last field of each line of the file.

➜  ~ awk '{print $NF}' test.log
one
two
three
four
five
- NF
    number of fields in the current record.

If you clearly know the fixed number of fields per line, you can print using the variable $N.

➜  ~ awk '{print $3}' test.log
one
two
three
four
five

Example 4, use the awk print function to print specific fields on each line of the file.

➜  ~ awk '{print $1,$3}' test.log
line one
line two
line three
line four
line five

Example 5, use the awk print function to print the specified line of the file.

➜  ~ awk '{if(NR == 2 || NR == 3){print $0}}' test.log
line 2 two
line 3 three
- NR     
    ordinal number of the current record.

Example 6, use the awk print function to print the line matching the pattern.

➜  ~ awk '/line 1/{print $0}' test.log
line 1 one

Example 7, use the awk print function to print the line matching the pattern and output it to the specified file.

➜  ~ awk '/line 1/{print $0 > "out.log"}' test.log
➜  ~ cat out.log
line 1 one

Posted in Awk Command, Text Processing | Tagged , , | Comments Off on awk print function in linux

ls -r command in Linux

ls -r option lists all files and directories in the current directory in reverse order.

Syntax

$ ls -r [options] [file|dir]

Examples

Before using the ls -r option, let’s look at the default of the ls command.

In the following example, we use the ls -r option to list all files and directories in the current directory in reverse order.

➜  ~ ls -r

In the above example, we use the ls -r option, but this is not easy to read.

In the following example, we use the ls -rl option to list all files and directories in the current directory in a long list in reverse order.

➜  ~ ls -rl

For better reading, we can add the – t option to sort by time based on the above options. Long listing format sorted by date / time in reverse order:

➜  ~ ls -rlt

Posted in File & Directory | Tagged , , , | Comments Off on ls -r command in Linux

ls -l command in Linux

ls -l option lists all files and directories in the current directory in long listing format.

Syntax

$ ls -l [options] [file|dir]

Examples

Before using the ls -l option, let’s look at the default of the ls command.

➜  ~ ls

In the following example, we will show you how to use the ls -l option to display files and directories in a long list.

➜  ~ ls -l

In the following example, we use the ls -al option to display all files and directories in the current directory in a long list, including hidden files.

➜  ~ ls -al
Posted in File & Directory | Tagged , , , | Comments Off on ls -l command in Linux

Awk built-in functions – mathematical functions

awk supports the following built-in mathematical functions:

  • atan2( y, x ) 
    Returns the arc tangent of y/x.
  • cos( x )
    Returns the cosine of x; x is radians.
  • exp( x )
    Returns the power function of x.
  • log( x )
    Returns the natural logarithm of x.
  • sin( x )
    Returns the sine of x; x is radians.
  • sqrt( x )
    Returns the square root of x.

Example

In the following example, we demonstrate the built-in function atan2 of awk.

➜  ~ awk 'BEGIN{
        PI = 3.1415926;
        x = -10;
        y = 10;
        result = atan2 (y,x) * 180 / PI;

        printf "The arc tangent for (x=%f, y=%f) is %f degrees\n", x, y, result
}'

In the following example, we demonstrate the built-in function cos of awk, which returns the cosine of X

➜  ~ awk 'BEGIN {
  PI = 3.1415926;
  param = 15;
  result = cos(param * PI / 180.0);

  printf "The cosine of %f degrees is %f.\n", param, result
}'

In the following example, we demonstrate the built-in function exp of awk, which returns the power function of x.

➜  ~ awk 'BEGIN {
  param = 15
  result = exp(param);

  printf "The exponential value of %f is %f.\n", param, result
}'

In the following example, we show you the built-in function sqrt of awk, which returns the square root of X.

➜  ~ awk 'BEGIN {
  param = 15
  result = sqrt(param)

  printf "sqrt(%f) = %f\n", param, result
}'

Posted in Awk Command, Text Processing | Tagged , , | Comments Off on Awk built-in functions – mathematical functions

ls -a command in Linux

ls -a option lists all files and directories in the current directory, including hidden files starting with “.”

Syntax

$ ls -a [options] [file|dir]

Examples

In the following example, we use the ls -a option to list files and directories, including hidden files starting with “.”.

➜  ~ ls -a

In the following example, we use the ls -al option to display files and directories in a long list format, including hidden files starting with “.”.

➜  ~ ls -al

Posted in File & Directory | Tagged , , | Comments Off on ls -a command in Linux

Awk built-in variables and custom variables

In awk, variables are divided into two types: built-in variables and user-defined variables.

Today, let’s introduce the commonly used built-in variables and their use methods and the use methods of user-defined variables.

Common awk built-in variables:

  • FILENAME
    the name of the current input file.
  • FNR
    ordinal number of the current record in the current file.
  • FS
    regular expression used to separate fields; also settable by option -Ffs.
  • NF
    number of fields in the current record.
  • NR
    ordinal number of the current record.
  • OFS
    output field separator (default space).
  • ORS
    output record separator (default newline).

Examples of using built-in variables

  1. Use the NF variable to output the number of fields in the current record.
➜  ~ awk '{print NF}' test.log

2. Use the NF variable to output the value of the last field in the current record.

➜  ~ awk '{print $NF}' test.log

3. Use the NR variable to output the serial number of the current record.

➜  ~ awk '{print NR, $0}' test.log

4. Use the FILENAME variable to output the file name of the processing file.

➜  ~ awk '{print FILENAME, NR, $0}' test.log

➜  ~ awk '{print FILENAME, NR, NF}' test.log

5. Use FS variable to separate records.

➜  ~ awk 'BEGIN{FS=" "}{print $1}' test.log
➜  ~ awk -F" " '{print $1}' test.log

6. Use the OFS variable to output the record separator.

➜  ~ awk 'BEGIN{FS=" ";OFS="-"}{print $1,$2}' test.log

User defined variables

There are two types of user-defined variables.

1. Use -v parameter, -v varname = value variable names are case sensitive

➜  ~ awk -v my_variable="hello" 'BEGIN{print my_variable}'

2. Reference variables defined on the command line

➜  ~ echo $my_variable
➜  ~ my_variable="hello"
➜  ~ awk -v var=$my_variable 'BEGIN{print var}'

Posted in Awk Command, Text Processing | Tagged , , | Comments Off on Awk built-in variables and custom variables

How to find empty lines in files in Linux

In Linux, how to find an empty line in a file and output an empty line number.


Recently, such problems have been encountered in processing data analysis.

This article will show you how I use Linux commands to find empty lines in a file.

There are three common ways to find empty lines in a file, using the grep and awk commands.

Method 1

In the following method, we use the grep command and regular expression to find empty lines in the file.

➜  ~ grep -n -e "^$" test.log

Explain:

n, indicating the number of output lines (starting from 1)

e, indicating specify a pattern used during the search of the input

^, indicating the beginning

$, indicating the end

Method 2 &Method 3

In the following two methods, we use the if conditional judgment expression of the awk command to find empty lines in the file.

➜  ~ awk '{if($0 == "") {print NR}}' test.log

➜  ~ awk '{if(!NF) {print NR}}' test.log

Explain:

Awk NR variable, which indicates the serial number of the current record. It is the line number when processing a single file.

Awk NF variable, indicating the number of fields in the current record.

Awk $0 variable, indicating the current sequence number value.

OK. The above is the method I use to find empty lines in the file.

Posted in Awk Command, File & Directory, Text Processing | Tagged , , | Comments Off on How to find empty lines in files in Linux