|
|
| Author |
Message |
chris
Dark Lord of the Sith
Joined: 10 May 2003
Posts: 6257
Location: Outer Space
|
 Posted: Tue Apr 20, 2004 9:01 pm Post subject: How to search recursively the files of a given directory |
|
 |
|
 Problem: Find my_search_pattern in *all* files of /directory/to/search and *all* directories beneath it. Take special care for directories with blanks in their names. Your solution must work even if there are thousands of files to check. This means that solutions that are in principle correct, but give errors of type:
| Code: |
Argument list too long
|
will NOT be accepted!
 Solution:
| Code: |
/usr/bin/find /directory/to/search -type f -print0 | /usr/bin/xargs -0 /usr/bin/grep my_search_pattern
|
Some remarks are in order:
The pipe to xargs will take care of an excessive number of files automatically - if there are too many of them, it will process them through grep and then will continue to process the rest!
The -print0 option tells find to append a 0 after each filename, the -0 option to xargs tells it to delimit filenames by 0 and not by blank. Thus we are able to process files with blanks in their names.
I was tired of having to type this each and every time I need it, so I wrote a script. I call it grepdir, it has the same syntax as grep and comes with a rudimentary help function which outputs the syntax and an example if you call it without arguments. I use grepdir quite often. Here it is:
| Code: |
#! /bin/sh
#
# /usr/local/bin/grepdir
#
# Author: Chris Karakas
# http://www.karakas-online.de
#
# Searches the files of directory DIR recursively for
# the expression EXP, excluding the contents of EXCLUDE_DIR
# from the search, if any.
#
# Usage: grepdir EXP DIR [EXCLUDE_DIR]
# Help function
#
function help() {
cat <<-EOF
Usage: grepdir EXP DIR [EXCLUDE_DIR]
Searches the files of directory DIR recursively for
the expression EXP, excluding the contents of EXCLUDE_DIR
from the search, if any.
-h, --help Display this help text
EOF
}
# Check arguments and issue a help statement, if wrong
#
if [ $# -eq 0 ]; then
help
exit 1
elif [ $1 = "-h" -o $1 = "--help" ]; then
help
exit 0
elif [ $# -eq 1 -o $# -gt 3 ]; then
help
exit 1
fi
FIND="/usr/bin/find"
XARGS="/usr/bin/xargs"
GREP="/usr/bin/grep"
# Program name
PN=${0##*/}
arg1=$1
shift
if [[ X"$2" = X"" ]]; then
echo "$PN: running $FIND $1 -type f -print0 | $XARGS -0 $GREP $arg1"
$FIND $1 -type f -print0 | $XARGS -0 $GREP $arg1
else
echo "$PN: running $FIND $1 -path $2 -prune -o -type f -print0 | $XARGS -0 $GREP $arg1"
$FIND $1 -path $2 -prune -o -type f -print0 | $XARGS -0 $GREP $arg1
fi
|
Just type "grepdir" or "grepdir -h" to get a help message.
_________________
Regards
Chris Karakas
www.karakas-online.de
Last edited by chris on Thu Apr 22, 2004 12:26 pm; edited 1 time in total |
|
| Back to top |
|
 |
chris
Dark Lord of the Sith
Joined: 10 May 2003
Posts: 6257
Location: Outer Space
|
| Posted: Thu Apr 22, 2004 12:05 pm Post subject: |
|
|
|
There is only a small inadequacy in the above script: you cannot serach for a phrase that contains blanks:
Suppose I want to search for "console with" in all files under /etc. If I try only to escape the blank:
| Code: |
grepdir console\ with /etc/
|
I get:
| Code: |
grepdir: running /usr/bin/find /etc/ -type f -print0 | /usr/bin/xargs -0 /usr/bin/grep console with
grep: with: No such file or directory
|
If I try
| Code: |
grepdir "console with" /etc/
|
I get:
| Code: |
/usr/local/bin/grepdir: line 32: [: too many arguments
grepdir: running /usr/bin/find /etc/ -type f -print0 | /usr/bin/xargs -0 /usr/bin/grep console with
grep: with: No such file or directory
|
If I try:
| Code: |
grepdir "console\ with" /etc/
|
I get:
| Code: |
grepdir: running /usr/bin/find /etc/ -type f -print0 | /usr/bin/xargs -0 /usr/bin/grep console\ with
grep: Trailing backslash
grep: Trailing backslash
grep: Trailing backslash
grep: Trailing backslash
|
Also, sometimes I get garbage that looks like
What am I doing wrong?
_________________
Regards
Chris Karakas
www.karakas-online.de |
|
| Back to top |
|
|
chris
Dark Lord of the Sith
Joined: 10 May 2003
Posts: 6257
Location: Outer Space
|
| Posted: Thu Apr 22, 2004 12:17 pm Post subject: |
|
|
|
Thanks to Kerry for pointing this to me:
$arg1 must also be enclosed in quotes, then you can use
| Code: |
grepdir "console with" /etc/
|
for search for "console with" in all files in and under /etc.
The corrected script source for grepdir is therefore:
| Code: |
#! /bin/sh
#
# /usr/local/bin/grepdir
#
# Author: Chris Karakas
# http://www.karakas-online.de
#
# Searches the files of directory DIR recursively for
# the expression EXP, excluding the contents of EXCLUDE_DIR
# from the search, if any.
#
# Usage: grepdir EXP DIR [EXCLUDE_DIR]
# Help function
#
function help() {
cat <<-EOF
Usage: grepdir EXP DIR [EXCLUDE_DIR]
Searches the files of directory DIR recursively for
the expression EXP, excluding the contents of EXCLUDE_DIR
from the search, if any.
-h, --help Display this help text
EOF
}
# Check arguments and issue a help statement, if wrong
#
if [ $# -eq 0 ]; then
help
exit 1
elif [ "$1" = "-h" -o "$1" = "--help" ]; then
help
exit 0
elif [ $# -eq 1 -o $# -gt 3 ]; then
help
exit 1
fi
FIND="/usr/bin/find"
XARGS="/usr/bin/xargs"
GREP="/usr/bin/grep"
# Program name
PN=${0##*/}
arg1=$1
shift
if [[ X"$2" = X"" ]]; then
echo "$PN: running $FIND $1 -type f -print0 | $XARGS -0 $GREP \"$arg1\""
$FIND $1 -type f -print0 | $XARGS -0 $GREP "$arg1"
else
echo "$PN: running $FIND $1 -path $2 -prune -o -type f -print0 | $XARGS -0 $GREP \"$arg1\""
$FIND $1 -path $2 -prune -o -type f -print0 | $XARGS -0 $GREP "$arg1"
fi
|
_________________
Regards
Chris Karakas
www.karakas-online.de |
|
| Back to top |
|
|
|
|
|
|
|
FAQ
Forum Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
