Wednesday, February 27, 2008

Bash Command-line Programming: Flow Control

Time for another post on handy techniques for command-line bash programming. This post covers some useful command-line techniques for flow control.

Even when writing quick programs on the command-line, I often need to branch or loop. Especially loop, as I often need to do something over every file in a directory, or every line in a file. Below are some techniques I commonly use. For more neat bash-isms, check out the Bash FAQ.

  • cmd && trueCmd || falseCmd: if cmd executes successfully, run trueCmd, else run falseCmd. This is a pithy version of
    if cmd; then trueCmd; else falseCmd; fi
  • while cmd; do stuff; done: execute stuff while cmd executes successfully. Use
    while true; do stuff; done
    for an infinite loop.
  • for W in words; do stuff; done: sets the variable $W to each word in words, then executes stuff. For instance, to run foo on every text file in a directory tree, use
    for W in $(find . -name '*.txt'); do foo "$W"; done
    Note that words are split automatically based on whitespace. This means that filenames with spaces will be split into multiple words (I know find has the -exec option, but it can be cumbersome, and this is just an example). To avoid splitting on whitespace, see the next tip.
    Edit: Originally, my example used /bin/ls *.txt rather than find. However, as HorsePunchKid points out,
    for W in *.txt; do foo "$W"; done
    works on filenames with whitespace (and is also cleaner). This is an excellent point, but the only expansion done after word splitting is pathname expansion, so it applies only to file globs. If you're processing the output of a command, or the contents of an environment variable, then you'll still have a word splitting problem.
  • while read L; do stuff; done: sets the variable $L to each line in stdin, then executes stuff. Use this to handle input with spaces. For example, to run foo on every text file in a directory tree, including those with spaces, use
    find . -name '*.txt' | while read L; do foo "$L"; done

The last tip has a caveat: a piped command executes in a subshell with its own scope. Thus, if I use cmd | while read L; do stuff; done, variables set in stuff are not available outside of the loop. For example, if I want to run foo on every text file, then print how many times foo succeeded, I could try this:

I=0
find . -name '*.txt' | while read L; do foo "$L" && let I++; done
echo $I

However, this prints 0. The reason is because $I outside the pipe is a different variable than $I inside the pipe. To fix this, avoid a pipe using a trick from my earlier post:

I=0
while read L; do foo "$L" && let I++; done < <(find . -name '*.txt')
echo $I

6 comments:

HorsePunchKid said...

I could be wrong about this, but when you're worried about spaces in filenames, won't this do the trick?

for file in *.txt; do stuff; done

A quick test suggests this works fine, though lord knows there could be edge cases that don't. It's not very flexible, but it's a common use case.

Pedro DeRose said...

Ah, no, you're absolutely right. I think word splitting happens after the * expansion, so it works out for file names. However, the while read trick is still handy for input with spaces that aren't caught by file globs.

Excellent point; I'll make an edit to the post to reflect this.

squith said...

What a bash script? This is a list of custom commands to a text file in one time. Bash script is like a small program that can do just about any series of tasks. Very powerful stuff when you get a few commands under your belt.

Dubturbo

jenil said...

Bash has its own versions of most of the standard structures. If you are familiar with some of languages like python, perl and C then it's quite east to cope with this language too.

Motorcycle Tail Light

eashina said...

As I am starting to learn bash programming, this will be really helpful to me to get much into bash and it's flow control.

flash menu

Steven said...

What a bash script?

internet product review