Bash

Bash scripting

Caches

All UNIX shells cache the command paths based on the contents of PATH enviromental variable. This can cause a problem if a cached path no longer exists. To clear the cached command path, run:

	PATH=$PATH

For Bash only, you can run:

	hash -r

Another cache is used for the locate command which can be updated by running:

	updatedb

Functions

Either of two forms:

  1. function functname {
    ...
    }
  2. functname() {
    ...
    }

Functions must appear in the script before they can be used
Local variables defined with local keyword

	local  [options] [var1[=value]] [var2[=value]] ...

Positional parameters are passed the same way as in scripts. i.e. $1, $2, etc

Variable Subsitution

Variable substitution is characterized by using curly braces of the form ${var...}.
If var is followed by one or two forward slashes ${var/...} then the substitution is of the form /pattern/replacement or /pattern drop if no /replacement given.
If var is followed by one or two octothorps ${var#...} then the substitution is of the the form #pattern drop where pattern occurs at the beginning.
If var is followed by one or two percent signs ${var%...} then the substitution is of the form %pattern drop where pattern occurs at the end.
If var is followed by a slash and then an octothorpe ${var/#...} then the substitution is of the form /#pattern drop where a greedy match is formed at the beginning.
If var is followed by a slash and then a percent sign ${var/%...} then the substitution is of the form /%pattern drop where a greedy match is formed at the end.
If the pattern is followed by /repl then a replacement is made where the matched pattern was dropped

Here are the constructs:

Note that while ${var%%pat} and ${var/%pat} produce identical results
and that ${var##pat} and ${var/#pat} produce identical results there is no equivalent ${var%%pat/repl} or ${var##pat/repl} so ${var/%pat/repl} and ${var/#pat/repl} must be used to replace the longest match and ${var/pat/repl} must be used to replace the shortest match from the beginning and there is no way to replace the shortest match from the end using the * wildcard. For example if a=ABCXYZABC and you want to replace the final ABC with 123 then

${a/%A?C/123}

works, producing ABCXYZ123 but

${a/%A*C/123}

produces 123 because of the greedy *. The obvious workaround is to simply chop the final ABC with ${a%A*} and then append the 123

${a%A*}123
	${#var}		==>	Length of var
	${#*}, ${#@}	==>	Number of command line parameters
	a=ABCXYZABC
	echo ${a:3}	==>	XYZABC		# substring (zero based)
	echo ${a:3:2}	==>	XY		# position:# of characters
	echo ${a/C}	==>	ABXYZABC	# remove first C
	echo ${a//C}	==>	ABXYZAB		# remove all C
	echo ${a/#A}	==>	BCXYZABC	# remove A if occurs at beginning
	echo ${a/%C}	==>	ABCXYZAB	# remove C if occurs at end
	${var/substr/repl}			# replace first match
	echo ${a/C/w}	==>	ABwXYZABC	# substitude w for first C

	${var//substr/repl}			# replace all matches
	echo ${a//C/w}	==>	ABwXYZABw	# global substitute

	${var/#substr/repl}	# replace if matches at beginning (non-greedy)
	echo ${a
	${var/##substr/repl}	# replace if matches at beginning (greedy)
	${var/%substr/repl}	# replace if matches at end (greedy)
	${var/%%substr/repl}	# replace if matches at end (greedy)

	NOTE: Use the $'\xNN' syntax for non-printing characters.

The colon is optional. If it's included, var must be nonnull, as well as set.

	$ unset b
	echo ${b:-word}	==>	word
	echo ${b:=word}	==>	word (also b now = word)(not for cl parms)
	echo ${b:?}	==>	b: parameter null or not set (script exits)
	echo ${b:?word}	==>	b: word (script exits)
	echo ${b:+word}	==>	b: null (otherwise would be word)

Pattern Magic

	c=ABCXYZABC
	echo ${c#*B}	==>	CXYZABC		# beginning match
	echo ${c##*B}	==>	C		# greedy beginning
	echo ${c%B*}	==>	ABCXYZA		# ending match
	echo ${c%%B*}	==>	A		# greedy ending

Bash and MySQL

This article shows seven methods of writing MySQL database shell scripts.

  1. Sending a single command to the MySQL server
  2. Sending multiple commands to the MySQL server
  3. Redirecting MySQL output to a variable
  4. Redirecting MySQL output to a file
  5. Redirecting MySQL output to a file in HTML format
  6. Redirecting MySQL output to a file in XML format
  7. Using a CGI script to serve MySQL data

Start with an appropriate ~/.my.cnf
(see the mysql man page for details)

#!/bin/sh -
# Send a single command to the MySQL server
MYSQL='/usr/bin/mysql'
$MYSQL -sse 'SELECT Artist, Year FROM music'

The -e (execute) switch is required when passing a SQL command on the command line. The -s (silent) switch supresses the ASCII symbol box. Use it twice to supress column headings also. Results are displayed in tab delimited format.

#!/bin/sh -
# Send multiple commands to the MySQL server
MYSQL='/usr/bin/mysql'
$MYSQL <<EOF
SHOW TABLES;
SELECT Artist, Year FROM music;
EOF

#!/bin/sh -
# Redirect MySQL output to a variable
MYSQL='/usr/bin/mysql'
output=$($MYSQL -e 'SELECT Artist, Year FROM music')
for line in "$output"; do
  echo "$line"
done

If you want "boxed" output include the --table (-t) option
To supress column headings use the --skip-column-names (-N) option

#!/bin/sh -
# Output MySQL result to a file
MYSQL='/usr/bin/mysql'
$MYSQL -e 'SELECT Artist, Year FROM music' > music.txt

#!/bin/sh -
# Output MySQL result to a file in HTML format
MYSQL='/usr/bin/mysql'
$MYSQL -He 'SELECT Artist, Year FROM music' > music.html

#!/bin/sh -
# Output MySQL result to a file in XML format
MYSQL='/usr/bin/mysql'
$MYSQL -Xe 'SELECT Artist, Year FROM music' > music.xml

#!/bin/sh -
# Using a CGI script to serve MySQL data
mysql=/usr/bin/mysql
user=myself
host=localhost
pass=secret
db=my_database
table=my_table
echo Content-type: text/html
echo ""
echo "<html><head><title>my_title</title></head>"
echo "<body><p align=center>"
$mysql -u$user -h$host --password=$pass -He "SELECT * FROM $table" $db
echo ""
echo "</p></body></html> "

You can also execute SQL statements in a script file (batch file) like this:
shell> mysql db_name < script.sql > output.tab

MySQL understands the use of double single quote marks in addition to understanding the use of a back slash to escape a single quote. I use the following code sequence to deal with double quotes appearing in the data, and then surround the field variable with escaped double quotes. Song titles, artist, and artistsort flac metadata are thus handled like this.
[ "$t"=~'"' ] && t=${t//\"/\\\"}
[ "$a"=~'"' ] && a=${a//\"/\\\"}
[ "$as"=~'"' ] && as=${as//\"/\\\"}
mysqsl -e "UPDATE music SET Artist=\"$a\", Title=\"$t\" ... "

By default, when running MySQL queries in batch mode (i.e. with the --batch (-B) option, the output is tab delimited with each row on a separate line, and includes an initial row of column headings. To supress these headings use the --skip-column-names (-N) option. To produce the same kind of "boxed" output you get when running interactively, use the --table (-t) option.

If you have problems due to insufficient memory for large result sets, use the --quick option. This forces mysql to retrieve results from the server a row at a time rather than retrieving the entire result set and buffering it in memory before displaying it.


Bash and Math

expr 8 \* 9

echo $[8 * 9]

echo " scale=4; 3.44 / 5 " | bc

echo $(bc << EOF
scale = 4
a1 = (3 + 4)
b1 = (5 + 6)
a1 + b1
EOF
)

let "t2 = ((a = 9, 15 / 3))" # Set "a =9" and "t2 = 15 / 3"

Number Base Conversions

# Convert a base 10 number to base 16
printf %x 65535 --> ffff

Convert a base 16 number to base 10
echo $(( 16#ffff )) --> 65535


Arrays in Bash

To put a list of items in an array
declare -a arr
arr=($(ls -ld))

To get the number of elements
${#arr[*]}
or
${#arr[@]}


The Test Constructs

if test condition; then

if [ condition ]; then
if [ ! condition ]; then

if [ condition1 ] && [ condition2 ]; then

if [ condition1 ] && [ \(condition2 \) -o \(condition3\) ]; then
  (parenthesis must be escaped)(-a and -o can only be used inside a test)

if [[ condition1 ]] || [[ condition2 ]]; then
  (word splitting and pathname expansion not done)

if (( $# < 3 )); then
  (use with arithmetic tests only)

NOTE: Truth values returned by test (and exit) are 0 for true, 1 for false.


Some Conditionals

Bash's regular expression comparison operator takes a string on the left and an extended regular expression on the right. It returns 0 (success) if the regular expression matches the string, otherwise it returns 1 (failure).

In addition to doing simple matching, bash regular expressions support sub-patterns surrounded by parenthesis for capturing parts of the match. The matches are assigned to an array variable BASH_REMATCH. The entire match is assigned to BASH_REMATCH[0], the first sub-pattern is assigned to BASH_REMATCH[1], etc..

haystack =~ needle
-n string	string length > 0
-z string	string length = 0
-a file		file exists
-e file		file exists
-s file		file exists and is not empty
-N file		file modified since last read
Integer conditionals: -lt -le -eq -ge -gt -ne
$((...)) is the preferred integer conditional test
[ $(((3 > 2) && (4 <= 1))) = 1 ]

NOTE: Truth values returned by $((...)) are 1 for true, 0 for false.

So [ 2 -gt 1 ] && [ $(( 2 < 1 )) ] returns true !!


Bash Anoyances

The read command works one way with the default variable $REPLY and a different way with any other variable name. See for yourself:

while : ; do
  read -n 1
  case $REPLY in
    ' ') break;;
    *) continue;;
  esac
done

Now try with a different variable name:

while : ; do
  read -n 1 somevar
  case $somevar in
    ' ') break;;
    *) continue;;
  esac
done

Pressing the space bar to exit the loop only works in the first example


Send mail to the Webmaster

logo This site best viewed with a browser
Warning: This is a Debian centric site and MAY contain peanuts.
Many thanks to Debra Lynn and Ian Murdock for making Debian possible
First created Dec 14, 2008 ~ Last revised September 22, 2011

Valid XHTML 1.0 Strict Valid CSS!