A script is a list of shells commands to be executed.
By convention, script files should have the ".sh" extension. This also helps editors with syntax highlighting to determine the type of file.
A script should begin with the shell that you want to use:
#!/bin/bash
Ubuntu now uses dash as its default shell and MAC OS uses zsh. If do not include
this as the first line of your script, you may have to run it as:
bash script_file.sh
Anything after the hash mark, "#", is a comments.
To run a scrip file, you have to make it executable:
chmode u+x script_file.sh
If you have sensitive information in a script file such as a password, do not give read permissions to other users.
To run a scipt file that is in the current directory:
./script_file.sh
The "./" is need for Linux distros that do not include the current path, "." , in the PATH variable.
bash does not have types. All variables in bash are strings.
Reference: How to Evaluate Arithmetic Expression in Bash
To declare a user script variable, you assign it a value:
var1="Hello World" # no spaces on either side of the equal sign (=).
To make a variable a global variable, so that it is seen by subshells, proceed the variable name by the word export:
export var1="Hello World".
Subshells can be spawned by one shell calling another shell.
You can assign the output of a command to a variable via command line subsitution:
var1=`date` or var1=$(date)
The later is prefered. It is easy to confuse the backtick (`) with a single quote (').
This example monitor the size and number of auth.log files:
today=$(date +%y%m%d)
ls /var/log/auth.log* -al > log.$today
# The output is redirected to a log file with the extension of todays date.
Because all variables in bash are strings, you can not simple write arithmetic operations as you would do in any other language. By defualt, they would be interpreted as operations on strings, not numbers.
However, parameter expansion allows us to substitue an expresson with its value. We use it to get values from bash variables, perform arithmetic operations, and invoke commands. You access the value of a variable using the dollor sign ($).
Bash natively only does interger arithmetic. You have to put the expressions in square brackets and preceed the bracket with a "$".
Example:
var1=$[1+5]
var2=$[3+4]
var3=$[$var1+$Var2]
echo $var3
# output = 12
var4=$[$var1/$var2]
echo $var4
# output = 0
Bash can do floating point arithmetic via the Bash Calcuator, which is a precision calculator language. Other shells such as zsh will do foating point arithmetic. In zsh, you have to put a decimal point after floating point numbers, and each line in the shell must be terminated with a semicolnm ";".
The Bash Calculator (bc) normally expects interactive input. bc variables can see bash variables, but bash cannot see bc variables.
Example 1:
var1=$(echo "scale=2; 3.44/5" | bc)
echo $var1
# output = 0.68
Example 2:
var1=100
var2=45
var3=$(echo "scale=2; $var1/$var2" | bc)
echo $var3
# output = 2.22
Example 3:
var1=11.11
var2=22.22
var3=33.33
var4=44
var5=$(bc << EOF
scale=4
a1=($var1 * $var2)
b1=($var3 * $var4)
a1 + b1
EOF
)
echo "The final answer is $var5"
# output = 1713.3842
Bash can test for a condition and branch according to the condition. The test condition can be exit codes, numerical values, strings, and file operators.
# Add the [expression] [[expression]] (expresion) ((expression))Reference: https://phoenixnap.com/kb/bash-if-statement
Linux Exit Codes | |
0 | Command Completed Successfully |
1 | General Unknow Error |
2 | Missue of Shell Command |
126 | Command Cannot Execute |
127 | Command Not Found |
128 | Invalid Exit Argument |
128+x | Fatal Error with Linux Signal x i.e kill |
130 | Command Terminated with Ctrl+C |
255 | Exit Status Out of Range |
echo $? will display the exit status of the last command.
Not all commands produce an Exit Code = 0, e.g.
grep jeff /etc/passwd # Exit Code = 1 if no line with "jeff" found
ls dir_path/* # Exit Code = 2 if no files are in the directory
You can add "exit number" to the last line of a script file to create your own exit code.
You do NOT put commands in brackets, [command]. If you do this, you will get an error and always an error code.
Comparing Numerical Values - New method | |
(( n1 == n2 )) | if n1 equals n2 |
(( n1 != n2 )) | if n1 Not equal to n2 |
(( n1 > n2 )) | if n1 greater than n2 |
(( n1 => n2 )) | if n1 is equal to or greater than n2 |
(( n1 < n2 )) | if n1 less than n2 |
(( n1 =< n2 )) | if n1 is equal to or less than n2 |
The new method for arithmetic conditonal statements requires the expression to be surrounded by double parenthese and a space after the first set and before the ending set. You do this instead of the brackets. This was not in the orignal bash, but it is now in most distros. This follow "C".
Comparing Numerical Values - old method | |
[ n1 -eq n2 ] | if n1 equals n2 |
[ n1 -ne n2 ] | if n1 Not equal to n2 |
[ n1 -gt n2 ] | if n1 greater than n2 |
[ n1 -ge n2 ] | if n1 greater than or equals n2 |
[ n1 -lt n2 ] | if n1 less than n2 |
[ n1 -le n2 ] | if n1 less than or equals n2 |
Comparing Strings | |
[ str1 = str2 ] | if str1 Same as str2 |
[ str1 != str2 ] | if str1 is Not the Same as str2 |
[ str1 > str2 ] | if str1 greater than str2 |
[ str1 < str2 ] | if str1 less than str2 |
[ -z str1 ] | if str1 is zero lenght |
[ -n str1 ] | if str1 not zero length |
File Test Operators | |
-f file_name | if file exist |
-r file_name | if file exist and Readable |
-w file_name | if file exist and Writable |
-x file_name | if file exist and Executable |
-O file_name | if file exist and Owned by Current User |
-G file_name | if file exist and Group is the same as Current User |
-s file_name | if file exist and not empty |
-d file | if file exist and a Directory |
-e file | if file exist and a File or Directory |
file1 nt file2 | if file1 is newer than file2 |
file1 ot file2 | if file1 is older than file2 |
n1 & n2 | AND Operator |
n1 | n2 | OR Operator |
~ n1 | Complement |
<< n1 | Left Shift |
>> n1 | Right Shift |
++val | Pre-increment |
--val | Pre-decrement |
val++ | Post-increment |
val-- | Post-decrement |
n1 ** power | Exponential |
Syntax: If the test condition produces a true or false result (binary), the test condition is placed inside of brackets, and bash requires a space after the leading bracket and before the closing bracket . To the best of my knowledge, the only test condition that does not produce a binary result is the Exit Code.
if [ condition ]
then
commands
fi # if spelled backwards
if [ condition ]
then
commands
else
commands
fi
if [ condition ]
then
commands
elif # short for else if
then
commands
else
commands
fi # note, only one fi
Compound Conditions
if [ condition_1 ] && [ condition_2 ] # Logical AND
if [ condition_1 ] || [ condition_2 ] # Logical OR
Case Statement Syntax
case $varible in
pattern_1)
commands;;
pattern_2 | pattern 3) # matches pattern_2 or pattern_3
commands;;
*)
commands;;
esac # case spelled backwards
For Loop Syntax
for varible in [list]
do
commands
done
For Loop C-Style Syntax
for (initaization; test; step)
do
commands
done
While Loop Syntax
while [ test ]
do
commands
done
Until Loop Syntax
utill [ test ]
do
commands
done
break # stop Loop
continue # continue loop from begining.
Exit Code Example
music_dir.sh
#!/bin/bash
# This script program tests the music directory to see if it is empty.
# The Exit Code is being tested.
# This is not a binary test that returns true or false;
# therefore, it is not enclosed in brackets.
# The * must be included in the ls command for this script to work.
# In the positional parameters section, this script will be expanded to test any directory.
if ls -A ~/Music/* &> /dev/null # redirect stdout and stderr
then
echo "The Music direction is NOT empty"
else
echo "The Music director IS empty"
fi
References:
Script files can take parameters (arguments) just like Linux and Bash commands.
$0 | the name of the script |
$1..$9 | variable $1 though $9 |
${10} | variable $10 and above |
$# | number of parameters passed |
$* | all parameters as a string |
"$@" | parameters placed in an array |
${!#} | the last parameters passed |
shift | shift parameters left by one and decrement $# |
shift 3 | shift parameters left by 3 and adjust $# |
Example 1:
./mycopy.sh soure_file desitnation_file
cat mycopy.sh
#!/bin/bash
# use interactive copy so as to not write over a file.
cp -i $1 $2
Example 2 - Shift Parameter Iteration
./shift_iteration.sh Paul Mary John
cat shift_iteration_shift.sh
#!/bin/bash
while (( $#>0 )) # while number of parameters > 0
do
echo $1
shift #shift paramters left by one and decrement #
done
Example 3 - Parameter Iteration
./script_iteration.sh Paul Mary John
cat script_iteration.sh
#!/bin/bash
for x in "$@"
do
echo $x
done
You must have double quotes around "$@" for it to handle parameters with spaces correctly ie "pass word". Using shift
and iterating over the parameters does not require double quotes even when the parameters contain spaces.
Misinformation & Bad Programing Practices
The Internet is full of misinformation and bad programing practices on how to use positional parameters.
It is common to see examples where they are using shift and testing the length of the parameter. They
are testing an unset parameter! Which can result in an infinite loop. To make this work, they place
double quotes around the parameter, which insures it is a string. For example:
while [-n "$1"]
Just google "double quotes around positional parameters". It's a mess that should be avoided. In the end,
they end up putting double quotes around everything.
The following is from the "Real-world example" at:
https:/www.computerhope.com/unix/bash/shift.htm
while (( "$#" )); do # this should be: while (( $# )); do
and from the same example:
if [ "$#" == "0"]; then; # this should be: if (( $# == 0 )); then
The following is from:
http://www.shellscript.sh/variable2.html":
while [ "$#" -gt "0" ]; # -gt should be used with intergers not strings
And there are coutless code examples that use:
echo "$1" # when $1 is already a string.
Script can also have options that determine how the script operates e.g ls -l says to list the files in the long format. By convention, opitions are preceded by a dish, - ,.
options_example.sh
#!/bin/bash
if [ -z "$1" ]
then # no options specified
echo "This commands requires a -a or -b option"
else
for x in "$@"
do
case $x in
-a) echo "-a option" ;;
-b) echo "-b option" ;;
*) echo "invalid option - valid options are -a and -b" ;;
esac
done
fi
Also by bash convention, options should come before parameters and a --
seperates options from parameters i.e.
script_file options -- parameters
options_with_parameters.sh
#!/bin/bash
until [ "$1" = -- ]
do
case $1 in
-x) echo "option -x" ;;
-y) echo "option -y" ;;
-z) echo "option -z" ;;
*) echo "invalid option" ;;
esac
shift
done
shift # get past --
x=1
for y in "$@"
do
echo "parameter $x is $y"
x=$[ $x+1 ]
done
To process catenated options e.g -alh, you have to use getopt and/or getopts.
Add to this -
--
bash command: set -- . If no arguments follow this option then the positional paramaters are unset.
Otherwise the positional paramaters are set to the args, even if some of them begin with a -.
-a | All |
-c | Count |
-d | Diretory |
-f | file |
-h | help |
-o | Output |
-r | Recursive |
-s | Silent |
-v | Verbose |
-y | Yes |
read [option]... [varible]... | |
if no variables default is $REPLY | |
-p | prompt for input |
-s | silence - do not echo typed chars |
-n 1 | get input after 1 chars |
-t 5.2 | timeout after 5.2 seconds |
Example:
#!/bin/bash
echo -n "Enter your first name >"
# -n no new line
read first_name
read -p "Enter your last name >" last_name
# read with promt for input
echo "Hello $first_name $last_name"
read -p "Enter your first and last names >" first_name last_name
# read more than one variable
echo "Hello Mr. $first_name $last_name"
read -p "Enter your first name >"
echo "Hello $REPLY"
To redirected the output inside a script, you can use exec.
Example:
exec 1> output_file_name
exec 2> err_file_name
echo "Hello There"
silly -o
# There is no silly command - error message
To run a scrip in the background use the "&":
> long_script.sh &
However, if the script is long and you close the terminal the script stops runing.
You can avoid this by using the nohup (no hang up) command:
:gt; nohup long_script.sh &
To assign a priority to a job use the nice command:
nice -n number my_script.sh
The default priority is 0. The highest priority is -20. The lowest priority is 19.
To assign a priority below 0, you have to sudo nice.
To change the priority of a running job, use renice with the process id (PID):
pidof my_script.sh # get PID
renice -n -5 pid_number
cron is named after the Greek word "Chronos" that is used for time.
To schedule a job to run at a certain time or repetively, you enter the job in the crontab (short for cron table), or you can place a script file that is to run repetively in one of the directories /etc/cron.{hourly, daily, weekly, monthly}.
A Debian system has the following cron tab files and directories:
/etc/crontab, usually only hold entries to run the jobs from /etc/cron.{daily, weekly, monthly}
/etc/cron.d, a directroy with files added by Packages. These text files have the same format as the crontab file.
/var/spool/cron/crontabs/, holds one crontab file per user. The files names in this directory are just the usernames.
/etc/cron.{hourly,daily,weekly,monthly} these are directories that hold jobs to be run hourly, daily, weekly and monthly.
These are not cron tables.
Optionally, there can be files /etc/cron.allow and/or /etc/cron.deny, which list users. By default, Debian does
not create these.
Do not edit the crontab files directly. Instead you use the crontab command. The first time you use the crontab command, it will prompt you to choose an editor e.g. vi or nano.
crontab [ -u user] file | |
crontab [ -u user] [ -l | -e | -r | -i ] | |
-l | list |
-e | edit |
-r | remove all user cron jobs |
-i | interactive |
Each entry in a cromtab file must end with the new line, \n , character - including the last line. The instructions for making a new entry in a crontab file are included in the comments of the file.
To access the root user's crontable: sudo crontab -e.
Some examples of setting the time a cron job runs are:
Minute | Hour | Day(Month) | Month | Day(week) | |
19 | 3 | * | * | 7 | @ 3:19 every Sunday |
*/10 | * | * | * | * | every 10 minutes |
9,39 | * | * | * | * | @ 9 and 39 minutes - every 30 minutes |
@reboot | @ reboot |
A user with administrator privious can modify another user's crontab file by:
sudo crontab -u another_users_name -e
Again, you do not edit crontab files directly. Debian makes this difficult to do by limiting access to the /var/spool/cron/crontabs directory. You can not sudo into this directory. However, you can access the directory by becoming the super user, root, i.e. sudo su.
anacron is the daemon that completes cron for computer that are not on at all times e.g. laptops. The format of /etc/anacrontab is not the same as crontab. anacron is part of the Debian Bullseye Destro. However, it is not installed by default in the Raspberry Pi OS even though there are references to it in /etc/crontab file.
Debian is in the process of replacing cron with cronie, which will combine cron and anacron. cronie was developed by Red Hat.
The "at" command can be used to run a script at a certain date and time. However, it can not be used for repetive jobs, and it is not installed by default in Debian or Raspberry Pi OS.
systemd timers are a totally different method for time-based process scheduling.
Security:
1. /etc/crontab: This is the system cron job(s), and this file can only be modified by the root user.
2. /etc/cron.d: These are installed by programs and packaages. All the cron jobs that I have seen
in this directory are owned by root. However, I need to lookthis futher. I do not like the idea
that you can install a program such as pihole, and it can run cron jobs as the root.
3. /etc/cron.{hourly, daily, weekly, monthly}: You have to have elevated privillages to place a job in these directories.
4. /var/spool/cron/contabs: The user is not specified in these cron jobs; hence, a user can not elevate his privillages.
To test cron, I created a one line bash shell script:
cron_date.sh
#!/bin/bash
date >> cron_date.txt
I then entering the follow line into my crontab file:
*/5 * * * * /home/pi/cron_date.sh # run every 5 mintues