Electrical-Forenics Home ray@RayFranco.com                       601.529.7473
   © Dr. Ray Franco, PhD, PE  -  208 Fairways Dr., Vicksburg, MS 39183

Linux - Scripts

A script is a list of shells commands to be executed.

By convention, script files should have the ".sh" extension. This also helps editors with syntax highlighting to determine the type of file.

A script should begin with the shell that you want to use:
 
#!/bin/bash
 
Ubuntu now uses dash as its default shell and MAC OS uses zsh. If do not include this as the first line of your script, you may have to run it as:
bash script_file.sh

Anything after the hash mark, "#", is a comments.

To run a scrip file, you have to make it executable:
 
chmode u+x script_file.sh
 
If you have sensitive information in a script file such as a password, do not give read permissions to other users.

To run a scipt file that is in the current directory:
 
./script_file.sh
 
The "./" is need for Linux distros that do not include the current path, "." , in the PATH variable.

Script Variable

bash does not have types. All variables in bash are strings.

Reference: How to Evaluate Arithmetic Expression in Bash

To declare a user script variable, you assign it a value:
var1="Hello World" # no spaces on either side of the equal sign (=).

To make a variable a global variable, so that it is seen by subshells, proceed the variable name by the word export:
export var1="Hello World".
Subshells can be spawned by one shell calling another shell.

You can assign the output of a command to a variable via command line subsitution:
var1=`date` or var1=$(date)
The later is prefered. It is easy to confuse the backtick (`) with a single quote (').

This example monitor the size and number of auth.log files:
 
today=$(date +%y%m%d)
ls /var/log/auth.log* -al > log.$today
  # The output is redirected to a log file with the extension of todays date.

Integer Arithmetic

Because all variables in bash are strings, you can not simple write arithmetic operations as you would do in any other language. By defualt, they would be interpreted as operations on strings, not numbers.

However, parameter expansion allows us to substitue an expresson with its value. We use it to get values from bash variables, perform arithmetic operations, and invoke commands. You access the value of a variable using the dollor sign ($).

Bash natively only does interger arithmetic. You have to put the expressions in square brackets and preceed the bracket with a "$".
 
Example:
var1=$[1+5]
var2=$[3+4]
var3=$[$var1+$Var2]
echo $var3
# output = 12
var4=$[$var1/$var2]
echo $var4
# output = 0

Bash can do floating point arithmetic via the Bash Calcuator, which is a precision calculator language. Other shells such as zsh will do foating point arithmetic. In zsh, you have to put a decimal point after floating point numbers, and each line in the shell must be terminated with a semicolnm ";".

Bash Floating Point Arithmetic

The Bash Calculator (bc) normally expects interactive input. bc variables can see bash variables, but bash cannot see bc variables.

Example 1:
var1=$(echo "scale=2; 3.44/5" | bc)
echo $var1
# output = 0.68
 
Example 2:
var1=100
var2=45
var3=$(echo "scale=2; $var1/$var2" | bc)
echo $var3
# output = 2.22
 
Example 3:
var1=11.11
var2=22.22
var3=33.33
var4=44
var5=$(bc << EOF
scale=4
a1=($var1 * $var2)
b1=($var3 * $var4)
a1 + b1
EOF
)
echo "The final answer is $var5"
# output = 1713.3842

Conditional Statements

Bash can test for a condition and branch according to the condition. The test condition can be exit codes, numerical values, strings, and file operators.

# Add the [expression] [[expression]] (expresion) ((expression))

Reference: https://phoenixnap.com/kb/bash-if-statement

Linux Exit Codes
0Command Completed Successfully
1General Unknow Error
2Missue of Shell Command
126Command Cannot Execute
127Command Not Found
128Invalid Exit Argument
128+xFatal Error with Linux Signal x i.e kill
130Command Terminated with Ctrl+C
255Exit Status Out of Range

echo $? will display the exit status of the last command.

Not all commands produce an Exit Code = 0, e.g.
grep jeff /etc/passwd   # Exit Code = 1 if no line with "jeff" found
ls dir_path/*        # Exit Code = 2 if no files are in the directory
You can add "exit number" to the last line of a script file to create your own exit code.

You do NOT put commands in brackets, [command]. If you do this, you will get an error and always an error code.

Comparing Numerical Values - New method
(( n1 == n2 )) if n1 equals n2
(( n1 != n2 )) if n1 Not equal to n2
(( n1 > n2 )) if n1 greater than n2
(( n1 => n2 )) if n1 is equal to or greater than n2
(( n1 < n2 )) if n1 less than n2
(( n1 =< n2 )) if n1 is equal to or less than n2

The new method for arithmetic conditonal statements requires the expression to be surrounded by double parenthese and a space after the first set and before the ending set. You do this instead of the brackets. This was not in the orignal bash, but it is now in most distros. This follow "C".

 

Comparing Numerical Values - old method
[ n1 -eq n2 ]if n1 equals n2
[ n1 -ne n2 ]if n1 Not equal to n2
[ n1 -gt n2 ]if n1 greater than n2
[ n1 -ge n2 ] if n1 greater than or equals n2
[ n1 -lt n2 ]if n1 less than n2
[ n1 -le n2 ] if n1 less than or equals n2

 

Comparing Strings
[ str1 = str2 ]if str1 Same as str2
[ str1 != str2 ]if str1 is Not the Same as str2
[ str1 > str2 ]if str1 greater than str2
[ str1 < str2 ] if str1 less than str2
[ -z str1 ]if str1 is zero lenght
[ -n str1 ] if str1 not zero length

 

File Test Operators
-f file_nameif file exist
-r file_nameif file exist and Readable
-w file_nameif file exist and Writable
-x file_nameif file exist and Executable
-O file_nameif file exist and Owned by Current User
-G file_nameif file exist and Group is the same as Current User
-s file_nameif file exist and not empty
-d fileif file exist and a Directory
-e fileif file exist and a File or Directory
file1 nt file2if file1 is newer than file2
file1 ot file2if file1 is older than file2

 

Bitwise Operators
n1 & n2 AND Operator
n1 | n2 OR Operator
~ n1Complement
<< n1 Left Shift
>> n1 Right Shift

 

Mathematical Operators
++val Pre-increment
--val Pre-decrement
val++ Post-increment
val-- Post-decrement
n1 ** power Exponential

Syntax: If the test condition produces a true or false result (binary), the test condition is placed inside of brackets, and bash requires a space after the leading bracket and before the closing bracket . To the best of my knowledge, the only test condition that does not produce a binary result is the Exit Code.

if then else

if [ condition ]
then
 commands
fi       # if spelled backwards

if [ condition ]
then
 commands
else
 commands
fi

if [ condition ]
then
 commands
elif    # short for else if
then
 commands
else
 commands
fi     # note, only one fi

Compound Conditions
if [ condition_1 ] && [ condition_2 ]   # Logical AND
if [ condition_1 ] || [ condition_2 ]   # Logical OR

Case Statement Syntax
case $varible in
pattern_1)
  commands;;
pattern_2 | pattern 3)   # matches pattern_2 or pattern_3
  commands;;
*)
  commands;;
esac            # case spelled backwards

For Loop Syntax
for varible in [list]
do
  commands
done

For Loop C-Style Syntax
for (initaization; test; step)
do
  commands
done

While Loop Syntax
while [ test ]
do
  commands
done

Until Loop Syntax
utill [ test ]
do
  commands
done

break # stop Loop
continue # continue loop from begining.

Exit Code Example
music_dir.sh
#!/bin/bash
# This script program tests the music directory to see if it is empty.
# The Exit Code is being tested.
# This is not a binary test that returns true or false;
# therefore, it is not enclosed in brackets.
# The * must be included in the ls command for this script to work.
# In the positional parameters section, this script will be expanded to test any directory.  
if ls -A ~/Music/* &> /dev/null  # redirect stdout and stderr
then
 echo "The Music direction is NOT empty"
else
 echo "The Music director IS empty"
fi

References:

  1. Get First Character in String in Bash

 

Input Positional Parameters

Script files can take parameters (arguments) just like Linux and Bash commands.

$0the name of the script
$1..$9variable $1 though $9
${10}variable $10 and above
$#number of parameters passed
$*all parameters as a string
"$@"parameters placed in an array
${!#}the last parameters passed
shiftshift parameters left by one and decrement $#
shift 3shift parameters left by 3 and adjust $#

Example 1:
./mycopy.sh soure_file desitnation_file
cat mycopy.sh
#!/bin/bash
# use interactive copy so as to not write over a file.
cp -i $1 $2

Example 2 - Shift Parameter Iteration
./shift_iteration.sh Paul Mary John
cat shift_iteration_shift.sh
#!/bin/bash
while (( $#>0 ))  # while number of parameters > 0
do
  echo $1
  shift      #shift paramters left by one and decrement #
done

Example 3 - Parameter Iteration
./script_iteration.sh Paul Mary John cat script_iteration.sh
#!/bin/bash
for x in "$@"
do
  echo $x
done
 
You must have double quotes around "$@" for it to handle parameters with spaces correctly ie "pass word". Using shift and iterating over the parameters does not require double quotes even when the parameters contain spaces.

Misinformation & Bad Programing Practices
The Internet is full of misinformation and bad programing practices on how to use positional parameters. It is common to see examples where they are using shift and testing the length of the parameter. They are testing an unset parameter! Which can result in an infinite loop. To make this work, they place double quotes around the parameter, which insures it is a string. For example:
 
while [-n "$1"]
 
Just google "double quotes around positional parameters". It's a mess that should be avoided. In the end, they end up putting double quotes around everything.
 
The following is from the "Real-world example" at: https:/www.computerhope.com/unix/bash/shift.htm
 
while (( "$#" )); do   # this should be: while (( $# )); do
 
and from the same example:
 
if [ "$#" == "0"]; then;   # this should be: if (( $# == 0 )); then
 
The following is from: http://www.shellscript.sh/variable2.html":
 
while [ "$#" -gt "0" ];   # -gt should be used with intergers not strings
 
And there are coutless code examples that use:
 
echo "$1"   # when $1 is already a string.

Options

Script can also have options that determine how the script operates e.g ls -l says to list the files in the long format. By convention, opitions are preceded by a dish, - ,.

options_example.sh
#!/bin/bash
if [ -z "$1" ]
then # no options specified
 echo "This commands requires a -a or -b option"
else
  for x in "$@"
  do
  case $x in
   -a) echo "-a option" ;;
   -b) echo "-b option" ;;
   *) echo "invalid option - valid options are -a and -b" ;;
  esac
 done
fi

Also by bash convention, options should come before parameters and a -- seperates options from parameters i.e.
script_file options -- parameters

options_with_parameters.sh
#!/bin/bash
until [ "$1" = -- ]
do
 case $1 in
   -x) echo "option -x" ;;
   -y) echo "option -y" ;;
   -z) echo "option -z" ;;
   *) echo "invalid option" ;;
 esac
 shift
done
 
shift # get past --
x=1
for y in "$@"
do
  echo "parameter $x is $y"
  x=$[ $x+1 ]
done

To process catenated options e.g -alh, you have to use getopt and/or getopts.
Add to this -

--
bash command: set -- . If no arguments follow this option then the positional paramaters are unset. Otherwise the positional paramaters are set to the args, even if some of them begin with a -.

-aAll
-cCount
-dDiretory
-ffile
-hhelp
-oOutput
-rRecursive
-sSilent
-vVerbose
-yYes

Interactive - Read Input

read [option]... [varible]...
if no variables default is $REPLY
-pprompt for input
-ssilence - do not echo typed chars
-n 1get input after 1 chars
-t 5.2timeout after 5.2 seconds

Example:
#!/bin/bash
echo -n "Enter your first name >"
# -n no new line
read first_name
read -p "Enter your last name >" last_name
# read with promt for input
echo "Hello $first_name $last_name"
read -p "Enter your first and last names >" first_name last_name
# read more than one variable echo "Hello Mr. $first_name $last_name"
read -p "Enter your first name >"
echo "Hello $REPLY"

Redirection

To redirected the output inside a script, you can use exec.
Example:
exec 1> output_file_name
exec 2> err_file_name
echo "Hello There"
silly -o
# There is no silly command - error message

 

Running Scripts in the Background

To run a scrip in the background use the "&":
> long_script.sh &

However, if the script is long and you close the terminal the script stops runing. You can avoid this by using the nohup (no hang up) command:
:gt; nohup long_script.sh &

Script Priority

To assign a priority to a job use the nice command:
nice -n number my_script.sh
The default priority is 0. The highest priority is -20. The lowest priority is 19. To assign a priority below 0, you have to sudo nice.

To change the priority of a running job, use renice with the process id (PID):
pidof my_script.sh   # get PID
renice -n -5 pid_number

Job Scheduling via cron

cron is named after the Greek word "Chronos" that is used for time.

To schedule a job to run at a certain time or repetively, you enter the job in the crontab (short for cron table), or you can place a script file that is to run repetively in one of the directories /etc/cron.{hourly, daily, weekly, monthly}.

A Debian system has the following cron tab files and directories:
 
/etc/crontab, usually only hold entries to run the jobs from /etc/cron.{daily, weekly, monthly}
/etc/cron.d, a directroy with files added by Packages. These text files have the same format as the crontab file.
/var/spool/cron/crontabs/, holds one crontab file per user. The files names in this directory are just the usernames.
/etc/cron.{hourly,daily,weekly,monthly} these are directories that hold jobs to be run hourly, daily, weekly and monthly. These are not cron tables.
 
Optionally, there can be files /etc/cron.allow and/or /etc/cron.deny, which list users. By default, Debian does not create these.

Do not edit the crontab files directly. Instead you use the crontab command. The first time you use the crontab command, it will prompt you to choose an editor e.g. vi or nano.

crontab [ -u user] file
crontab [ -u user] [ -l | -e | -r | -i ]
 -llist
 -eedit
 -rremove all user cron jobs
 -iinteractive

Each entry in a cromtab file must end with the new line, \n , character - including the last line. The instructions for making a new entry in a crontab file are included in the comments of the file.

To access the root user's crontable: sudo crontab -e.

Some examples of setting the time a cron job runs are:

MinuteHourDay(Month)MonthDay(week) 
193**7@ 3:19 every Sunday
*/10****every 10 minutes
9,39****@ 9 and 39 minutes - every 30 minutes
@reboot    @ reboot

A user with administrator privious can modify another user's crontab file by:

sudo crontab -u another_users_name -e

Again, you do not edit crontab files directly. Debian makes this difficult to do by limiting access to the /var/spool/cron/crontabs directory. You can not sudo into this directory. However, you can access the directory by becoming the super user, root, i.e. sudo su.

anacron is the daemon that completes cron for computer that are not on at all times e.g. laptops. The format of /etc/anacrontab is not the same as crontab. anacron is part of the Debian Bullseye Destro. However, it is not installed by default in the Raspberry Pi OS even though there are references to it in /etc/crontab file.

Debian is in the process of replacing cron with cronie, which will combine cron and anacron. cronie was developed by Red Hat.

The "at" command can be used to run a script at a certain date and time. However, it can not be used for repetive jobs, and it is not installed by default in Debian or Raspberry Pi OS.

systemd timers are a totally different method for time-based process scheduling.

Security:
1. /etc/crontab: This is the system cron job(s), and this file can only be modified by the root user.
2. /etc/cron.d: These are installed by programs and packaages. All the cron jobs that I have seen in this directory are owned by root. However, I need to lookthis futher. I do not like the idea that you can install a program such as pihole, and it can run cron jobs as the root.
3. /etc/cron.{hourly, daily, weekly, monthly}: You have to have elevated privillages to place a job in these directories.
4. /var/spool/cron/contabs: The user is not specified in these cron jobs; hence, a user can not elevate his privillages.

To test cron, I created a one line bash shell script:

cron_date.sh
#!/bin/bash
date >> cron_date.txt

I then entering the follow line into my crontab file:

*/5 * * * * /home/pi/cron_date.sh # run every 5 mintues