Published on

CITS4407 Open Source Tools and Scripting Final Exam CheetSheet

Download link: CITS4407.pdf (Chinese Version)

Files and Permissions

Permission Representation

  • rwx (read, write, execute)
  • User categories: ugo (user, group, others)
  • Directory: drwx (d for directory, `` for regular file)

Modifying Permissions

  • Octal: chmod <octal mode> <files>
  • Symbolic: chmod <symbolic mode> <files>
  • Adding/Removing Permissions:
    • chmod ugo+rwx myapp
    • chmod go-rw myapp

File Management Commands

Creating Directories

  • mkdir <Dir> -p to create multiple levels of directories

Moving Directories

  • cd <dir> (.. for parent directory)

Current Directory

  • pwd

Listing Files

  • ls <dir>
    • l for detailed information
    • a to include hidden files
    • R to list all files recursively
    • t to sort by modification time

Renaming/Moving Files

  • mv <file1> <file2> or mv <file> <directory>

Creating Links

  • ln <file1> <file2>

Viewing File Content

  • cat <file>

Deleting Files

  • rm <file> (r for recursive delete)

File Search

Basic Usage

  • find [<options>] <path> [<expression>]

Common Expressions

  • name <file pattern>: search by file name pattern
  • type <c>: search by type (d for directory, f for file, l for link)
  • size [+-]size[cwbkMG]: search by file size
  • newer <file>: search for files newer than the specified file

Operations

  • print: print file path
  • exec <command> \\;: execute command on each found file

Example

  • Find and move script files:
  find /lab/week6/mess -type f -name '*lab*' -exec chmod +x {} \\; -exec mv {} /lab/week6/safe \\;

Explanation

  1. find ~:
    • find is a command to search files and directories in a directory tree.
    • ~ represents the current user's home directory, so it searches within the home directory and its subdirectories.
  2. name '*.gz':
    • name is an option of the find command, specifying the pattern to search for.
    • .gz is a wildcard pattern, representing all files ending with .gz (usually gzip compressed files).
  3. exec a.sh '{}' \\;:
    • exec is an option of the find command to execute a specified command on each found file.
    • a.sh is the script to be executed. In this example, it is a script named a.sh.
    • {} is a placeholder for the current found file name. find replaces each found file name into this place.
    • \\; marks the end of the exec option. The backslash (\\) is used to escape the semicolon (;) to prevent it from being interpreted prematurely by the shell.

Summary

The code searches for all files ending with .gz in the current user's home directory and its subdirectories and executes the a.sh script on each found file, passing the file name as a parameter.

Copying Files

  • Move the listed file(s) into the directory:

    cp <file1> <file2>
cp <files> <directory>

  • Options:

    • r: recursive copy of directories and their contents
    • i: interactive mode, prompts for confirmation if the destination file exists

mv Command

mv is a basic command in Unix/Linux systems for moving or renaming files and directories.

Basic Syntax

mv [options] source target

Basic Usage

  1. Move a file - Move file1.txt to destination directory:

    mv file1.txt destination/

  2. Rename a file - Rename file1.txt to file2.txt:

    mv file1.txt file2.txt

  3. Move a directory - Move dir1 to destination directory:

    mv dir1/ destination/

  4. Rename a directory - Rename dir1 to dir2:

    mv dir1/ dir2/

Common Options

  1. i (interactive)

    • Prompts for confirmation before overwriting files:

      mv -i file1.txt destination/

  2. f (force)

    • Force overwrite of the target file without prompting for confirmation.

  3. u (update)

    • Only move or rename if the source file is newer than the destination file or if the destination file does not exist:

      mv -u file1.txt destination/

  4. v (verbose)

    • Display detailed information for each file or directory moved:

      mv -v file1.txt destination/

Examples

  1. Move multiple files to a directory:

    mv file1.txt file2.txt destination/

  2. Move and rename a file:

    mv file1.txt destination/file2.txt

  3. Move multiple files and prompt for confirmation if overwriting:

    mv -i file1.txt file2.txt destination/

Notes

  • If the target file or directory exists, mv will overwrite them by default unless the i option is used.
  • The mv command does not change file permissions and attributes.
  • Ensure write permissions for both the source file and the target directory when using the mv command.

File Statistics

Counting Lines, Words, and Characters

  • wc [options] [files]
    • l: display the number of lines
    • w: display the number of words
    • m: display the number of characters
    • c: display the number of bytes

Compression and Decompression

Compress

  • gzip <files>

Decompress

  • gunzip <files>

Processes

Process Management

Viewing Processes

  • ps <options>
    • l: long listing
    • f: full-format listing
  • top: view all processes and CPU usage

Terminating Processes

  • kill -9 <pid>: force terminate
  • kill -15 <pid>: normal terminate

Redirection

  • Standard Input: 0<file
  • Standard Output: >file
  • Standard Error: 2>file
  • Combined Redirection: >log.txt 2>&1

Test Commands

File Tests

  • d filename: checks if a directory exists
  • f filename: checks if a file exists
  • s filename: checks if a file is not empty

String Tests

  • n string: checks if the string is not empty
  • z string: checks if the string is empty

File Comparison

diff Command

  • Used to compare the differences between two files:
    diff file1 file2

Variables

Definition and Reference

  • Define variable: <variable>=<value> (no spaces around =)
  • Reference variable: $variable
    • Use parentheses: $(command) to execute a command and return the result
    • Indirect reference: ${!var}

Special Variables

  • $#: number of positional parameters
  • $*: all positional parameters
  • $@: all positional parameters (individually quoted)
  • $?: exit status of the last command
  • $$: process ID of the current shell

Arithmetic Operations

  • Basic operations: ((expression))
    • Example: a=$((1 + 2)) results in 3
  • Increment/Decrement: ++, -

Conditional Testing

File Tests

  • e filename: checks if a file exists
  • r filename: checks if a file is readable
  • w filename: checks if a file is writable
  • x filename: checks if a file is executable

String Tests

  • z string: checks if a string is empty
  • n string: checks if a string is not empty
  • string1 = string2: checks if strings are equal
  • string1 != string2: checks if strings are not equal

Numeric Tests

  • eq: equal to
  • ne: not equal to
  • gt: greater than
  • ge: greater than or equal to
  • lt: less than
  • le: less than or equal to

Globbing

Wildcards

  • ``: matches any sequence of characters
  • ?: matches a single character
  • [seq]: matches any character in seq
  • [!seq]: matches any character not in seq
  • {}: matches any of the patterns within braces

Examples

  • ll {*.log,*.txt}: list files ending in .log or .txt
  • ls /bin/g*: list all files in /bin starting with g
  • ls /bin/*t: list all files in /bin ending with t
  • ls /bin/g[^e]*t: list all files in /bin starting with g, ending with t, and the second character not being e

Conditional Statements

for Statement

for file in *.txt;
do
    echo "Processing $file"
done

case Statement

case $day in
    "Monday")
        echo "Today is Monday"
        ;;
    "Tuesday")
        echo "Today is Tuesday"
        ;;
    *)
        echo "Today is not Monday or Tuesday"
        ;;
esac

if Statement

if [ $num -gt 5 ]; then
    echo "num is greater than 5"
elif [ $num -eq 5 ]; then
    echo "num equals 5"
else
    echo "num is less than 5"
fi

while Statement

while <condition>
do
    <commands>
done

shift Command

Purpose

  • Shifts the positional parameters to the left

Usage

shift [n]  # default is to shift by one position

Example

#!/bin/bash
while [[ $# -gt 0 ]]
do
    echo "Processing $1"
    shift
done

Common Operators

  • +: addition
  • ``: subtraction
  • ``: multiplication
  • /: division
  • %: modulo
  • *: exponentiation

Regular Expressions

Basic Metacharacters

  • .: matches any single character
  • ^: matches the beginning of a line
  • $: matches the end of a line
  • [seq]: matches any character in seq
  • [^seq]: matches any character not in seq
  • [ - ]: within brackets, specifies a range, e.g., [A-Z]

Quantifiers

  • ?: matches 0 or 1 of the preceding element
  • ``: matches 0 or more of the preceding element
  • +: matches 1 or more of the preceding element
  • {n}: matches exactly n occurrences of the preceding element
  • {n,m}: matches between n and m occurrences of the preceding element
  • {n,}: matches n or more occurrences of the preceding element
  • {,m}: matches up to m occurrences of the preceding element

Escape Characters

  • .: matches any single character except newline
  • ^: matches the beginning of the string
  • $: matches the end of the string
  • ``: matches 0 or more occurrences of the previous element
  • +: matches 1 or more occurrences of the previous element
  • ?: matches 0 or 1 occurrence of the previous element
  • {: marks the beginning of a quantifier expression
  • }: marks the end of a quantifier expression
  • [: marks the beginning of a character class
  • ]: marks the end of a character class
  • \\: escape character
  • |: OR operator, means "or"
  • (: marks the beginning of a subexpression
  • ): marks the end of a subexpression

sed Command

Basic Usage

  • sed [options] 'command' <file>
    • e script: specify a script to process the input file
    • n: only show processed results
    • i: modify the original file directly

Common Commands

  • Replace:
    • s/pattern/replacement/: replace pattern
    • s/pattern/replacement/g: global replace
    • Case insensitive: s/pattern/replacement/gI
  • Delete:
    • sed '/pattern/d' file: delete matching lines
    • sed '/pattern/!d' file: delete non-matching lines
    • sed 'nd' file: delete the nth line
    • sed '2,$d' file: delete from the 2nd line to the last line
  • Insert and Append:
    • Insert text before each line: sed 'i\\text' file
    • Append text after each line: sed 'a\\text' file
  • Capture Group:
    • Define and reference capture groups: \\(...\\) and \\1, \\2, \\3, ...
    • Example: sed 's/\\([a-z]*\\), \\([a-z]*\\)/\\2, \\1/' file
  • Extended Regular Expressions:
    • Use extended regular expressions: sed -r 's/.../.../' file
    • Match one or more characters: +
    • Match zero or one character: ?

Example Script

#!/bin/bash
# Clean up australian-universities.csv file
sed -e '/university/I!d' \\
    -e '/,.*[a-zA-Z].*/d' \\
    -e 's/[.]/,/g' \\
    -e 's/,$//' australian-universities.csv

# Modify heroes.json and villains.json files
# Change "seat" to "crew"
sed -e 's/"seat":/"crew":/' heroes.json

# Add comma in villains.json
sed -e 's/\\([^][{},]\\)$/\\1,/' villains.json

# Add quotes to unquoted field names
sed -r 's/([a-z]+):/"\\1":/' villains.json

Git

Basic Commands

  • Initialize repository: git init
  • View status: git status
  • Add files to staging area: git add <files>
  • Commit changes: git commit -m "message"
  • View commit history: git log
  • Compare commits: git diff <commit> <commit>
  • Create branch: git branch <name>
  • Switch branch: git checkout <name>
  • Merge branch: git merge <branch>

Advanced Commands

  • View branches: git branch
  • Create new branch and switch: git checkout -b <name>
  • Merge branch: git merge <branch>
  • View differences: git diff <commit>
  • View file differences: git diff <file>
  • View staged changes: git diff --cached

Advantages of Git

  • Understand code change history
  • Determine the latest version of the code
  • Remote code backup
  • Easily revert errors/restore working versions

grep Command

Basic Usage

grep is used to search for lines matching a specific pattern in files and output the matching lines.

Common Options

  • i: ignore case
  • v: invert match, display lines that do not match
  • c: display count of matching lines
  • o: display only the matching part
  • r or R: recursive search in directories

Example Scripts

#!/bin/bash
# Count occurrences of "rabbit" and "Rabbit"
echo "rabbit count:"
grep rabbit Alice_in_Wonderland.txt | wc -l  # 5
echo "Rabbit count:"
grep Rabbit Alice_in_Wonderland.txt | wc -l  # 47
echo "Both rabbit and Rabbit count:"
grep -i rabbit Alice_in_Wonderland.txt | wc -l  # 52

# Count occurrences of "Alice"
echo "Alice count:"
grep -o Alice Alice_in_Wonderland.txt | wc -l  # 401

# Count lines not containing "Caterpillar" or "caterpillar"
echo "Lines not containing caterpillar:"
grep -vic caterpillar Alice_in_Wonderland.txt  # 3732

# Find specific team and player scores in arcade.csv
echo "Lines with team GREEN:"
grep GREEN arcade.csv

echo "Molly's score:"
grep Molly arcade.csv | cut -d, -f6

echo "Lines with machine b and score starting with 4:"
grep ',b,' arcade.csv | grep ',4[0-9]*$'

awk Command

Basic Usage

  • Read each line of a file: awk '{print $0}' file
  • Field separator: F <char>
  • Create variables: v <variable>=<value>

Built-in Variables

  • ARGC: number of command-line arguments
  • ARGV: array of command-line arguments
  • FILENAME: name of the current input file
  • FNR: line number in the current file
  • FS: input field separator
  • NF: number of fields in the current line
  • NR: total number of lines
  • OFS: output field separator
  • $0: entire current line
  • $1, $2, ...: first, second, ... fields in the current line

Mathematical Expressions

  • Basic operations: +, , , /, %, ^
  • Increment: ++variable, variable++

Conditional Rules

  • Test format: <expression><relop><expression>
    • Relational operators: ==, !=, <, <=, >, >=
  • Truth value: non-zero is true, zero is false

Formatted Output

  • printf formatting:
    • %c: character
    • %s: string
    • %d: integer
    • %f: floating-point number
    • %e: scientific notation
    • %g: shortest representation (e or f)

Example Scripts

Convert Date Format

awk -F / '{print $2"/"$1"/"$3}' american_dates.txt

Print Cumulative Sum

awk '{x += $1; print x}' add_this.txt

Print Total Sum

awk '{x += $1} END {print x}' add_this.txt

Print Invalid Lines

Print invalid lines in /lab/week9/australian-universities.csv:

awk -F"," 'NF!=4 || $1 !~ /University/ {print}' australian-universities.csv

Add Numbers to CSV File

Add numbers to /lab/week9/original_151.csv and save to numbered_151.csv:

awk '{print FNR "," $0 > "numbered_151.csv"}' original_151.csv

Print Height and Weight with printf

awk -F',' '{printf("%s is %.1fm tall and weighs %.1fkg\\n", $1, $2, $3)}' height_weight_151.csv

Print Name of Tallest Creature

awk -F',' '{if ($2 > x) {x = $2; max = $1}} END {print max}' height_weight_151.csv

Print Tallest Creature from Multiple Files

awk -F',' '{if ($2 > x) {x = $2; max = $1}} END {print max}' height_weight_151.csv more_height_weight.csv

Merge and Format CSV Files

paste original_151.csv height_weight_151.csv -d"," | awk -F"," '{printf("Name: %s, Height: %.1f, Weight: %.1f, Type: %s\\n", $1, $6, $7, $2)}'

Count Creatures by Type

# types.awk
{
    types[$2]++;
    if ($3 != "") {
        types[$3]++;
    }
}
END {
    for (i in types)
        print i ":" types[i];
}

# Usage
awk -F"," -f types.awk original_151.csv

List Creatures by Type

# types.awk
{
    types[$2] = types[$2] " " $1;
    if ($3 != "") {
        types[$3] = types[$3] " " $1;
    }
}
END {
    for (i in types)
        print i ":" types[i];
}

# Usage
awk -F"," -f types.awk original_151.csv

tr, comm, uniq, sort Commands

tr Command

  • Character translation:

    tr '[:upper:]' '[:lower:]' < file

    Example: tr '0' 'o' < woonky.txt

  • Delete characters:

    tr -d 'a-z' < file

comm Command

  • Compare two sorted files: Options:

    comm [options] file1 file2

    ```

    - `1`: suppress lines unique to `file1`
    - `2`: suppress lines unique to `file2`
    - `3`: suppress lines that appear in both files

uniq Command

  • Remove duplicate lines: Options:

    uniq [options] file

    ```

    - `c`: prefix lines by the number of occurrences
    - `d`: only print duplicate lines
    - `u`: only print unique lines

sort Command

  • Sort lines: Options:

    sort [options] file

    ```

    - `u`: unique, remove duplicates
    - `t`: specify field separator
    - `k`: sort by key (column)
    - `n`: numerical sort
    - `r`: reverse order
    - `o`: output to file
    - `m`: merge already sorted files

Example:

sort -k 6 -t, -n data.csv > sorted_data.csv
  • sort: command to sort text files
  • k 6: sort by the 6th column
  • t,: use comma as field separator
  • n: numerical sort
  • o sorted_data.csv: output to sorted_data.csv

head Command

Basic Usage

head command displays the beginning part of files. By default, it shows the first 10 lines.

Common Options

  • n: specify the number of lines to display

    head -n 5 filename

  • c: specify the number of bytes to display

    head -c 20 filename

Examples

  • Display the first 10 lines:

    head filename

  • Display the first 5 lines:

    head -n 5 filename

  • Display the first 20 bytes:

    head -c 20 filename

cut Command

Basic Usage

cut command is used to extract sections from each line of files. It is often used for processing delimited text files.

Common Options

  • b: extract by byte positions

    cut -b 1-5 filename

  • c: extract by character positions

    cut -c 1-5 filename

  • d: specify a delimiter

    cut -d ',' -f 1 filename

  • f: specify the fields to extract

    cut -d ',' -f 1,3 filename

Examples

  • Extract bytes 1 to 5:

    cut -b 1-5 filename

  • Extract characters 1 to 5:

    cut -c 1-5 filename

  • Extract the first field using a comma delimiter:

cut -d ',' -f 1 filename

  • Extract the 2nd and 4th fields using a tab delimiter:

    cut -d $'\\t' -f 2,4 filename

Error Handling

Check Command Line Arguments

#!/bin/bash
if [[ -z $1 ]]
then
    echo "ERROR: No input file!" > /dev/stderr
    exit 2
elif [[ ! -f $1 ]]
then
    echo "ERROR: File not found!" > /dev/stderr
    exit 1
else
    head -n 3 $1 | -n 1
fi

Add Optional Parameters

if [[ $# -gt 1 && $1 == '-r' ]]
then
    size=$2
else
    size=4
fi

Specific Script Examples

Compare File Lengths

#!/bin/bash
if [[ -z $1 ]] || [[ -z $2 ]]
then
    echo "Error: missing argument!" > /dev/stderr
    exit 2
elif [[ ! -f $1 ]]
then
    echo "Error: could not find file $1" > /dev/stderr
    exit 1
elif [[ ! -f $2 ]]
then
    echo "Error: could not find file $2" > /dev/stderr
    exit 1
else
    first_len=$(cat $1 | wc -l)
    second_len=$(cat $2 | wc -l)
    if [[ $first_len -gt $second_len ]]
    then
        echo "$1 is longer"
    elif [[ $first_len -lt $second_len ]]
    then
        echo "$2 is longer"
    else
        echo "equal lengths"
    fi
fi

Compare First and Last Lines of a File

#!/bin/bash
if [[ -z $1 ]] || [[ ! -e $1 ]]
then
    echo ERROR > /dev/stderr
    exit 1
fi
first=$(head -n1 $1)
last=$(tail -n1 $1)
if [[ $first == $last ]]
then
    echo "First and last lines are the same"
else
    echo "First and last lines differ"
fi

Optional Parameters Script

#!/bin/bash
if [[ ! -z $2 ]]
then
    bread="$2"
else
    bread="%============================%"
fi
echo "$bread"
echo "$1"
echo "$bread"

Running Scripts from Anywhere

Add the script directory to the PATH environment variable. For example, move sandwichsay to ~/bin and add the following to .bashrc:

export PATH=$PATH:~/bin

For Loop and File Size Statistics

Example: sizes.sh script to count the size of each file in the current directory, output the filename and size, and find the largest file and the total size.

#!/bin/bash
sum=0
max_size=0
max_name=''
dir='.'

while [[ $# -gt 0 ]]
do
    case $1 in
    -h) echo -d DIRECTORY Count files in DIRECTORY instead of .
        echo
        echo -h Display this help message and exit
        exit 0;;
    -d) if [[ -z $2 ]]
        then
            echo "Missing operand for -d argument"
            exit 1
        fi
        dir=$2
        shift;;
    *) echo "Unknown argument $1"
        exit 2;;
    esac
    shift
done

for f in $dir/*
do
    if [[ -f $f ]]
    then
        size=$(wc -c $f | cut -f1 -d " ")
        echo "$size $f"
        if [[ $size -gt $max_size ]]
        then
            max_size=$size
            max_name=$f
        fi
        sum=$((sum + size))
    fi
done
echo "Largest is $max_size in $max_name"
echo "Sum is $sum"

make Command

Overview

make is an automation tool for managing and building projects. By writing a Makefile, you can define how to compile and link programs and manage dependencies.

Basic Concepts

  • Target: the file or action make should generate or execute, usually an executable or intermediate file.
  • Dependencies: files or other targets required to create the target.
  • Commands: steps to generate the target, typically compile or link commands.

Makefile Syntax

  • Target, dependencies, and commands format:

    target: dependencies
    commands

Variables

  • Define and use variables in Makefile.

    CC = gcc
CFLAGS = -Wall -Werror

Built-in Variables

  • CC: compiler name (default is cc)
  • CFLAGS: compiler flags

Built-in Rules

make has built-in rules, such as how to compile .c files to .o files.

Phony Targets

Phony targets do not generate actual files, typically used for executing actions (e.g., cleaning).

.PHONY: clean
clean:
    rm -f *.o monty

Example Makefile

Compile and Link C Program

CC = gcc
CFLAGS = -Wall -Werror
TARGET = monty
SRCS = montyhall.c
OBJS = $(SRCS:.c=.o)

$(TARGET): $(OBJS)
    $(CC) $(CFLAGS) -o $(TARGET) $(OBJS)

%.o: %.c
    $(CC) $(CFLAGS) -c $< -o $@

clean:
    rm -f $(OBJS) $(TARGET)

Makefile with Install Rule

CC = gcc
CFLAGS = -Wall -Werror
TARGET = monty
SRCS = montyhall.c
OBJS = $(SRCS:.c=.o)

.PHONY: all clean install

all: $(TARGET)

$(TARGET): $(OBJS)
    $(CC) $(CFLAGS) -o $(TARGET) $(OBJS)

%.o: %.c
    $(CC) $(CFLAGS) -c $< -o $@

clean:
    rm -f $(OBJS) $(TARGET)

install: $(TARGET)
    cp $(TARGET) /usr/games

.gitignore Example

echo "monty" > .gitignore
git add .gitignore
git commit -m "Add .gitignore to ignore monty binary"

Usage

  • Build project: make
  • Clean project: make clean
  • Install program: make install

Example Scripts from Past Exams

2023 Example

Bash and awk Script col_count

This script returns the most frequently occurring item in a specified column of a given file. If there are multiple items with the same frequency, any one of them may be returned.

#!/bin/bash
# Check number of arguments
if [ "$#" -ne 2 ]; then
    echo "Usage: $0 <filename> <column_number>"
    exit 1
fi

# Get arguments
filename=$1
column_number=$2

# Check if file exists
if [ ! -f "$filename" ]; then
    echo "File not found!"
    exit 1
fi

# Check if column number is a positive integer
if ! [[ "$column_number" =~ ^[1-9][0-9]*$ ]]; then
    echo "Column number must be a positive integer."
    exit 1
fi

# Check if column number is within file's column range
max_columns=$(head -1 "$filename" | awk -F'\\t' '{print NF}')
if [ "$column_number" -gt "$max_columns" ]; then
    echo "Column number exceeds the number of columns in the file."
    exit 1
fi

# Use awk to count each item's occurrence and find the most frequent one
awk -v col="$column_number" -F'\\t' '
{
    count[$col]++
}
END {
    max_count = 0
    for (item in count) {
        if (count[item] > max_count) {
            max_count = count[item]
            max_item = item
        }
    }
    print max_item
}
' "$filename"

2022 Example

awk Script to Filter and Count Submissions

This script reads an input file, filters records with time after 23:00 (2300) for Ass2, counts the submissions for each student ID, and outputs IDs with three or more submissions in alphabetical order.

very_suby.awk

#!/usr/bin/awk -f
BEGIN {
    FS = "\\t"  # Set field separator to tab
}

# Process each line
{
    time = $2
    id = $3
    assessment = $4

    # Only consider records with time after 2300 and assessment "Ass2"
    if (time >= 2300 && assessment == "Ass2") {
        count[id]++
    }
}

# Execute after processing all lines
END {
    # Find IDs with three or more submissions
    for (id in count) {
        if (count[id] >= 3) {
            result[id] = count[id]
}
}

# Output results in alphabetical order
n = asorti(result, sorted_ids)
for (i = 1; i <= n; i++) {
    id = sorted_ids[i]
    print id, result[id]
}

}

very_suby.sh

#!/bin/bash
# Check number of arguments
if [ "$#" -ne 1 ]; then
    echo "Usage: $0 <filename>"
    exit 1
fi

# Get argument
filename=$1

# Call awk script to process file
awk -f very_suby.awk "$filename"

Usage

  1. Save very_suby.awk and very_suby.sh in the same directory.

  2. Ensure both scripts have executable permissions:

    chmod +x very_suby.awk very_suby.sh

  3. Run very_suby.sh with the data file:

    ./very_suby.sh datafile

Explanation

  • very_suby.awk:
    • BEGIN { FS = "\\t" }: Set field separator to tab.
    • time = $2, id = $3, assessment = $4: Extract time, student ID, and assessment fields.
    • if (time >= 2300 && assessment == "Ass2") { count[id]++ }: Filter records and count submissions for Ass2.
    • END { ... }: Find IDs with three or more submissions and print them in alphabetical order.
  • very_suby.sh:
    • if [ "$#" -ne 1 ]; then ... fi: Check number of arguments.
    • filename=$1: Get the data file name.
    • awk -f very_suby.awk "$filename": Call awk script to process the file.

Antibugging

Antibugging involves adding tests, typically near the start of a program, to ensure that data coming from the user is consistent with what is expected. It is important because it prevents computing nonsensical results from absent, out-of-range, or otherwise problematic data.