Literature: Advanced Bash Scripting Guide
http://www.tldp.org/LDP/abs/html/index.html

This workshop's theme is shell scripting. The goal is that after this and next Thursday, you will have crash-coursed the necessary skills to complete the second homework assignment (involving a programming task where nodes communicate with each other) using shell scripting and netcat. If all goes well, the last homework exercise of the course will build on the work in this second exercise and further strengthen these skills.

1. Making a shell script and some example code.

a) Copy the following lines into a text file. The first line indicates
that the file should be run by a Bash shell script interpreter. Save the
file (to e.g. myscript.sh) and make it executable (chmod u+x myscript.sh).
Now you can run it by typing ./myscript.sh.

#!/bin/bash

# Usually, anything from # to the end of the line is a comment.
# If you want to output a #, you should protect it with quotes or
# apostrophes: '#' or "#" ("'#'" protects the apostrophes too).

echo "Hello world, this is $(hostname)."
echo "Today it is $(date +%d-%m-%Y)."  # Instructing 'date' on formatting.

for HOST in ukko023 ukko024 ukko025
do
  if ping -c 1 "$HOST"   # Ping the host exactly once.
  then
    echo "$HOST is allliiiveee." # Note: exclamation mark is special.
  fi
done
 

2. Writing to a file from a script.

The special character '>' can be used to output text to a file; '>>'
appends, while '>' overwrites any existing file.

Try the following commands:

echo "Hei" > foo.txt
echo "Hallo hallo" >> foo.txt

Then change the script in 1 to write the aliveness of hosts to a file
result.txt instead.

(For completeness, '<' reads a file and sends it to a command's standard
input. Try "sort < foo.txt", for example.)

3. Redirecting the output of a command to another command.

The special character '|' takes the output of a command and passes it to
another's STDIN (standard input). A script can read one line of input
with 'read'; more than one line requires a while or for loop.

For example, there are three ways you can provide input to the command
'grep', which tries to match strings:

(All these will write out the lines in foo.txt which include the word "Hei".)

grep Hei foo.txt
cat foo.txt | grep Hei    # 'cat' reads a file and prints it out
grep Hei < foo.txt
 

The pipes can go on and on:
cat foo.txt | grep Hei | rev  # 'rev' turns a string backwards.

4. For the future tasks, you will need to pick a port number that is not
in use by others on the cluster machines you are on (you will get an
error "address already in use" otherwise). Port numbers go up to 65000
(you cannot reserve the lowest numbers as a regular user, though).

You can for example use the last 5 numbers from your student number (if
the first number is 6 or larger, replace it with something smaller. If
it is 0, leave it out or replace.

You can use the following command to list ports on a host that are being listened to already (the parameters are for Listen, TCP, UDP, IPv4 only, and listing them by numbers). For example a line with "0.0.0.0:36841" in the local address column means, for our purposes, that port 36841 is already taken.

netstat -ltu4 --numeric-ports

If you try to start a netcat listening on a port that is already in use, you will get an error message in any case.

5. Making a selective netcat listener.

Log in to two Ukko cluster nodes, e.g. ukko023 and ukko024. (You can use
the command 'ukko' on melkki and other interactive servers to connect to
a node that is not reserved.)

Recall the netcat exercise from last Thursday and start a listener on
one node, and connect to it from another node.

nc -l <port> | grep -i hei   # -i means "ignore case", matches HEI, hei etc

nc <hostname> <port>

Try typing 'hei', 'hoi' and 'huhhei' into the latter nc.

6. Making a script output to netcat.
 

Set up a netcat listener with the -k parameter to keep it alive between
connections.

nc -l -k <port>

On the connecting node, write and run the following script with the
appropriate modifications to hostname and port:

#!/bin/bash

for NUMBER in $(seq 1 5)
do
  # {} protects the variable name.
  echo "Hello, number ${NUMBER}."
done  | nc <hostname> <port>

Modify the script to have the pipe ('|') after the echo line instead of
after the entire for loop. Try this with the "nc -l -k" listener, and
then with a listener set up with just "nc -l" without the -k.

Can you explain what causes the difference? See 'man nc' for more
information.
 

Fun obscure fact: If you open a netcat listener using UDP (the default is TCP, so this only applies once you enter the wild side), it will lock onto a single sender address and port, so you cannot connect to it from multiple sources. This is not brought up on the man page for nc.