Month: April 2016

Simple rsync backup of your remote server w/ crontab

If you have a remote server, e.g. for you website, small business or whatever, it is a good idea to replicate it regularly onto local storage. You at least want /etc, /home, /var/www, /usr/local to be in the backup. Maybe more stuff, depending on what you are doing.

For now I assume you are running the backup on OS X’s user with admin/sudo rights, or some Linux user with sudo rights.

The reason why we want to do this with sudo and root on the remote system is to have a simple way to backup the whole system with all files, users and permissions. If you only want to backup a remote user, you don’t need the whole sudo and root part, but can use the remote user login instead.

First, you need the backup script:

#!/bin/bash

if [ "${UID}" != 0 ]
then
    echo "Must be run as root."
    exit 1
fi

shopt -s nocasematch

pushd `dirname $0` > /dev/null
SCRIPTPATH=`pwd`
popd > /dev/null

if [[ "${PWD}" != "${SCRIPTPATH}" ]]
then
    echo "Wrong path: ${PWD}"
    exit 2
fi

RSYNC_RSH="ssh -C -o IdentitiesOnly=yes -i /Path/to/key/without/password"
RSYNC_CMD='rsync -aP --delete --delete-after'
RSYNC_HOSTNAME='your.server.name'

($RSYNC_CMD -e "${RSYNC_RSH}" root@${RSYNC_HOSTNAME}:/etc :/home :/root :/usr/local :/var/www . > /tmp/rsync.log 2>&1 ) || \
(>&2 echo "An error occured while doing the backup: " ; cat /tmp/rsync.log )

exit $?

Adjust your hostname accordingly and also the directories to be backed up. Use the :/dir notation to reuse the SSH connection. Also make sure to use SSH key authentication, so no password is needed, since this backup is supposed to run via crontab. SSH access as root must only be allowed via key exchange! Make sure to disable password access, or else you are running a security risk! Also, keep the SSH key secret — it is the door to your server!

There are probably more secure ways, but they are more complicated than this.

Next up, put you script somewhere your sudo user can run it, and edit the crontab via “crontab -e”:

MAILTO=user@domain  # or simply your local user, to put it into the local mbox
0 * * * *       cd /Users/youruser/Documents/Backup && sudo /Users/youruser/Documents/Backup/backup.sh

You can make the script runnable with sudo without password by adding the following line to your /etc/sudoers file:

# allow passwordless access to the backup script
youruser ALL = (ALL) NOPASSWD: /Users/youruser/Documents/Backup/backup.sh

The script will only generate output on an error, hence cron will only send an email if there is an error.

If you are now also running TimeMachine, your server will have a nice, hourly history of backups.

Indexed Java 8 enums with gaps

This is an example enum for Java 8 that supports gaps in its indices is relatively fast for enums with a large amount of values:

import java.util.Arrays;
import java.util.Map;
import java.util.stream.Collectors;

public enum TestEnum {
   VALUE1( 0 ),
   VALUE2( 2 ),
   VALUE3( 50 );
   
   private final Integer id_;
   private static Map< Integer, TestEnum > values_ = 
      Arrays.stream( TestEnum.values() ).collect( Collectors.toMap(e -> e.id_, e -> e) );;
   
   private TestEnum( int i ) { id_ = i; }
   public static TestEnum fromInteger( int i )
   {
      return values_.get( i );
   }
   
}

How to concatenate strings efficiently in bash

The naive implementation for concatenating strings in bash goes something like this:

#!/bin/bash

statement="1234567890"

result=""

for i in $(seq 1 100000)
do
  result=${result}${statement}
done

wc <<< ${result}

This takes around two minutes on a modern machine. This is slow. Very, very slow.

Instead you can build an intermediate array and use the star operator to expand it into a string. Make sure to change the input field seperator to empty, so that you don’t get spaces in between the individual entries:

#!/bin/bash

statement="1234567890"

for i in $(seq 1 100000)
do
statements[${#statements[@]}]=${statement}
done

IFS= eval 'result="${statements[*]}"'

wc <<< ${result}

This will run in a few hundred milliseconds. You get roughly three orders of magnitude speedup.