Hackviking He killed Chuck Norris, he ruled dancing so he took up a new hobby…

6Apr/160

Pi: Python script for BtSync status LCD

Adding and LCD display to a Pi project can make it so much easier to use. Displaying current IP address and status of some task that you only have to interact with when something went wrong. In this example we have the 20x4 (20 characters x 4 lines) LCD status display of my BtSync Satellite that I built a while back. Since this box is going to sit on a DHCP network I wanted to display the IP-address so I know what to SSH against. I also wanted to display some status metrics about disk mounts, services and application specific performance counters.

In this scenario the box is running BtSync to keep an offsite encrypted backup off my NAS. For security reasons I have to SSH to the box after a power cycle or failure to enter the encryption key for the disk. That's why I want it to display it's current IP-address on the display. I also want to see the current status of the encryption mount, BtSync service and the upload/downloads going on. That way I know when I have to SSH into the box to sort something out.

So what does this script actually do? It runs an infinit loop until you kill the process. Every 45 seconds it checks the stuff that doesn't need updating all that often and every 3 seconds it checks the current status of the BtSync operations.

Every 45 seconds:

  • Check the current IP-address
  • Check if the Truecrypt volume is actually mounted
  • Check if the BtSync daemon is running

Every 3 seconds:

  • Checks number of files synced
  • Checks number of files to be synced
  • Checks the current download speed
  • Checks the current upload speed

Pre-Requirements

First you need to wire up the LCD, it differs a bit from model to model but there are ton of descriptions on pinouts if you Google your specific model. Then go ahead and run raspi-config or what ever equivalent your brand of Pi uses. Go under Advanced and enable I2C. Then we download some tools that we need:

sudo apt-get install i2c-tools python-dev libxml2-dev libxslt1-dev zlib1g-dev python-smbus

This will install all the things you need to communicate over the GPIO header to your LCD and also libraries needed for the features in the script. Then you can go ahead and download the script:

 wget -O https://raw.githubusercontent.com/kallsbo/PiBtSyncLCD/master/lcd_info.py

Configuration

There are a few configs you can do in the script, just use nano to edit the script file.

# Configuration - LCD
LCD_BUS = 2 # The bus that the LCD is connected to. (Raspberry Pi usually 1, Banana Pi usually 2 - can be checked with i2cdetect)
LCD_I2C_ADDR = 0x27 # I2C device address of the LCD, use i2cdetect to find your displays address
LCD_WIDTH = 20 # Number of characters that each line can handle
LCD_BACKLIGHT = 0x08 # On
#LCD_BACKLIGHT = 0x00 # Off

# Enviorment config
NETWORK_NIC = "eth0" # Network card used
TRUECRYPT_MOUNT_PATH = "/mnt/tc_disk" # path where the truecrypt disk is mounted
BTSYNC_SRV_NAME = "btsync" # name of the btsync service
BTSYNC_URL = "https://localhost:8888/gui/" # Web GUI address for btsync
BTSYNC_CRED_FILE = "/mnt/tc_disk/btsync_cred.json" # JSON file with btsync credentials

Script functions

If we first look at the main method it is simple enough. We run the lcd_init() function to initialize the LCD. All the LCD functions was forked from a script written by Matt Hawkins @ Raspberry Pi Spy. Then we set a simple update counter that keeps track of if the 45 second mark has been hit and if we should check the IP, mount and daemon status. It's initially set to 16 so it will run the first loop and the counter is reset. Then it pluses one for every 3 second run so whenever it's larger then 15 the 45 seconds has elapsed.

get_ip_address() - Simple function that takes the adapter name (eth0) as a parameter and then grabs the current IP-address of that adapter.

is_trucrypt_mounted() - Uses the os.path.ismount() function to check if the mount point is actually utilized by the Truecrypt drive.

get_btsync_cred() - Checks for the json file on the encrypted volume containing the UI username and password for BitTorrent Sync. I used this approach to keep the credentials safe. This function is executed every 45 seconds to make sure that the script get's the credentials when the disk get's mounted.

get_btsync_token() - Sends the initial request to the BitTorrent Sync UI (api) to get the token needed for all the requests to the API. This will also run every 45 seconds to make sure the token never times out and to counter any recycles of the web service.

Every three seconds the script checks if it has the credentials and token needed for the requests and if so runs the get_btsync_info().

get_btsync_info() - This function takes two parameters LLforSpeed and LLforFiles which stands for LCD Line. This value is used to display the information on the LCD panel row you like. It simply builds an url with the GLOBAL credentials and token and get the same json that the UI uses. Then parses it and get the total file count for downloaded files as well for files that are in the download queue. It also grabs the current upload and download speed and converts it to Mb/s and displays it on the LCD.

Credentials JSON file

This is just a plain JSON file containing the credentials. You can modify the script to hard code the credentials in the script but that will impact the security of the script. Here is an example of the credential files:

{
"BTSYNC_USR": "btuser",
"BTSYNC_PSW": ":wDHz56L.blDgM,3Jm"
}

Cred and final thoughts

This is a simple setup for keeping track of your BitTorrent Sync daemon. It can be modified to just display the current info about btsync and not care about Truecrypt and the other extras I implemented for the "satellite" build.

I want to give cred to, as mentioned before, Matt Hawkins for the LCD example scripts that my LCD code is based upon. Also want to thank all bloggers and forum users for the posts I have read to be able to do this. This was my first time ever to use the GPIO header on the Pi for anything else then pre-built stuff like touch displays.

Any questions or suggestions? Please comment! And please follow me on a social media of your choice for updates...

3Apr/160

Pi: Geo-location backup with BtSync

Building a geo-location backup for your NAS is a good idea! To spread the risk over two or more locations increases your backup value a lot. Most people confuse redundancy and backup. If you only have a USB-disk backup of your NAS it only protects you against hardware failure. If there is a fire or a break in you will still lose your data. A lot of people take a USB-disk to a second location, like their office, to mitigate this problem. But to be honest how often will that backup be done if you have to remember to bring the disk back and forth? We want automatic backups to our offsite location, in this case my office. So we are going to build a BitTorrent Sync "satellite"
Continue reading...

22Mar/1618

OpenVPN performance on the Pi

Setting up an OpenVPN router on the Pi is pretty straight forward but what about performance? How much performance do we lose by using the Raspberry Pi or the Banana Pi? I have been testing a few different models to see what the overall performance difference is. I also wanted to compare them against each other. OpenVPN is heavy on the CPU due to it's encryption, there are a lot of guides out there about turning the encryption of but why even use a VPN then? It all depends on what you use your VPN tunnel for and what kind of through put you actually need. In this test I have used all three main versions of the Raspberry Pi and a Banana Pi.

Continue reading...

19Mar/1630

Pi: Make a VPN gateway with UPnP port forwarding

Tunneling your traffic over an encrypted VPN can be good for both privacy concerns and circumventing geoblocking. If a service is only offered in a specific country or blocked at your current location. My use case is a bit of both. Currently living in the USA which is the biggest surveillance state on earth I want my traffic to originate from my home country, Sweden, where I know the law and whats allowed and not allowed. Avoiding the mighty force of the NSA completely can only be done by unplugging but at least it's a little bit better. Also several services I want to use is only offered in Sweden, like local Swedish news as an example. Both of these can be solved by setting up a VPN tunnel back Sweden!
Continue reading...

12Mar/164

Banana Pi: First run

Banana Pi was created to fill the need for more powerful hardware than the Raspberry Pi supplied. There are a lot of single board computers spinning of the Raspberry Pi success. Even though Raspberry Pi got the throne much thanks to it's simplicity and relatively ease of use, compared to for example the Odroid, it has been lacking hardware vice for some applications. Raspberry has maintained it's position thanks to it's growing community and further development. With the release of the Raspberry Pi 3 they have at least done a good catch up in terms of performance but are still lacking in other hardware areas.

The Banana Pi I used for the first time today is the very first Banana Pi. This particular one is a bit of a globe-trotter!  I ordered it from Chine over a year ago, while I was still living in Sweden. Un-boxed it, put it in it's case and put it away in a drawer. When a moved to California last year it got stuffed in one of the moving boxes and I finally had time to use it. Even though it's first generation and old it still leaves the Raspberry Pi behind in some ways. Back in the day the dual core 1Ghz processor was a step up from the Raspberry Pi so was the 1Gb memory that was twice the size of what the Raspberry offered at the time.

Putting the Banana Pi along side the Raspberry Pi 3 we see that Raspberry is back on the throne when it comes to performance. It has also added on board wifi and bluetooth which makes wonders for my bedroom Kodi install but the Banana Pi isn't beaten yet if you ask me. It still have a 1Gbit ethernet port while the Raspberry still only supplies you with 100Mbit. Why would this matter? When I started testing BitTorrent Syncing for my geo-location backup I ended up not using a Raspberry Pi for just that reason. Since my data was on a NAS the indexing of files over a 100Mbit connection was just to slow. In the end the 1Gbit ethernet connection on a Odroid-C1 performed so much better then the Raspberry Pi.

Another feature that I really like with the Banana Pi is the SATA port and SATA power connector included on the board. The ability to connect a SATA hard drive directly to the board without using USB opens up for some interesting implementations. In the end I really like Raspberry Pi and Odroid and Banana Pi.... They all share a great base to stand on and are good for different applications. The Raspberry Pi is my first choice for "mainstream" applications like Kodi, OpenVpn servers or Transmission bittorrent servers. But when it comes to building the little more specialized stuff there are other, and some times better, options out there.

When I did the first run of the Odroid (also over a year after I bought it!) I realized it was a bit more complicated then the Raspberry Pi. No sleek easy config tools already on the image. Not as much safety nets to prevent you from messing up your kernel etcetera. So taking out my Banana Pi I expected the same! First I realized that Raspbian is available for the Banana Pi as well! And the sleek easy, step by step setup and configuration was available as well!

bananian-config

If you have ever used the CLI config tool on the Raspberry Pi you will feel right at home! One addition that I really liked is that it forces you to change the root password, in my opinion that should be implemented on the Raspberry Pi as well! You would be amazed how many unsecure Raspberry Pi's there are connected to the internet with SSH ports available. When I first started looking into that I was actually surprised since this isn't something that the regular consumer buys and plugs in to there network.

The tool will also let you configure the following:

  • set your timezone
  • set your locale
  • set your hostname
  • set which hardware your on, Banana Pi - Banana Pro etc...
  • expand your root file system

Then just reboot the system and make sure that everything is up to date!

bananian-update
apt-get update
apt-get upgrade

From what I have read so far, and tested my self, you can more or less run anything on the Banana as you can do on the Raspberry. I'm really looking forward to setting up some implementation utilizing the SATA port. What are your thoughts on this feel free to comment either here or on Google+.

11Mar/160

BtSync: Let my NAS sleep

BitTorrent Sync (btsync) will re-index all files every 10 minutes to look for new files to sync. After installing btsync on Raspberry Pi initially and then ending up running it on an Odroid-C1 my WD NAS never sleeps. The first 48 hours it was expected since it took all that time to index all the files. But now my NAS never sleeps. Since this is a backup solution I don't have the need for it to actually re-index every 10 minutes. If the Western Digital LiveBook Duo is left alone for 20 minutes it will spin down it's hard drives. So here is a quick guide on how to configure the re-index interval for btsync installed on an arm system. This is done on the system initially installed with the Raspberry Pi: BitTorrent Sync guide.

Continue reading...

10Mar/160

Raspberry Pi: BitTorrent sync alternative Syncthing

Btsync is a great tool for syncing data and backups to several different locations. By leveraging the torrent technology you can increase the download/upload speeds by adding more nodes to the swarm. Installing Btsync on a built in system like the Raspberry Pi or Odroid-C1 is a challenge since it consumes a lot of system resources, read more about installing BitTorrent Sync on Raspberry Pi. Reading up on the subject a lot of people did suggest that Syncthing takes up less system resources then btsync so let's install it and try it out.

Pre requirements

Same as for the btsync install I need to access my NAS root NFS share to get all the data that I will sync. So first I setup a mount point and do a persistent mount of that share.

sudo mkdir /mnt/stroot
sudo chmod 777 /mnt/stroot

Then let's edit /etc/fstab so run sudo nano /etc/fstab and add a line representing your share.

192.168.0.5:/nfs /mnt/stroot nfs rsize=8192,wsize=8192,timeo=14,nolock,intr 0 0

This will give me the root for all the shares so I don't have to map every share individually but are able to access all of them as subfolders.

To test this out we can run sudo mount -a. Then reboot the system and check that the mount is persistent. If you have issues with the persistence and are running DHCP please see my Raspbian:fstab doesn't mount NFS on boot article for reference.

Install

Download the Syncthing repository key and add it.

wget -O - https://syncthing.net/release-key.txt | sudo apt-key add -

Add the repository

echo "deb http://apt.syncthing.net/ syncthing release" | sudo tee /etc/apt/sources.list.d/syncthing.list

Update the repository with sudo apt-get update and then install with

sudo apt-get install syncthing

Then try to run it with syncthing, as soon as you see it start properly you can stop it with CTRL + C. To set it up to run as a service we need first reconfigure the bind address.

nano /home/pi/.config/syncthing/config.xml

Look for the gui option, it should look like this:

<gui enabled="true" tls="false">
<address>127.0.0.1:8384</address>
<apikey>2Yy41r5k5oocz6rDIWVsKeP4wUx9uknD</apikey>
<theme>dark</theme>
</gui>

Here you can change all the settings in regard to the UI. Change the bind address to 0.0.0.0 so it will respond to all ip addresses. If your running on a fixed IP and not DHCP you can go ahead and bind it to a specific address if you like. You can also change the tls setting to true, enabling https for the UI. This will however result in a security warning when you browse to the UI due to the self signed certificate.

If you now run syncthing from the command line it will bind to the IP and you will be able to navigate to the UI.

Autostart Syncthing as a "service"

First we need to create a init.d script for Syncthing.

sudo nano /etc/init.d/syncthing

Copy and paste the following into it:

#!/bin/sh
### BEGIN INIT INFO
# Provides:          Syncthing
# Required-Start:    $local_fs $remote_fs $network
# Required-Stop:     $local_fs $remote_fs $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Syncthing
# Description:       Syncthing is for backups
### END INIT INFO
 
 
# Documentation available at
# http://refspecs.linuxfoundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/iniscrptfunc.html
# Debian provides some extra functions though
. /lib/lsb/init-functions
 
 
DAEMON_NAME="syncthing"
DAEMON_USER=pi
DAEMON_PATH="/usr/bin/syncthing"
DAEMON_OPTS=""
DAEMON_PWD="${PWD}"
DAEMON_DESC=$(get_lsb_header_val $0 "Short-Description")
DAEMON_PID="/var/run/${DAEMON_NAME}.pid"
DAEMON_NICE=0
DAEMON_LOG='/var/log/syncthing'
 
[ -r "/etc/default/${DAEMON_NAME}" ] &amp;&amp; . "/etc/default/${DAEMON_NAME}"
 
do_start() {
  local result
 
	pidofproc -p "${DAEMON_PID}" "${DAEMON_PATH}" &gt; /dev/null
	if [ $? -eq 0 ]; then
		log_warning_msg "${DAEMON_NAME} is already started"
		result=0
	else
		log_daemon_msg "Starting ${DAEMON_DESC}" "${DAEMON_NAME}"
		touch "${DAEMON_LOG}"
		chown $DAEMON_USER "${DAEMON_LOG}"
		chmod u+rw "${DAEMON_LOG}"
		if [ -z "${DAEMON_USER}" ]; then
			start-stop-daemon --start --quiet --oknodo --background \
				--nicelevel $DAEMON_NICE \
				--chdir "${DAEMON_PWD}" \
				--pidfile "${DAEMON_PID}" --make-pidfile \
				--exec "${DAEMON_PATH}" -- $DAEMON_OPTS
			result=$?
		else
			start-stop-daemon --start --quiet --oknodo --background \
				--nicelevel $DAEMON_NICE \
				--chdir "${DAEMON_PWD}" \
				--pidfile "${DAEMON_PID}" --make-pidfile \
				--chuid "${DAEMON_USER}" \
				--exec "${DAEMON_PATH}" -- $DAEMON_OPTS
			result=$?
		fi
		log_end_msg $result
	fi
	return $result
}
 
do_stop() {
	local result
 
	pidofproc -p "${DAEMON_PID}" "${DAEMON_PATH}" &gt; /dev/null
	if [ $? -ne 0 ]; then
		log_warning_msg "${DAEMON_NAME} is not started"
		result=0
	else
		log_daemon_msg "Stopping ${DAEMON_DESC}" "${DAEMON_NAME}"
		killproc -p "${DAEMON_PID}" "${DAEMON_PATH}"
		result=$?
		log_end_msg $result
		rm "${DAEMON_PID}"
	fi
	return $result
}
 
do_restart() {
	local result
	do_stop
	result=$?
	if [ $result = 0 ]; then
		do_start
		result=$?
	fi
	return $result
}
 
do_status() {
	local result
	status_of_proc -p "${DAEMON_PID}" "${DAEMON_PATH}" "${DAEMON_NAME}"
	result=$?
	return $result
}
 
do_usage() {
	echo $"Usage: $0 {start | stop | restart | status}"
	exit 1
}
 
case "$1" in
start)   do_start;   exit $? ;;
stop)    do_stop;    exit $? ;;
restart) do_restart; exit $? ;;
status)  do_status;  exit $? ;;
*)       do_usage;   exit  1 ;;
esac

Make the init.d script executable and make it start on boot.

sudo chmod +x /etc/init.d/syncthing
sudo update-rc.d syncthing defaults

Now you can start the Syncthing service like any other service.

sudo service syncthing start

Conclusion

The reason I looked into Syncthing in the first place was the memory consumption of btsync. After running a few tests I can see that it's the same with Syncthing. By the time I'm writing this my btsync instance running on my Odroid-C1 has index almost 4Tb of data and is running smoothly. Syncthing is opensource, which I like a lot, but it's a much more immature product then btsync. The windows client is just an .exe that will run a local web server that you connect to. Btsync seems more mature and robust from my initial tests. Syncthing will probably develop into something better but at this time I will go with btsync instead. Btsync is also developed by the torrent technology creators which make me believe in it even more. Install guide for btsync...

Why don't you install both of them and try them out and let me know what you think in the comments?

9Mar/160

Odroid: Swap on USB-stick

Swap is used when the kernel runs out of memory and need to swap less used information to the disc to make space for information currently in use. In a good setup the swap will never be used but if it's needed and not existing it will make processes crash.

I have just started out testing BitTorrent Sync (btsync) on my Odroid-C1 and noticed that all the memory was allocated more or less straight away after starting the btsync service. So just to be safe during my test, to avoid crashes, I added a 8Gb USB stick to use for swap space. You can of course use the SD-card but that will take up space needed for the system as well as shorten the life of your SD-card.

Continue reading...

8Mar/160

Raspberry Pi: BitTorrent Sync

BitTorrent is not only pirated movies and tv-shows it's an actual technology for moving large amounts of data between several nodes at once. There are several applications for this sync technology. Large companies like Facebook use it to sync new releases among all their production servers for example. If three or more nodes are syncing the same content it will increase the sync speed since it's a non centralized technology. All the clients will help to seed the data they already have which will make the sync go faster and faster the more nodes the data propagates to.

Btsync will make use off all the memory available in your Raspberry Pi so I recommend running this on it's own. I have tested this out on both a Raspberry Pi 2 B and the older Raspberry Pi B and they more or less run the same. It will take some time to index all your data but when testing it on a regular windows pc it was pretty slow as well. Continue reading...

7Mar/160

Odroid: First setup

If you have installed Raspberry Pi in the past the Odroid will not be a problem. It a little more hands on and require a little more effort then the Raspberry Pi. Here is a quick guide how to get you Odroid up and running. In this example I use an old Odroid-C1 that I found in one of my drawers.

Download image

First you need to download an image for your SD-card (or eMMC module, which is supported by Odroid). The Hardkernal download page have all the available images. Then make use of SD-formatter and Win32DiskImager to get the image onto the SD-card. When you first boot you Odroid you will notice that it doesn't give HDMI output as appose to the Raspberry Pi. You will need to connect it to your network and make sure you have a router or similar running a DHCP server. Once you see the IP in the DHCP list you can go ahead and use putty to SSH into it. Default username is root and default password is odroid.

Initial config

First thing you want to do is to change the default root password by running passwd. Now you have a much more secure box then before. The Raspberry Pi ships with a config script that you can use to make your basic config, the Odroid doesn't. I did however find a great script called odroid-utility written by Mauro Ribeiro, which seems to be an employee of Hardkernal actually. The script is open sourced on Github and gives you more or less the same capabilities as the Raspberry Pi config script. Since it's not shipped we need to download it, make sure that your Odroid have an internet connection and run the following:

sudo -s
wget -O /usr/local/bin/odroid-utility.sh https://raw.githubusercontent.com/mdrjr/odroid-utility/master/odroid-utility.sh
chmod +x /usr/local/bin/odroid-utility.sh
odroid-utility.sh

odroid-utility

It gives you the ability two do basic configuration like HDMI, resize root partion and change the hostname. You can also update the kernel with this tool. Each time you run the script it updates it self so once you download it you can be sure you have the latest release.

Update your system

After all this just do a basic package upgrade and your Odroid is ready for use.

sudo apt-get update
sudo apt-get upgrade

Timezone

This one is really important for some implementations since correct date and time can break some setups.

sudo dpkg-reconfigure tzdata