Hackviking He killed Chuck Norris, he ruled dancing so he took up a new hobby…

11Mar/160

BtSync: Let my NAS sleep

BitTorrent Sync (btsync) will re-index all files every 10 minutes to look for new files to sync. After installing btsync on Raspberry Pi initially and then ending up running it on an Odroid-C1 my WD NAS never sleeps. The first 48 hours it was expected since it took all that time to index all the files. But now my NAS never sleeps. Since this is a backup solution I don't have the need for it to actually re-index every 10 minutes. If the Western Digital LiveBook Duo is left alone for 20 minutes it will spin down it's hard drives. So here is a quick guide on how to configure the re-index interval for btsync installed on an arm system. This is done on the system initially installed with the Raspberry Pi: BitTorrent Sync guide.

Continue reading...

10Mar/160

Raspberry Pi: BitTorrent sync alternative Syncthing

Btsync is a great tool for syncing data and backups to several different locations. By leveraging the torrent technology you can increase the download/upload speeds by adding more nodes to the swarm. Installing Btsync on a built in system like the Raspberry Pi or Odroid-C1 is a challenge since it consumes a lot of system resources, read more about installing BitTorrent Sync on Raspberry Pi. Reading up on the subject a lot of people did suggest that Syncthing takes up less system resources then btsync so let's install it and try it out.

Pre requirements

Same as for the btsync install I need to access my NAS root NFS share to get all the data that I will sync. So first I setup a mount point and do a persistent mount of that share.

sudo mkdir /mnt/stroot
sudo chmod 777 /mnt/stroot

Then let's edit /etc/fstab so run sudo nano /etc/fstab and add a line representing your share.

192.168.0.5:/nfs /mnt/stroot nfs rsize=8192,wsize=8192,timeo=14,nolock,intr 0 0

This will give me the root for all the shares so I don't have to map every share individually but are able to access all of them as subfolders.

To test this out we can run sudo mount -a. Then reboot the system and check that the mount is persistent. If you have issues with the persistence and are running DHCP please see my Raspbian:fstab doesn't mount NFS on boot article for reference.

Install

Download the Syncthing repository key and add it.

wget -O - https://syncthing.net/release-key.txt | sudo apt-key add -

Add the repository

echo "deb http://apt.syncthing.net/ syncthing release" | sudo tee /etc/apt/sources.list.d/syncthing.list

Update the repository with sudo apt-get update and then install with

sudo apt-get install syncthing

Then try to run it with syncthing, as soon as you see it start properly you can stop it with CTRL + C. To set it up to run as a service we need first reconfigure the bind address.

nano /home/pi/.config/syncthing/config.xml

Look for the gui option, it should look like this:

<gui enabled="true" tls="false">
<address>127.0.0.1:8384</address>
<apikey>2Yy41r5k5oocz6rDIWVsKeP4wUx9uknD</apikey>
<theme>dark</theme>
</gui>

Here you can change all the settings in regard to the UI. Change the bind address to 0.0.0.0 so it will respond to all ip addresses. If your running on a fixed IP and not DHCP you can go ahead and bind it to a specific address if you like. You can also change the tls setting to true, enabling https for the UI. This will however result in a security warning when you browse to the UI due to the self signed certificate.

If you now run syncthing from the command line it will bind to the IP and you will be able to navigate to the UI.

Autostart Syncthing as a "service"

First we need to create a init.d script for Syncthing.

sudo nano /etc/init.d/syncthing

Copy and paste the following into it:

#!/bin/sh
### BEGIN INIT INFO
# Provides:          Syncthing
# Required-Start:    $local_fs $remote_fs $network
# Required-Stop:     $local_fs $remote_fs $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Syncthing
# Description:       Syncthing is for backups
### END INIT INFO
 
 
# Documentation available at
# http://refspecs.linuxfoundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/iniscrptfunc.html
# Debian provides some extra functions though
. /lib/lsb/init-functions
 
 
DAEMON_NAME="syncthing"
DAEMON_USER=pi
DAEMON_PATH="/usr/bin/syncthing"
DAEMON_OPTS=""
DAEMON_PWD="${PWD}"
DAEMON_DESC=$(get_lsb_header_val $0 "Short-Description")
DAEMON_PID="/var/run/${DAEMON_NAME}.pid"
DAEMON_NICE=0
DAEMON_LOG='/var/log/syncthing'
 
[ -r "/etc/default/${DAEMON_NAME}" ] &amp;&amp; . "/etc/default/${DAEMON_NAME}"
 
do_start() {
  local result
 
	pidofproc -p "${DAEMON_PID}" "${DAEMON_PATH}" &gt; /dev/null
	if [ $? -eq 0 ]; then
		log_warning_msg "${DAEMON_NAME} is already started"
		result=0
	else
		log_daemon_msg "Starting ${DAEMON_DESC}" "${DAEMON_NAME}"
		touch "${DAEMON_LOG}"
		chown $DAEMON_USER "${DAEMON_LOG}"
		chmod u+rw "${DAEMON_LOG}"
		if [ -z "${DAEMON_USER}" ]; then
			start-stop-daemon --start --quiet --oknodo --background \
				--nicelevel $DAEMON_NICE \
				--chdir "${DAEMON_PWD}" \
				--pidfile "${DAEMON_PID}" --make-pidfile \
				--exec "${DAEMON_PATH}" -- $DAEMON_OPTS
			result=$?
		else
			start-stop-daemon --start --quiet --oknodo --background \
				--nicelevel $DAEMON_NICE \
				--chdir "${DAEMON_PWD}" \
				--pidfile "${DAEMON_PID}" --make-pidfile \
				--chuid "${DAEMON_USER}" \
				--exec "${DAEMON_PATH}" -- $DAEMON_OPTS
			result=$?
		fi
		log_end_msg $result
	fi
	return $result
}
 
do_stop() {
	local result
 
	pidofproc -p "${DAEMON_PID}" "${DAEMON_PATH}" &gt; /dev/null
	if [ $? -ne 0 ]; then
		log_warning_msg "${DAEMON_NAME} is not started"
		result=0
	else
		log_daemon_msg "Stopping ${DAEMON_DESC}" "${DAEMON_NAME}"
		killproc -p "${DAEMON_PID}" "${DAEMON_PATH}"
		result=$?
		log_end_msg $result
		rm "${DAEMON_PID}"
	fi
	return $result
}
 
do_restart() {
	local result
	do_stop
	result=$?
	if [ $result = 0 ]; then
		do_start
		result=$?
	fi
	return $result
}
 
do_status() {
	local result
	status_of_proc -p "${DAEMON_PID}" "${DAEMON_PATH}" "${DAEMON_NAME}"
	result=$?
	return $result
}
 
do_usage() {
	echo $"Usage: $0 {start | stop | restart | status}"
	exit 1
}
 
case "$1" in
start)   do_start;   exit $? ;;
stop)    do_stop;    exit $? ;;
restart) do_restart; exit $? ;;
status)  do_status;  exit $? ;;
*)       do_usage;   exit  1 ;;
esac

Make the init.d script executable and make it start on boot.

sudo chmod +x /etc/init.d/syncthing
sudo update-rc.d syncthing defaults

Now you can start the Syncthing service like any other service.

sudo service syncthing start

Conclusion

The reason I looked into Syncthing in the first place was the memory consumption of btsync. After running a few tests I can see that it's the same with Syncthing. By the time I'm writing this my btsync instance running on my Odroid-C1 has index almost 4Tb of data and is running smoothly. Syncthing is opensource, which I like a lot, but it's a much more immature product then btsync. The windows client is just an .exe that will run a local web server that you connect to. Btsync seems more mature and robust from my initial tests. Syncthing will probably develop into something better but at this time I will go with btsync instead. Btsync is also developed by the torrent technology creators which make me believe in it even more. Install guide for btsync...

Why don't you install both of them and try them out and let me know what you think in the comments?

8Mar/160

Raspberry Pi: BitTorrent Sync

BitTorrent is not only pirated movies and tv-shows it's an actual technology for moving large amounts of data between several nodes at once. There are several applications for this sync technology. Large companies like Facebook use it to sync new releases among all their production servers for example. If three or more nodes are syncing the same content it will increase the sync speed since it's a non centralized technology. All the clients will help to seed the data they already have which will make the sync go faster and faster the more nodes the data propagates to.

Btsync will make use off all the memory available in your Raspberry Pi so I recommend running this on it's own. I have tested this out on both a Raspberry Pi 2 B and the older Raspberry Pi B and they more or less run the same. It will take some time to index all your data but when testing it on a regular windows pc it was pretty slow as well. Continue reading...

16Feb/162

Raspberry PI: Central kodi database

When you run more then one Kodi device with a shared media repository you should also run a shared library. With a shared library you only need to update the library on one of the Kodi devices when new media is added. There are a few other great benefits of running a shared library like the ability to stop media on one device and continue watching on another. It will also show what has been watched regardless of which device you actually watched it on. I run OpenELEC on Raspberry Pi's for all my media stations so this guide is focused on a Raspberry Pi implementation but the principal is the same for all devices capable of running Kodi and MySQL.

Preparations

Download the latest raspian image from https://www.raspberrypi.org/downloads/raspbian/ and put it on an sd-card with Win32DiskImager. Then put the sd-card in the Raspberry Pi and power it on. You can connect a monitor, mouse and keyboard for the initial setup. I usually just wait for it to boot up and check my routers DHCP list for it to show up and connect to it over SSH. This article is based on SSH terminal access but you can do the same on the Pi with a connected monitor.

Basic setup of the Raspberry Pi

First we need to configure the Raspberry Pi. Connect over SSH and login as pi with password raspberry. We want to:

sudo raspi-config
  • Expand Filesystem - Expand the pi file system to use the entire SD-card.
  • Change User Password - Change the default raspberry password for the pi user.
  • Boot Options - Select B1 Console - Text console, requiring user to login
  • Advanced - Some additional settings
    • Hostname - Select a hostname for your pi. I user PISRV
    • Memory Split - Set to 16 since we will not use a monitor or run any other graphics on this machine.
  • Finish - Exit the raspi-config and reboot.

When the pi has booted up again we can login with our new password we configured. When in the console run:

sudo apt-get update

Update the package lists from all the repositories.

sudo apt-get dist-upgrade

Upgrade all installed packages.

Install MySQL

Install MySQL server. It will ask you to select a root password.

sudo apt-get install mysql-server

We need to be able to access the MySQL server from other locations then localhost. Open up the MySQL config file.

sudo nano /etc/mysql/my.cnf

And change:

bind-address = 127.0.0.1

To:

bind-address = 0.0.0.0

Then restart the MySQL server.

sudo /etc/init.d/mysql restart

Setup the databases

Now we need to setup the databases for our video and music library. Login to MySQL, you will be prompted for your password.

mysql -uroot -p

What ever you do don't create any databases. Kodi will use the name that we supply later but just as a prefix, it will add an additional identifier. We need to create a user for the kodi machines. You can use whatever username and password you like but once again make a note of it.

 CREATE USER 'kodi' IDENTIFIED BY 'password';

The first time Kodi connects it needs to be able to create it's databases, we need to grant the account full access. We will lock that down later to secure our MySQL if we want to use it for other things then Kodi as well.

GRANT ALL ON *.* TO 'kodi';

Test to connect to the database from another machine. Either via MySQL command line tool, the same we used on the Pi, or download MySQL Workbench and test the connection. If all is working just type quit and press enter to return to the main shell.

Setup kodi

Then we need to make a backup of our current media library. I'm using OpenELEC but most Kodi versions should be the same. In the Kodi UI goto System/Settings -> Video -> Library and select Export library. If you don't see the last options make sure that Settings Oprions is set to Advanced in the lower left corner of the screen. Select Multiple files, this will create .nfo files along side all the media files. I already use this setup since I scan all my libraries With local information only. This is by far the safest way to migrate your library otherwise you have to scrape all the media again. So if you use local information only on your scrapes your good to go. Otherwise do the export!

Now we need to setup Kodi to use the MySQL server. Connect to the Kodi with SSH, username and password depends on the distribution you used to install your Kodi. The location of the userdata folder also varies from different distributions. I have one XBMC installed on a Raspberry Pi where the path is /home/pi/.kodi/userdata. On the OpenELEC installs I'm doing this for the userdata folder is located in /storage/.kodi/userdata/.

When you have found the folder you need to create a file named advancedsettings.xml. You can also do this via smb share if that is enabled on your Kodi machine. I prefer to do it over SSH to prevent any encoding issues. If you want to you can try it out by going \\{ip of kodi}\userdata.

sudo advancedsettings.xml

In this file we will put the configuration for accessing our MySQL server.

<advancedsettings>
  <videodatabase>
    <type>mysql</type>
    <host>{IP address or FQDN of your MySQL server}</host>
    <port>3306</port>
    <name>{prefix of your db name, I used kodi_video}</name>
    <user>kodi</user>
    <pass>{password you selected}</pass>
  </videodatabase> 
  <musicdatabase>
    <type>mysql</type>
    <host>{IP address or FQDN of your MySQL server}</host>
    <port>3306</port>
    <name>{prefix of your db name, I used kodi_music}</name>
    <user>kodi>/user>
    <pass>{password you selected}</pass>
  </musicdatabase>
  <videolibrary>
    <importwatchedstate>true</importwatchedstate>
    <importresumepoint>true</importresumepoint>
  </videolibrary>
</advancedsettings>

Save the file and reboot your Kodi machine. Depending on distribution you may need to do sudo reboot. Once it recycles you can scan your locations again, it will use the local .nfo files you already had or the once created during your export.

Securing MySQL again

We don't want the kodi MySQL user to have full access going forward. If your not using your MySQL for anything else then you can leave it be, but I want to secure mine. So back to the SSH console on the MySQL server.

mysql -uroot -p

You will once again be prompted for your MySQL root password and then dropped into the MySQL console. So now we check the name of the databases.

mysql&amp;amp;amp;gt; show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| kodi_music52 |
| kodi_video93 |
| mysql |
| performance_schema |
+--------------------+
5 rows in set (0.00 sec)

So we have the kodi_ databases named after the <name> parameter in our advancedsettings.xml. Now we revoke all the access we gave the kodi account.

REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'kodi';

And then only grant it full access to the two kodi_ databases.

GRANT ALL ON `kodi%`.* TO 'kodi';
flush privileges;

Setting up the other Kodi

Once all this is up and running setting up the second one is really easy. If you enabled SMB shares on OpenELEC like I did you just connect to the first kodi like \\kodi\UserData copy the advancedsettings.xml and sources.xml to the other kodies UserData share. If you don't have SMB enabled you have to edit the files via SSH.

 

17Sep/150

Win32 Disk Imager

Reading and writing images to SD cards made easy! I more or less us it every day to write images to SD cards for my Raspberry Pi projects or for doing a backup of them. Win32 Disk Imager has received some bad press because it some times breaks SD cards. Every time that happen to me it was the image that was bad, so I can not really agree with the bad comments. If the card becomes unreadable it's easily fixed with SD Formatter. Win32 Disk Imager can be downloaded from SourceForge!

15Sep/150

Raspberry Pi as a torrent server

A Raspberry Pi is a great for creating an always on torrent box that can take care of all your downloading and seeding. If you combine it with a NAS and a Raspberry Pi Kodi media center you will have a really sweat setup. The Raspberry Pi has a low power consumption, I run my of the USB port on my NAS. It also have no fans so it's quiet! In this guide we setup Transmission on a Raspberry Pi which includes both a web gui and third party apps for IOS and Android.

I presume that you have some basic knowledge of Linux and the Raspberry Pi. If not you might need to check out the installation guide for Raspberry Pi. When you have your Raspberry Pi up and running just follow the guide below. Use an image and not NOOBs it will come back and haunt you!
Continue reading...

15Sep/155

Raspbian: fstab doesn’t mount NFS on boot

Ran out of disc space in one of my Raspberry Pi projects last night. Of course I did a quick and dirty install with NOOBs so cloning to a larger SD-card felt like a drag. So I decided it was time to upgrade from a 4GB SD to a 16GB SD as well as the latest version  4.1.6+. Installation went like a charm until I went to edit my /ect/fstab. I added the same NFS line as I used before:

192.168.0.5:/nfs/Download /mnt/download nfs rsize=8192,wsize=8192,timeo=14,intr 0 0

sudo mount -a work just fine but the share wasn't mounted after reboot. Googled the issue and found a lot of different suggestions, many related to USB drives. The number one suggestion was adding rootdelay=10 or rootdelay=5 to /boot/cmdline.txt. That would probably solve the issue for USB drives because the system are unable to identify the drive that early in the boot. Same suggestion was given for NFS failures as well but will not work. Tried a lot of suggestions, even found scripts to run mount -a after boot. That is not a solution just a work around!

Suggestion for adding x-systemd.automount,noauto to the mount options failed as well. Tried a lot of different configurations with one thing in common, no error in /var/log/syslog.

Finally I realized that the network was not ready! I checked the /etc/network/interfaces settings for eth0.

iface eth0 inet manual

It will still get a DHCP address but that will happen later in the boot process. So when the fstab entries are processed there is no network connection and therefore the disc will not mount. So if you change it to:

iface eth0 inet dhcp

Then the NFS drive will mount just fine after a reboot.