How to Generalize a Linux VM Template

When building out an environment of any kind, you need to have a good starting base, a strong foundation and the same holds true for many things. It’s no different when building a virtual machine (VM) template to deploy in your environment in a repeatable fashion.

Recently I spent some time developing a generalized/sanitized VM template for Red Hat Enterprise Linux (RHEL) versions 6 and 7. This script should be run right before you shutdown the VM and template it for reuse.

#!/bin/bash
echo "Generalize the template..."
echo "Remove RHN system ID..."
rm -f /etc/sysconfig/rhn/systemid

echo "Create sanitized ifcfg-eth0..."
echo -en 'DEVICE=eth0\nTYPE=Ethernet\nONBOOT=yes\nBOOTPROTO=dhcp\n' > /etc/sysconfig/network-scripts/ifcfg-eth0

echo "Clear /etc/sysconfig/network file..."
cat > /etc/sysconfig/network < /dev/null

echo "Remove SHH keys..."
rm -f /etc/ssh/ssh_host_*

echo "Remove udev rules..."
rm -f /etc/udev/rules.d/70-*

echo "Remove fixed hostname..."
rm -f /etc/hostname

echo "Clear Machine ID (SID)..."
> /etc/machine-id

echo "Remove all logs from /var/log..."
rm -rf /var/log/*

echo "Remove all logs from /root..."
rm -rf /root/*

Debian 8 Jessie sources.list, missing installation sources.

debian8_with_release_date

Each time I install Debian Jessie or Debian 8 I notice that I can not install any software packages afterwards. I’m not sure if it is me, or the installation media, but the sources.list is missing entries. Recently I tried installing xrdp, which allows Windows OS machines to RDP into Linux OS machines.

Commands such as…

apt-get update
apt-get upgrade
apt-get install package_name

… yielded no results. When I navigate to /etc/apt/sources.list I noticed there were only 3 entries in the file.

deb cdrom:[Debian GNU/Linux....

deb http://security.debian.org/ jessie/updates main contrib
deb-src http://security.debian.org/ jessie/updates main contrib

However the Debian Wiki specifies two additional sources.

deb http://httpredir.debian.org/debian jessie main
deb-src http://httpredir.debian.org/debian jessie main

deb http://httpredir.debian.org/debian jessie-updates main
deb-src http://httpredir.debian.org/debian jessie-updates main

Once you add these to your sources list and run the above commands the
packages should update, and you can proceed with installing XRDP.

Install and Configure OpenVPN on OSMC/Kodi

Let’s face it, Kodi is pretty popular right now, everyone is talking about it. One of the first things I did after I installed OSMC on my Raspberry Pi was to configure OpenVPN. There is a little bit of work involved so I figured I’d share what I did to get it up and running!

Login to OSMC via SSH using PuTTy or your client of choice.

Elevate to Super User.
osmc@KODI:~$ sudo su

Update the software repositories.
root@KODI:/home/osmc# apt-get update

Install OpenVPN.
root@KODI:/home/osmc# apt-get install openvpn

Reboot.
root@KODI:/home/osmc# reboot

Create a folder to put your OpenVPN configuration files in.
osmc@KODI:~$ sudo su
root@KODI:/home/osmc# mkdir vpn-conf

Copy your .opvn file/files and your .crt file into /home/osmc/vpn-conf, there are a few ways to copy the files here, I personally like to use PSCP. This
example is using PSCP from a Windows computer.
C:\temp>pscp c:\temp\ca.crt osmc@192.168.1.100:/home/osmc/vpn-conf

Create a new file that will contain your login credentials for OpenVPN
root@KODI:/home/osmc# cd vpn-conf
root@KODI:/home/osmc/vpn-conf# vi login.conf

  • Press Insert
  • Type your Username on first line press enter and type your password on the next
    line.
  • Press Esc, type :wq
Username
Password

Now edit the .opvn file/files of choice to make sure the login.conf and <ca_file_name>.crt file are referenced correctly.
root@KODI:/home/osmc/vpn-conf# vi <filename>.ovpn

  • Find the following lines that begin with:
    • auth-user-pass
    • ca
  • If they exist – edit them accordingly, if they don’t exist you will need to add them.
auth-user-pass /home/osmc/vpn-conf/login.conf
ca /home/osmc/vpn-conf/ca.crt

Let’s test out OpenVPN, the service should start and connect successfully after running this command.
root@KODI:/home/osmc/vpn-conf# openvpn /home/osmc/vpn-conf/<filename>.ovpn

Confirm VPN connectivity by using curl, this should retrieve your VPN’d IP address.
root@KODI:/home/osmc/vpn-conf# curl http://checkip.dyndns.org

If everything checks out and is working so far it’s time to install the OpenVPN Add-On for
Kodi and import a profile. Grab the latest OpenVPN Add-On for Kodi, the quickest way is to just grab it is to use wget right from OSMC, using your web browser right-click on the script.openvpn-x.x.x.zip and Copy the link address.
root@KODI:/home/osmc/vpn-conf# wget -c <paste_link_addr_here>

From Kodi on your TV, go to Settings -> Add-ons -> Install from zip file

  • Select Install from zip file.
  • Navigate to the ZIP file and select it.
  • In the bottom right corner, Kodi notifies when the add-on is installed and enabled.

Now the OpenVPN Add-On for Kodi should be installed… go to Programs > Add-Ons
> OpenVPN from Kodi and import your .opvn files, once complete try to connect. It should work successfully since it’s literally just an interface to the actual OpenVPN service that we just installed and configured.

That’s it! …but if you’d like to take it one step further you can. I personally like to have a certain OpenVPN profile connect at startup of Kodi. It’s pretty simple to do this.

Browse to the userdata folder for Kodi and create an autoexec.py file.
root@KODI:/home/osmc# cd /home/osmc/.kodi/userdata
root@KODI:/home/osmc/.kodi/userdata# vi autoexec.py

  • Type the following where <profile_name> is the profile
    name of the profile you have created in the OpenVPN Add-On for Kodi.
import xbmc
xbmc.executebuiltin('XBMC.RunScript(script.openvpn,<profile_name>)')

Now every time you power on your Kodi box, OpenVPN will launch and the VPN
profile of choice will connect automatically. Cheers and happy streaming!

Code::Blocks in Linux Mint 17.2 and Ubuntu 14.04

codeblocks

Recently I ran into a problem finding a solution for properly installing Code::Blocks on a Linux Mint 17.2 Cinnamon machine. I had the option to opt in and run the application on a Windows lab machine, but I said nah, I want to expose myself more to the linux environment and I want to be able to use my own laptop. So I went ahead and opted to install Code::Blocks on my Linux Mint 17.2 laptop.

After much searching I came across an obscure reference in a forum post to the problem. What was happening is that when I included printf() and scanf() in my simple terminal application it wouldn’t compile or run, the compiler complained about an assumed reference. Code Blocks would also generate errors in regards to stdio.h and that it couldn’t find the file. After further reading the solution is part of the normal compiler/code blocks installation. I was not aware, I’ve never actually compiled code on linux before. Here is what I did to resolve it.

Start by checking that your gcc is up to date. You might not need to but it is always good to run the install as it will update your gcc if it needs to.

sudo apt-get update
sudo apt-get install gcc

The above if up to date, will skip over the installation and tell you that you already have the latest packages installed.

Now go ahead and install Code::Blocks.

sudo apt-get install codeblocks

This will install the application on your computer and you will be able to run it and create projects. However this is where I initially ran into an issue where if I Included stdio.h in the source it would generate an error. Something along the lines of…

… stdio.h: No such file or directory …

There are two things that you need to do to resolve this. You need to install build-essentials and xterm.

sudo apt-get install build-essentials

The above will install the dependencies such as the stdio.h file, however if you go to compile a simple application and run it in command line or bash you will get another error. Generally an XTerm error. You will need to also install XTerm in order for Code::Blocks to be able to open up a terminal window and execute the application.

sudo apt-get install xterm

Once you have all the above packages installed give it a go and you should be able to execute and run simple c/c++ code.

HTPC? Go with Linux, try and avoid Windows based HTPCs.

A year and a half ago I switched my media server from Windows to Linux, this opened my eyes. A lot of people are afraid of Linux, people are afraid of what they don’t understand. I was a bit skeptical at first, but having made the switch I would never go back to a Windows based machine. Also I wanted to state that this article is not put here to start a flame war. I’m a heavy Windows user, these are just some observations of mine.

Main_logo_3DMy server serves up all the media from 7 Hard Drives totalling about 14 Terabytes of storage. I use XBMC to view all that media in my home. For those of you not familiar with XBMC, first I must ask, “Where have you been?”. XBMC is an award-winning free and open source (GPL) software media player and entertainment hub that can be installed on Linux, OSX, Windows, iOS, and Android, featuring a 10-foot user interface for use with televisions and remote controls. It allows users to play and view most videos, music, podcasts, and other digital media files from local and network storage media and the internet. That bit is straight from the horses mouth. It is probably one of the most popular media players for home theatre enthusiasts. If you have multiple televisions in your home you can point XBMC to an SQL database and sync content across rooms. If you are half way into a movie and you decide to stop it, you can continue where you left off in another room. It’s a very feature filled and rich media centre environment.

In my main living room where I have my home theatre setup, I’ve been running XBMC on top of a Windows machine for quite some time now. It has been very unstable, and more often than not if I brought the PC out of suspended mode XBMC would crash to desktop. My main reason for running this on top of Windows was that I also wanted to browse the web while in the living room. Finally I got annoyed at all the crashes and hangups with Windows. I could have ran in on top of Ubuntu, but instead I opted for an embedded XBMC solution. I had a choice either XBMCbuntu or OpenElec. I decided to go with OpenElec, to me it seemed like to more logical choice of the two. OpenElec is XBMC built from the ground up, on top of a Linux kernel. It is designed for light and small embedded media players. XBMCbuntu is an XBMC distro built on top of the Ubuntu kernel. I wanted the one with less bloatware. OpenElec stands for open embedded Linux entertainment center.

openelec_logoOnce I installed OpenElec and setup all my sources in XBMC I moved on to the testing phase. I wanted to test a couple movies. I ran a test on a movie i recently watched on my Windows based XBMC. Same hardware but different software. I played the movie and immediately I noticed the sound was a little louder, so i dropped the volume a couple notches, and continued watching. Meanwhile this is all playing through a 5.1 surround sound setup, and DTS is being decoded on the receiver. While listening I noticed something nice but not that surprising. The dynamic ranges in the movie were clearer and being reproduced a little better. The new XBMC Frodo (v12) Audio Engine (AE) and the linux audio stack was handling the audio a lot better than the Windows equivalent. Then I immediately recalled an article I read about 6 months ago where an audio engineer discusses the merits and accuracy of the Linux audio stack over the competition such as Windows. I wish I could link to the article, it was an interesting read. I also recalled that Windows audio stack or API as of the Vista version was no longer controlled by the hardware manufacturers but by the kernel of the OS. Some say that since Vista you don’t have true hardware DirectSound, instead you have software WASAPI which does mandatory resampling and frequency cutoff, and doesn’t allow any hardware effects such as CMSS-3D. This lead to some back lash from a lot of people back then, both hardware manufacturers and end users. In the end the audio experience is just as important to me as the video experience. I still buy Blue Rays because of the uncompressed video and audio quality, after all a movie should be an audio visual experience.

linux-wallpaperIn the end I’m glad I switched the HTPC to Linux. There is a noticeable performance increase on the audio side and just a general performance increase due to the smaller OS overhead from Linux and OpenElec. As for the web browsing part on my HTPC that I mentioned earlier, I found a third party Opera Browser plugin for OpenElec. Also as a side note, if you’re using a Raspberry Pi for XBMC, there is a huge performance increase with OpenElec over Raspbmc.

Here is a little clarification of the Windows audio subsystem taken from the XBMC Wiki:

Since Vista SP1, Windows has two primary audio interfaces, DirectSound and Wasapi (Windows Audio Session Application Programming Interface). The latter was a replacement for XP’s Kernal Streaming mode.

DirectSound acts as a program-friendly middle layer between the program and the audio driver, which in turn speaks to the audio hardware. With DS, Windows controls the sample rate, channel layout and other details of the audio stream. Every program using sound passes it’s data to DS, which then resamples as required so it can mix audio streams from any program together with system sounds.

The advantages are that programs don’t need resampling code or other complexities, and any program can play sounds at the same time as others, or the same time as system sounds, because they are all mixed to one format.

The disadvantages are that other programs can play at the same time, and that a program’s output gets mixed to whatever the system’s settings are. This means the program cannnot control the sampling rate, channel count, format, etc. Even more important for this thread is that you cannot pass through encoded formats, as DS will not decode them and it would otherwise bit-mangle them, and there is a loss of sonic quality involved in the mixing and resampling.

Partly to allow for cleaner, uncompromised or encoded audio, and for low-latency requirements like mixing and recording, MS re-vamped their Kernal Streaming mode from XP and came up with WASAPI.

WASAPI itself has two modes, shared and exclusive. Shared mode is in many ways similar to DS, so I won’t cover it here.

WASAPI exclusive mode bypasses the mixing/resampling layers of DS, and allows the application to negotiate directly with the audio driver what format it wishes to present the data in. This often involves some back-and-forth depending on the format specified and the device’s capabilities. Once a format is agreed upon, the application decides how it will present the data stream.

The normal manner is in push mode – a buffer is created which the audio device draws from, and the application pushes as much data in as it can to keep that buffer full. To do this it must constantly monitor the levels in the buffer, with short “sleeps” in between to allow other threads to run.

WASAPI, and most modern sound devices, also support a “pull” or “event-driven” mode. In this mode two buffers are used. The application gives the audio driver a call-back address or function, fills one buffer and starts playback, then goes off to do other processing. It can forget about the data stream for a while. Whenever one of the two buffers is empty, the audio driver “calls you back”, and gives you the address of the empty buffer. You fill this and go your way again. Between the two buffers there is a ping-pong action: one is in use and draining, the other is full and ready. As soon as the first is emptied the buffers are switched, and you are called upon to fill the empty one. So audio data is being “pulled” from the application by the audio driver, as opposed to “pushed” by the application.

WASAPI data is passed-through as-is, which is why you must negotiate capabilities with the audio driver (i.e. it must be compatible with the format you want to send it as there is no DS between to convert it), and why encoded formats like DTS can reach the receiver unchanged for decoding there.

Because WASAPI performs no mixing or resampling, it is best used in the exclusive mode, and as a result the application gets the exclusive rights to the audio buffers, to the exclusion of all other sounds or players. WASAPI shared mode does allow this, but that’s not a common mode and not what we want for an HTPC. I myself have a dislike of Window’s cutesy system sounds happening at 110db

Hope some of you found today’s primer of use. Please pick up a scorecard from the desk and drop it in the big round “collection box” on your way out

Cheers, Damian