How to Generalize a Linux VM Template

When building out an environment of any kind, you need to have a good starting base, a strong foundation and the same holds true for many things. It’s no different when building a virtual machine (VM) template to deploy in your environment in a repeatable fashion.

Recently I spent some time developing a generalized/sanitized VM template for Red Hat Enterprise Linux (RHEL) versions 6 and 7. This script should be run right before you shutdown the VM and template it for reuse.

#!/bin/bash
echo "Generalize the template..."
echo "Remove RHN system ID..."
rm -f /etc/sysconfig/rhn/systemid

echo "Create sanitized ifcfg-eth0..."
echo -en 'DEVICE=eth0\nTYPE=Ethernet\nONBOOT=yes\nBOOTPROTO=dhcp\n' > /etc/sysconfig/network-scripts/ifcfg-eth0

echo "Clear /etc/sysconfig/network file..."
cat > /etc/sysconfig/network < /dev/null

echo "Remove SHH keys..."
rm -f /etc/ssh/ssh_host_*

echo "Remove udev rules..."
rm -f /etc/udev/rules.d/70-*

echo "Remove fixed hostname..."
rm -f /etc/hostname

echo "Clear Machine ID (SID)..."
> /etc/machine-id

echo "Remove all logs from /var/log..."
rm -rf /var/log/*

echo "Remove all logs from /root..."
rm -rf /root/*

Hyper-V replication in a workgroup or across domains using a self signed certificate.

Why would you want a HyperV server in a workgroup environment?

Well if your Domain Controller is a VM you really don’t want to add the HyperV server to the domain as it will boot before the DC comes up. This type of setup is ripe for domain issues, so we’re left with a server that is only in a workgroup. Also if you are doing cross site replication, you might be replicating from/to different domains, this is where the self signed certificate authentication comes in to play as it is domain agnostic.

Kerberos authentication does not work in this setup, so we need to use a certificate authority as a means of authenticating the two servers with each other. The Primary server is where all the VMs are, and the Replica server is where the VMs will be copied to. HyperV replication is native and built into Server 2012 +, so there are no extra licenses necessary.

What are the steps involved?:

  1. Change the DNS suffix on both Primary and Replica servers.
  2. Reboot both servers.
  3. Create self signed certificates on both servers.
  4. Open the Certificate MMC snap-in on the Primary server and export the certificate to a .pfx file.
  5. Copy the export file and RootCA certificate from the Primary to the Replica server.
  6. Import the Primary RootCA certificate file on the Replica server.
  7. Import the .pfx file on the Replica server.
  8. Copy the RootCA certificate from the Replica to the Primary server and import it.
  9. Disable Certificate Revocation Check on both servers for replication and fail over replication.
  10. Setup the Replica server as a replica in HyperV.
  11. Start replication of a Server on the Primary server.

First we need to change the server names, or rather add a DNS suffix to them. Bring up System Properties in the Control Panel, under the Computer Name tab click change. In the Computer Name/Domain Changes window click More…

In the DNS Suffix and NetBIOS Computer Name add a Primary DNS suffix. Something along the lines of “hypervreplica.local”, it doesn’t matter call it what you will.

Click OK and save all the changes. Note that you will be required to reboot the server in order for changes to take effect. Do this to both the Primary and Replica server.

Primary is the server where your VMs reside, and Replica is where your VMs will be replicated or copied to.

Next we need to create a self signed certificate. For this you will either need Visual Studio or the Windwos SDK(https://www.microsoft.com/en-us/download/details.aspx?id=8442).

What we really need out of either of these is the makecert.exe file.

If you have VS installed the makecert.exe file is located under C:\Program Files (x86)\Windows Kits\8.1\bin\x64, or a similar path, the 8.1 will change depending on the version of Visual Studio you have installed.

Copy the makecert.exe file from here to the primary and the replica servers.

On both those servers create an empty directory somewhere, place the makecert file in there. This is also where we will create and store the self signed certificates.

On the Replica server open up an elevated command prompt and navigate to the directory where the “mekecert.exe” file is located and type in the following:

makecert -pe -n “CN=ReplicaRootCA” -ss root -sr LocalMachine -sky signature -r “ReplicaRootCA.cer”

The above command assigns a signature certificate issuer name to the replica server of “ReplicaRootCa”

Followed by:

makecert -pe -n “CN=replicahostname” -ss my -sr LocalMachine -sky exchange -eku 1.3.6.1.5.5.7.3.1,1.3.6.1.5.5.7.3.2 -in “ReplicaRootCA” -is root -ir LocalMachine -sp “Microsoft RSA SChannel Cryptographic Provider” -sy 12 ReplicaCert.cer

…where the replicahostname is replaced by the name of the server with the DNS suffix and all. Ex. hostname.domain.local.

Now move over to the Primary server, open up an elevated command prompt and navigate over to the folder where “makecert.exe” is located, and type the following:

makecert -pe -n “CN=PrimaryRootCA” -ss root -sr LocalMachine -sky signature -r “PrimaryRootCA.cer”

Followed by:

makecert -pe -n “CN=primaryhostname” -ss my -sr LocalMachine -sky exchange -eku 1.3.6.1.5.5.7.3.1,1.3.6.1.5.5.7.3.2 -in “PrimaryRootCA” -is root -ir LocalMachine -sp “Microsoft RSA SChannel Cryptographic Provider” -sy 12 PrimaryCert.cer

… where the primaryhostname reflects the name of the Primary server with the added DNS suffix. The above 4 commands will create two files on each server.

On the Primary server run > mmc, click File and select Add/Remove Snap-in…

Select the Certificates snap in and click Add>, on the next windows select Computer Account, click Next>, and then select Local computer:. Click Finish.

In the Certificates snap in, expand the Personal store and then click on Certificates.

 

You should have a certificate here with the Replica serve name and Issued by ReplicaRootCa.

Right click this certificate, select All Tasks and select Export…

Capture7

This will open the Certificate Export Wizard, when prompted select Yes, export the private key.

Export File Format, use Personal Information Exchange…. (.PFX), Include all certificates and the certification path if possible.

On the Security page check the password box, and input a password you will remember.

Click the Browse button to save the export in a *.pfx file format, give it a file name (PrimaryServer.pfx) and click save.

Double check all your settings on the final page and click Finish.

Copy the PrimaryRootCA.cer file and the PrimaryServer.pfx files to the Replica server. Put it in the folder where you created your Replica Server certificates.

On the Replica server we will now import the cer and pfx files. Open up an elevated command prompt and navigate to the file location. Type in the follwing:

certutil -addstore -f Root “PrimaryRootCA.cer”

The quotes are only necessary if you have spaces or special characters in the file name.

Open up MMC, expand the Personal section, right click on Certificates and select All Tasks > Import.

The Certificate Import Wizard will open up. Click Next. On the File to Import page click Browse…

You might have to change the file type to view the pfx file.

Navigate over to the location of your PrimaryServer.pfx file and select it, click Open. Click Next. On the next screen enter the password for the Private Key. You can mark the key as exportable if you’d like, this means you can export it at a later time if you do not keep a copy of it somewhere. Also check off Include all extended properties. Click Next.

Place the certificate in the Personal certificate store. Click Next. On the final page inspect all the details and make they are correct. Finally click Finish.

*Please be aware that for fail over replication you will more than likely need to export the certificate in the pfx format from both servers and then copy them over and import them on both servers as well. The reason for this is that replication is only one way, where as fail over replication goes both ways. Something to think about.

Now copy the ReplicaRootCA.cer file over to the primary server, place it in the folder with all the other certificate files. In and elevated command prompt add it to the certificate store.

certutil -addstore -f Root “ReplicaRootCA.cer”

Run the following two commands on both servers in an elevated command prompt.

reg add “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization\FailoverReplication” /v DisableCertRevocationCheck /d 1 /t REG_DWORD /f

reg add “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization\Replication” /v DisableCertRevocationCheck /d 1 /t REG_DWORD /f

Note that the only difference between the two registry commands is FailoverReplication and Replication. You shouldn’t need to restart either of the servers after these commands.

Next you need to enable the Replica server as the replica.

On the Replica server open the HyperV manager.

Select HyperV Replication Configuration, in the configuration window check off Enable this computer as a Replica server. Check off Use certificate-based Authentication (HTTPS). It should prompt you to select a certificate, if not click on Select Certificate… you should have the option to select the certificate you created on the Replica server. Select it and click Apply.

*Note, that you will get a prompt about the Windows Firewall, I have mine disabled on all servers so this message never applied to my setup. However if you have your fire wall turned on I would recommend adding a rule to it to allow the traffic on port 443 to pass.

Now the real test, go to your Primary HyperV server and attempt to enable replication. Open up HyperV manager, select a virtual machine, right click it and slect Enable Replication…

You will be presented with a Before You Begin page, click Next >.

On the Specify Replica Server page, type in the FQDN of the replica server, for example, replicahostname.domain.local, or whatever the hostname and dns suffix that you assigned to your replica server is. Click Next >.

On the Specify Connection Parameters, make sure that certificate-based authentication is selected. Kerberos authentication only works on a domain. You may need to select the proper certificate, this will be the Primary server certificate. Also check off Compress the data that is transmitted over the network. If you see a yellow exclamation sign with the text “Could not get configuration details of the specified server.” at the bottom, don’t worry about it, if everything is setup properly it should not impact the replication in any way, shape, or form. Click Next >.

Choose Replication VHDs, here you can pick and choose which storage attached to the server you want to replicate. Select the storage you want and click Next >.

Configure Replication Frequency. The options here are 30 seconds, 5 minutes, or 15 minutes. Depending on how mission critical your data is choose accordingly. Note that replication frequency differs from Server 2012 to Server 2012 R2.

Configure Additional Recovery Points. Depending on how many recovery points you require here is where you set that up. You can setup additional hourly recovery points and even use VSS for snapshots. Hourly recovery points provide granularity, no only can you recover form the last replication point, but with this option enabled you can go back hours. You also have the option of VSS snapshots which, from personal experience, can fail. I don’t have experience with VSS on replication, but VSS on backups and more often than not VSS was always the culprit for failed backups. VSS has a tendency to fail, not ofter but every once in a while. Either way I usually only maintain the latest recovery point. Again the number of recovery points differs from 2012 to 2012 R2, 15 vs 24.

Pick your poison and click Next >.

Choose Initial Replication Method. These options are self explanatory. Chose your replication method and when. I usually just send it over the network, I find that the impact is minimal. You can also start the initial replication at a defined time, perhaps when your system is not as busy at night etc.

 

*One thing to note about replication and this is important, replication creates an avhdx file. This is a HyperV change file, and during the initial replication this can grow quite large in size. On a normal active system I have observed that this file can grow to 33% size of the original VHDX/VHD file. So be careful and be warned, because if the storage medium runs out of space it will pause the VM.

Click Next >. Confirm your settings and click Finish. Your replication should now begin.