Setting up Razor on CentOS 7


This guide will walk you through the steps to setup Razor a next generation provisioning solution for bare metal and virtual servers. Razor puts an API on top of bare metal and virtual server provisioning. Razor makes it extremely easy to provision one, two or two hundred servers very quickly. The software was co-developed between EMC and Puppet Labs. The entire Razor project was open sourced and freely available to anyone who wants to use the software.

This tutorial was written using CentOS 7

STEP 1. Install Postgres
To setup Razor you will first need the Postgres SQL database installed. Postgres is used to stored facts about the nodes being provisioned through razor.

Install the Postgres database server and initialize the databases

$ sudo yum install postgresql-server postgresql-contrib

$ sudo postgresql-setup initdb

Configure Postgres to allow remote access by adding the Subnet or IP address of the razor server connecting to Postgres

$ sudo vim /var/lib/pgsql/data/pg_hba.conf
host     all      all     trust

$ sudo vim /var/lib/pgsql/data/postgresql.conf
listen_addresses = '*'
$ sudo systemctl start postgresql
su - postgres
CREATE USER razoruser WITH PASSWORD 'password';

CREATE DATABASE razor_prd OWNER razoruser;

# to exit postgres cli

STEP 2. Install Razor Server and Client

$ sudo yum install
$ sudo yum install razor-server

Install the Razor client

$ sudo yum install ruby
$ sudo gem install razor-client

STEP 3. Configure Razor

Configure the razor server database connection string by editing the Razor config file /etc/razor/config.yaml

  database_url: 'jdbc:postgresql:razor_prd?user=razoruser&password=password'

Start the razor server and verify connectivity to the Razor server

$ sudo systemctl start razor-server
$ razor -u http://localhost:8150/api -v

STEP 4. Download and extract the Razor microkernel

$ cd /var/lib/razor/repo-store/
$ wget
$ tar -xvf ./razor-microkernel-latest.tar

STEP 5. Install the tftp server

yum install tftp tftp-server xinetd
cd /var/lib/tftpboot/
service xinetd start
service xinetd status
chmod 664 /var/lib/tftpboot/*

vim /etc/xinetd.d/tftp
# configure disable to no
disable = no

Install the ipxe boot

cd /var/lib/tftpboot
chmod 644 ./undionly.kpxe

Save the Razor iPXE bootstrap script as bootstrap.ipxe

cd /var/lib/tftpboot
curl > ./bootstrap.ipxe
chmod 644 ./bootstrap.ipxe

STEP 6. Install the CentOS ISO

This is the ISO Which we will install when we PXE boot servers with Razor
Download the CentOS iso

yum install nginx
systemctl start nginx
systemctl enable nginx

cd /usr/share/nginx/html/

chmod 644 ./CentOS-7-x86_64-DVD-1511.iso

# Verify the ISO can be downloaded
wget http://localhost/CentOS-7-x86_64-DVD-1511.iso

STEP 7. Configure the DHCP server on your network so clients can PXE boot.

Getting the PXE boot to work can be a bit tricky. You must define an iPXE class in the DHCP server.

Create a Policy / Class

Configure the DHCP Scope Options for the iPXE class

When you are complete conifguring the iPXE DHCP class your scope options should look like this.

STEP 8. Use the Razor Client to create a CentOS 7 Repo

$ razor create-repo --name centos70 \
--iso-url \
--task centos/7

STEP 9. Use the Razor Client to create a tag. In this example tag I create a very specific tag which specifies if a node boots on the network with this MAC address then install CentOS 7.

$ razor create-tag --name node01 --rule '["in", ["fact", "macaddress"], "00:0c:29:49:92:71"]'

Here is an example tag which includes two MAC addresses

razor create-tag --name twoNodes --force --rule '["in", ["fact", "macaddress"], "00:0c:29:49:92:71", "00:0c:29:6a:d1:74"]'

STEP 10. Define you policy

$ vim policy.json
"name": "centos-for-small",
"repo": "centos70",
"task": "centos/7",
"broker": "noop",
"enabled": true,
"hostname": "host${id}",
"root_password": "password",
"max_count": 20,
"tags": ["node01"]

STEP 11. Create the Policy

razor create-policy --json policy.json

STEP 12.
At this point you should be able to boot the server with the specified MAC address above and CentOS should begin installation.

Allow remote Postgres connections

Install Postgres on CentOS 7

Install Razor

Setup Mesos-DNS

2016-01-31 21_18_49-MarathonOver the last month I have been evaluating container clustering software. I started with Kubernetes, Rancher which uses swarm and Mesos. I am going through these evaluations to determine which container clustering software will fit my employer’s needs best.

ENVIRONMENT CENTOS 7.0 running three Mesos masters and two Mesos slaves

services: zookeeper, marathon, mesos-master

services: zookeeper, marathon, mesos-master

services: zookeeper, marathon, mesos-master

services: mesos-slave

services: mesos-slave

STEP 1. Prerequisites install golang and git

$ yum install go git
$ export GOPATH=$HOME/go
$ export PATH=$PATH:$GOPATH/bin
$ go get

$ go get
$ go get
$ go get

STEP 2. Clone the mesos-dns repository and build the mesos-dns binary.

$ git clone
$ cd ./mesos-dns
$ go build -o mesos-dns

After building mesos-dns you should have a mesos-dns binary file in your
./mesos-dns directory

STEP 3. In the ./mesos-dns directory there is a config.json.sample example file.
Copy this file and edit it for your own environment.

$ cp config.json.sample config.json

ThisĀ link describes the each of the fields in the config.json file.

  "zk": "zk://,,",
  "masters": ["","",""],
  "stateTimeoutSeconds": 300,
  "refreshSeconds": 60,
  "ttl": 60,
  "domain": "mesos",
  "ns": "ns1",
  "port": 53,
  "resolvers": [""],
  "timeout": 5,
  "listener": "",
  "SOAMname": "root.ns1.mesos",
  "SOARname": "ns1.mesos",
  "SOARefresh": 60,
  "SOARetry":   600,
  "SOAExpire":  86400,
  "SOAMinttl": 60,
  "dnson": true,
  "httpon": true,
  "httpport": 8123,
  "externalon": true,
  "recurseon": true,
  "IPSources": ["mesos", "host"],
  "EnforceRFC952": false

STEP 4. Run the mesos-dns with the config.json file to verify it is properly formatted.

$ ./mesos-dns -config=config.json
On the mesos slave create a directory for the config.json file.
I have designated as the mesos-dns server for my
$ mkdir /etc/mesos-dns

STEP 5. Copy the mesos-dns binary to the mesos slave which you have designated as the mesos-dns server. In this example I copy the mesos-dns service to mesos slave mesos04.

$ scp ./mesos-dns/mesos-dns

STEP 6. Configure the constraints for the mesos-dns service. This essentially tells the marathon to constrain the mesos-dns service to host For example, you may want to designate two nodes in your cluster to run mesos-dns. The constrains directive ensures that mesos-dns does not try to run on other hosts.


STEP 7. Update the network-script file with IP address of the host running mesos-dns.

$ vim /etc/sysconfig/network-scripts/ifcfg-ens160

STEP 8. After updating the network-script file restart the network service

systemctl restart network

STEP 9. If you have any applications running in marathon you should be able to look them up using mesos-dns. For example, I had a application named nodehello2. I was able to resolve the application using mesos-dns.

$ nslookup nodehello2.marathon.mesos

Name:   nodehello2.marathon.mesos
Name:   nodehello2.marathon.mesos

2016-01-31 21_13_24-2016-01-31 21_13_05-kube.txt - Notepad.png - Greenshot image editor

STEP 10. Additional verification can be done by hitting the node hello world app end point using the application name http://nodehello2.marathon.mesos with curl.

[root@mesos04 mesos-dns]$ docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS                     NAMES
2f6d8a4f99fd   "/bin/sh -c '/node/bi"   35 hours ago        Up 35 hours>8081/tcp   mesos-a78b235a-8427-4743-9bcc-5d6aed338412-S3.3698d9f9-a25a-457a-8602-50d9c26e70a7
38ca56e041f3   "/bin/sh -c '/node/bi"   35 hours ago        Up 35 hours>8081/tcp   mesos-a78b235a-8427-4743-9bcc-5d6aed338412-S3.e16a1e3e-a662-40da-b353-318de55178dc

[root@mesos04 mesos-dns]$ curl http://nodehello2.marathon.mesos:31495
Version 2.0
Hello World
[root@mesos04 mesos-dns]$ curl http://nodehello2.marathon.mesos:31884
Version 1.0
Hello World

STEP 11. You can also return the ports of the application. For example, nodehello2 is running on port 31472 on s2.marathon.slave.mesos and port 31495 on s3.marathon.slave.mesos.

[root@mesos04 mesos-dns]$ dig _nodehello2._tcp.marathon.mesos SRV

_nodehello2._tcp.marathon.mesos. 60 IN  SRV     0 0 31472 nodehello2-uhq4s-s2.marathon.slave.mesos.
_nodehello2._tcp.marathon.mesos. 60 IN  SRV     0 0 31495 nodehello2-sbk5j-s3.marathon.slave.mesos.

Setting Up a Docker Registry

I have been spending a lot of time getting familiar with Docker. Mostly working with Docker and Kubernetes. If you deploy a Kubernetes cluster you will most likely want to setup your own Docker Registry to pull down custom Docker images from.
I performed the following configuration on CentOS 7 and Docker version 1.8.

STEP 1. Pull down the Docker Registry image file.

docker run -d -p 5000:5000 --restart=always --name registry registry:2

STEP 2. Verify the image is running

docker ps

STEP 3. Verify the image you are trying to push exists on the local machine

docker images

At this point should have the ability to push docker images to the Registry.

However, while trying to configure the Docker Registry I keep receiving the following errors. It turns out by default docker uses HTTPS. Since this is a lab environment I opted to disable the certificate check. I would not recommend this in a production environment.

[root@docker01 node]> docker push
The push refers to a repository [] (len: 1)
unable to ping registry endpoint
v2 ping attempt failed with error: Get tls: oversized record received with length 20527
 v1 ping attempt failed with error: Get tls: oversized record received with length 20527

To enable a an insecure registry add the line below to your /etc/sysconfig/docker file.

[root@docker01 nfs]> vim /etc/sysconfig/docker

restart the docker engine

systemctl restart docker

Now tag your image in the Repository.

[root@docker01 nfs]> docker tag node_hello

And push the image to the local Repository.

[root@docker01 nfs]> docker push
The push refers to a repository [] (len: 1)
f2f0cae3343d: Pushed
f27103454f2e: Pushed
e05d047e2eff: Pushed
193fe95a2328: Pushed
53af2affa24d: Pushed
4432b60369bb: Pushed
51dd390dfbe4: Pushed
60e65a8e4030: Pushed
5764f0a31317: Pushed
838c1c5c4f83: Pushed
47d44cb6f252: Pushed
latest: digest: sha256:df1a645eb83e9059c6284d9787ca4c9a872f957b8dfa1bbb43fe83f6eeb5c8ee size: 22854
[root@docker01 nfs]>

Reference Documents:

Restore Couchbase Bucket

I wrote those post mainly for my own notes. The cbrestore command is a bit picky on the order in which you pass in the arguments. This is just a quick note for myself as it a task I don’t perform everyday.

Note that you do not have to pass in the exact location of the bucket folder you are trying to restore. Just the root folder which holds the bucket-* folder.

[someuser@host ~]> ls -la /mnt/mount/tmp/
total 16
drwxr-xr-x.  0 root root 4096 Jan 15 10:55 .
drwxr-xr-x. 23 root root    0 Jan 15 10:51 ..
drwxr-xr-x.  0 root root 4096 Jan 15 10:55 bucket-Store
drwxr-xr-x.  0 root root 4096 Jan 15 10:55 bucket-Web
[someuser@host ~]>

/opt/couchbase/bin/cbrestore /mnt/mount/tmp/ --bucket-destination=Web --bucket-source=Web http://cb01:8091 -u Administrator -p devPassword

Setup OpenVPN

OpenVPN was surprisingly easy to setup on in my lab environment. My setup included a CentOS 7 server running the latest version of OpenVPN server and a Windows 7 client running the latest OpenVPN client. I also have a Netgear router which I configured with a static route.

Before we begin you will need certificates
1. A computer to run OpenVPN (I used CentOS 7)
2. OpenVPN server will need a certificate
3. OpenVPN client will need a certificate
4. A home router which can be configured with static routes
5. A way to generate certificates for your vpn server and clients

Download the openVPN source code and compile it into an RPM

rpmbuild -tb /root/openvpn-2.3.8.tar.gz
rpm -ivh /root/rpmbuild/RPMS/x86_64/openvpn-2.3.8-1.x86_64.rpm

OpenVPN Server Configuration

Copy the sample server.conf file to /etc/openvpn/server.conf
Here is a list of settings I configured from the defaults server.conf file to get my OpenVPN server working.

# Which TCP/UDP port should OpenVPN listen on?
# If you want to run multiple OpenVPN instances
# on the same machine, use a different port
# number for each one. You will need to
# open up this port on your firewall.
port 1194

# TCP or UDP server?
proto udp

# "dev tun" will create a routed IP tunnel,
# "dev tap" will create an ethernet tunnel.
# Use "dev tap0" if you are ethernet bridging
# and have precreated a tap0 virtual interface
# and bridged it with your ethernet interface.
# If you want to control access policies
# over the VPN, you must create firewall
# rules for the the TUN/TAP interface.
# On non-Windows systems, you can give
# an explicit unit number, such as tun0.
# On Windows, use "dev-node" for this.
# On most systems, the VPN will not function
# unless you partially or fully disable
# the firewall for the TUN/TAP interface.

dev tun

# SSL/TLS root certificate (ca), certificate
# (cert), and private key (key). Each client
# and the server must have their own cert and
# key file. The server and all clients will
# use the same ca file.
# See the "easy-rsa" directory for a series
# of scripts for generating RSA certificates
# and private keys. Remember to use
# a unique Common Name for the server
# and each of the client certificates.
# Any X509 key management system can be used.
# OpenVPN can also use a PKCS #12 formatted key file
# (see "pkcs12" directive in man page).
ca lab-ca.pem
cert vpnserver.pem
key vpnserver-key-nopass.pem # This file should be kept secret

# Diffie hellman parameters.
# Generate your own with:
# openssl dhparam -out dh1024.pem 1024
# Substitute 2048 for 1024 if you are using
# 2048 bit keys.
dh dh1024.pem

# Configure server mode and supply a VPN subnet
# for OpenVPN to draw client addresses from.
# The server will take for itself,
# the rest will be made available to clients.
# Each client will be able to reach the server
# on Comment this line out if you are
# ethernet bridging. See the man page for more info.

# Push routes to the client to allow it
# to reach other private subnets behind
# the server. Remember that these
# private subnets will also need
# to know to route the OpenVPN client
# address pool (
# back to the OpenVPN server.
push "route"

# Certain Windows-specific network settings
# can be pushed to clients, such as DNS
# or WINS server addresses. CAVEAT:
push "dhcp-option DNS"

# The keepalive directive causes ping-like
# messages to be sent back and forth over
# the link so that each side knows when
# the other side has gone down.
# Ping every 10 seconds, assume that remote
# peer is down if no ping received during
# a 120 second time period.
keepalive 10 120

# Select a cryptographic cipher.
# This config item must be copied to
# the client config file as well.
cipher AES-128-CBC # AES

OpenVPN client Configuration
Download the OpenVPN client from here.
OpenVPN client configuration is saved here on Windows:
C:\Program Files\OpenVPN\config\client.ovpn

Here is a list of OpenVPN client settings I configured to get my OpenVPN client connected.

# Specify that we are a client and that we
# will be pulling certain config file directives
# from the server.

# Use the same setting as you are using on
# the server.
# On most systems, the VPN will not function
# unless you partially or fully disable
# the firewall for the TUN/TAP interface.
;dev tap
dev tun

# Are we connecting to a TCP or
# UDP server? Use the same setting as
# on the server.
;proto tcp
proto udp

# The hostname/IP and port of the server.
# You can have multiple remote entries
# to load balance between the servers.
# put my public IP here
remote 71.XX.XX.XXX 1194

# SSL/TLS parms.
# See the server config file for more
# description. It's best to use
# a separate .crt/.key file pair
# for each client. A single ca
# file can be used for all clients.
ca lab-ca.pem
cert usercert.pem
key key-pass.pem

# Select a cryptographic cipher.
# If the cipher option is used on the server
# then you must also specify it here.
cipher AES-128-CBC

Generate Certificates:
I generated my certificates using a Microsoft 2012 Certificate Authority. I generated one for certificate for the VPN server and another for the VPN client. I exported them from Microsoft CA in PFX format and used this Guide to convert them to PEM format.

My openVPN server certificate properties:
Subject Alternative Name=vpn
Subject Alternative
Subject Alternative Name=test02

My openVPN user certificate properties:
CN=user OU=WAU OU=US DC=lab DC=net

On the OpenVPN server copy the PEM files to /etc/openvpn/
On the OpenVPN Windows client copy the PEM files to C:\Program Files\OpenVPN\config\

Router Configuration
Configure your home router with a static route to the OpeVPN server on your home network
VPN client subnet:
OpenVPN Server:
2015-11-29 19_45_08-NETGEAR Router WNDR3400v2

Start the OpenVPN service on the OpenVPN server

systemctl start openvpn

Test Client Connection
On Windows 7 I noticied it was required to run the OpenVPN as administrator
Program Manager_2015-11-30_19-53-49

If you where successfully connected you should see “client is now connected”

Setup Rundeck with SSL

In this blog post I will describe the steps needed to configure rundeck to use SSL. I go through the steps of requesting a certificate from a Microsoft CA then exporting them to a Linux rundeck server. I then go through the steps of importing the certificates into a java keystore. And finally the configuration steps needed to get rundeck working with SSL.

STEP 1. Request a certificate
Open the mmc.exe > add/remove snapin > certificates > local computer

STEP 2. Click Next

STEP 3. Configure the CN (common name) and Subject Alternative names.

STEP 4. Mark private key as exportable

STEP 5. Select Enroll

STEP 6. Export the certificate

STEP 7. Export private key

STEP 8. Export the certificate and private key in PKCS #12 format

STEP 9. Set private key password

STEP 10. Export the the Certificate Authorities certificate.
This certificate will be placed in the the trusted CA Java keystore. Do not export the private key for the CA, export the CA as DER format.

STEP 10. SFTP the certificate to your Linux Rundeck Server
I placed the rundeck.pfx file in /etc/rundeck/ssl
Also place the ca.cer file in /etc/rundeck/ssl

STEP 11. Create a keystore for the rundeck.pfx certificate
Create a Java keystore to hold the new rundeck certificate

keytool -keystore /etc/rundeck/ssl/keystore -alias rundeck -genkey -keyalg RSA -keypass password -storepass password

STEP 12. Retrieve the alias from the PKCS #12 file
Save the alias id, you will need this for the next step

keytool -v -list -storetype pkcs12 -keystore /etc/rundeck/ssl/rundeck.pfx


STEP 13. Import the Certificate and Private Key into the Java keystore
Use the alias id from the previous command as the source alias and destination alias.

keytool -importkeystore -deststorepass password -destkeypass password -destkeystore /etc/rundeck/ssl/keystore -srckeystore /etc/rundeck/ssl/rundeck.pfx -srcstoretype PKCS12 -srcstorepass password -srcalias le-webserver-e8683358-23d9-4477-a6c8-21cc2c400c10 -alias le-webserver-e8683358-23d9-4477-a6c8-21cc2c400c10

STEP 14. Create a keystore for the ca.cer certificate authority

keytool -keystore /etc/rundeck/ssl/ca -alias rundeck -genkey -keyalg RSA -keypass password -storepass password

STEP 15. Add the CA cert to the CA keystore

keytool -import -alias ca -file /etc/rundeck/ssl/lab-ca-der.cer -keystore /etc/rundeck/ssl/ca -storepass password
Trust this certificate? [no]:  yes
Certificate was added to keystore

STEP 16. Review of previous steps
a. At this point we should have requested and received a certificate from the Microsoft CA
b. Export the CA’s certificate
c. Created a java keystore for our rundeck certificate
d. Created a java keystore for our CA certificate

STEP 17. Configure Rundeck /etc/rundeck/etc/
Configure the path to the certificate keystore and CA keystore you created earlier


STEP 18. Configure /etc/rundeck/profile
Add the following options the rundeck JVM

export RDECK_JVM="
        -Drundeck.ssl.config=/etc/rundeck/ssl/ \

STEP 19. Configure /etc/rundeck/
Update the property below with https and 4443


STEP 20. Configure /etc/rundeck/
Configure the appropriate port 4443 and update the url https

framework.server.port = 4443
framework.server.url =

At this point you should be able to hit https://rundeck:4443 and make a secure connection.
For troubleshooting look at the /var/log/rundeck/service.log.

Configure Rundeck to use Active Directory Authentication

This guide was written using the rundeck 2.4.2 RPM installed on CentOS 6.5. I go over the steps needed to setup Active Directory authentication in Rundeck

STEP 1. CREATE Active Directory Group

In Active Directory create a new group named “rundeckusers.” Then add your users to that AD group.

STEP 2. Create jaas-activedirectory.conf file

touch /etc/rundeck/jaas-activedirectory.conf
chown rundeck:rundeck /etc/rundeck/jaas-activedirectory.conf

Enter the following configuration settings into your jaas-ldap.conf file. You will need to configure the username/password for the user which will bind to Active Directory. You will also need to configure the userBaseDn. This is the OU which recursive searches for users will be performed on. In addition, configuring the roleBaseDn. The roleBaseDn is the OU where your “rundeck” AD user group is.

activedirectory {
    com.dtolabs.rundeck.jetty.jaas.JettyCachingLdapLoginModule required

STEP 3. Modify /etc/rundeck/profile

You’ll need to configure / modify to two lines. Add the path to the jaas-activedirectory.conf file and the loginmodule name, “activedirectory.” The login module name is the same as the name used in the jaas-activedirectory.conf file.

export RDECK_JVM=" \"

STEP 4. Create file /etc/rundeck/rundeckusers.aclpolicy
Add the ACL policy below for the admin in Rundeck. The group field should be the Active Directory user group “rundeckusers.” All users in the AD group with have admin access in rundeck.

touch /etc/rundeck/rundeckusers.aclpolicy
chown rundeck:rundeck /etc/rundeck/rundeckusers.aclpolicy
description: Admin project level access control. Applies to resources within a specific project.
  project: '.*' # all projects
    - equals:
        kind: job
      allow: [create] # allow create jobs
    - equals:
        kind: node
      allow: [read,create,update,refresh] # allow refresh node sources
    - equals:
        kind: event
      allow: [read,create] # allow read/create events
    - allow: [read,run,runAs,kill,killAs] # allow running/killing adhoc jobs
    - allow: [create,read,update,delete,run,runAs,kill,killAs] # allow create/read/write/delete/run/kill of all jobs
    - allow: [read,run] # allow read/run for nodes
  group: [rundeckusers]


description: Admin Application level access control, applies to creating/deleting projects, admin of user profiles, viewing projects and reading system information.
  application: 'rundeck'
    - equals:
        kind: project
      allow: [create] # allow create of projects
    - equals:
        kind: system
      allow: [read] # allow read of system info
    - equals:
        kind: user
      allow: [admin] # allow modify user profiles
    - match:
        name: '.*'
      allow: [read,import,export,configure,delete] # allow full access of all projects or use 'admin'
    - allow: [read,create,update,delete] # allow access for /ssh-key/* storage content

  group: [rundeckusers]

STEP 5. Configure Secure LDAP
Import the CA certificate which was used to setup Secure LDAP on the Active Directory Domain Controller. To secure the LDAP connection between the rundeck server and the AD domain controller it is recommended to import and trust the CA used on the domain controller. Then configure the jaas-ldap.conf file to use ldaps.

keytool -import -alias -file /root/CA.pem -keystore /usr/lib/jvm/java-1.7.0-openjdk- -storepass changeit