Setup Mesos-DNS

2016-01-31 21_18_49-MarathonOver the last month I have been evaluating container clustering software. I started with Kubernetes, Rancher which uses swarm and Mesos. I am going through these evaluations to determine which container clustering software will fit my employer’s needs best.

ENVIRONMENT CENTOS 7.0 running three Mesos masters and two Mesos slaves

NAME: mesos01.lab.net
IP 172.16.1.80
services: zookeeper, marathon, mesos-master

NAME: mesos02.lab.net
IP 172.16.1.81
services: zookeeper, marathon, mesos-master

NAME: mesos03.lab.net
IP 172.16.1.82
services: zookeeper, marathon, mesos-master

NAME: mesos04.lab.net
IP 172.16.1.83
services: mesos-slave

NAME: mesos05.lab.net
IP 172.16.1.84
services: mesos-slave

STEP 1. Prerequisites install golang and git

$ yum install go git
$ export GOPATH=$HOME/go
$ export PATH=$PATH:$GOPATH/bin
$ go get github.com/tools/godep

$ go get github.com/mesosphere/mesos-dns/logging
$ go get github.com/mesosphere/mesos-dns/records
$ go get github.com/mesosphere/mesos-dns/resolver

STEP 2. Clone the mesos-dns repository and build the mesos-dns binary.

$ git clone https://github.com/mesosphere/mesos-dns.git
$ cd ./mesos-dns
$ go build -o mesos-dns

After building mesos-dns you should have a mesos-dns binary file in your
./mesos-dns directory

STEP 3. In the ./mesos-dns directory there is a config.json.sample example file.
Copy this file and edit it for your own environment.

$ cp config.json.sample config.json

ThisĀ link describes the each of the fields in the config.json file.

{
  "zk": "zk://172.16.1.80:2181,172.16.1.81:2181,172.16.1.82:2181/mesos",
  "masters": ["172.16.1.80:5050","172.16.1.81:5050","172.16.1.82:5050"],
  "stateTimeoutSeconds": 300,
  "refreshSeconds": 60,
  "ttl": 60,
  "domain": "mesos",
  "ns": "ns1",
  "port": 53,
  "resolvers": ["172.16.1.21"],
  "timeout": 5,
  "listener": "0.0.0.0",
  "SOAMname": "root.ns1.mesos",
  "SOARname": "ns1.mesos",
  "SOARefresh": 60,
  "SOARetry":   600,
  "SOAExpire":  86400,
  "SOAMinttl": 60,
  "dnson": true,
  "httpon": true,
  "httpport": 8123,
  "externalon": true,
  "recurseon": true,
  "IPSources": ["mesos", "host"],
  "EnforceRFC952": false
}

STEP 4. Run the mesos-dns with the config.json file to verify it is properly formatted.

$ ./mesos-dns -config=config.json
On the mesos slave create a directory for the config.json file.
I have designated mesos04.lab.net as the mesos-dns server for my
cluster.
$ mkdir /etc/mesos-dns

STEP 5. Copy the mesos-dns binary to the mesos slave which you have designated as the mesos-dns server. In this example I copy the mesos-dns service to mesos slave mesos04.

$ scp ./mesos-dns/mesos-dns root@mesos04.lab.net:/usr/local/bin/mesos-dns

STEP 6. Configure the constraints for the mesos-dns service. This essentially tells the marathon to constrain the mesos-dns service to host mesos04.lab.net. For example, you may want to designate two nodes in your cluster to run mesos-dns. The constrains directive ensures that mesos-dns does not try to run on other hosts.
Constraints: hostname:CLUSTER:mesos04.lab.net

marathon-mesos-dns

STEP 7. Update the network-script file with IP address of the host running mesos-dns.

$ vim /etc/sysconfig/network-scripts/ifcfg-ens160
DNS1="172.16.1.83"
DNS2="172.16.1.21"

STEP 8. After updating the network-script file restart the network service

systemctl restart network

STEP 9. If you have any applications running in marathon you should be able to look them up using mesos-dns. For example, I had a application named nodehello2. I was able to resolve the application using mesos-dns.

$ nslookup nodehello2.marathon.mesos
Server:         172.16.1.83
Address:        172.16.1.83#53

Name:   nodehello2.marathon.mesos
Address: 172.16.1.84
Name:   nodehello2.marathon.mesos
Address: 172.16.1.83

2016-01-31 21_13_24-2016-01-31 21_13_05-kube.txt - Notepad.png - Greenshot image editor

STEP 10. Additional verification can be done by hitting the node hello world app end point using the application name http://nodehello2.marathon.mesos with curl.

[root@mesos04 mesos-dns]$ docker ps
CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS                     NAMES
2f6d8a4f99fd        172.16.1.60:5000/node_hello:2.0   "/bin/sh -c '/node/bi"   35 hours ago        Up 35 hours         0.0.0.0:31495->8081/tcp   mesos-a78b235a-8427-4743-9bcc-5d6aed338412-S3.3698d9f9-a25a-457a-8602-50d9c26e70a7
38ca56e041f3        172.16.1.60:5000/node_hello:1.0   "/bin/sh -c '/node/bi"   35 hours ago        Up 35 hours         0.0.0.0:31884->8081/tcp   mesos-a78b235a-8427-4743-9bcc-5d6aed338412-S3.e16a1e3e-a662-40da-b353-318de55178dc

[root@mesos04 mesos-dns]$ curl http://nodehello2.marathon.mesos:31495
Version 2.0
Hello World
[root@mesos04 mesos-dns]$ curl http://nodehello2.marathon.mesos:31884
Version 1.0
Hello World

STEP 11. You can also return the ports of the application. For example, nodehello2 is running on port 31472 on s2.marathon.slave.mesos and port 31495 on s3.marathon.slave.mesos.

[root@mesos04 mesos-dns]$ dig _nodehello2._tcp.marathon.mesos SRV

;; ANSWER SECTION:
_nodehello2._tcp.marathon.mesos. 60 IN  SRV     0 0 31472 nodehello2-uhq4s-s2.marathon.slave.mesos.
_nodehello2._tcp.marathon.mesos. 60 IN  SRV     0 0 31495 nodehello2-sbk5j-s3.marathon.slave.mesos.
Advertisements