Over the last month I have been evaluating container clustering software. I started with Kubernetes, Rancher which uses swarm and Mesos. I am going through these evaluations to determine which container clustering software will fit my employer’s needs best.
ENVIRONMENT CENTOS 7.0 running three Mesos masters and two Mesos slaves
NAME: mesos01.lab.net
IP 172.16.1.80
services: zookeeper, marathon, mesos-master
NAME: mesos02.lab.net
IP 172.16.1.81
services: zookeeper, marathon, mesos-master
NAME: mesos03.lab.net
IP 172.16.1.82
services: zookeeper, marathon, mesos-master
NAME: mesos04.lab.net
IP 172.16.1.83
services: mesos-slave
NAME: mesos05.lab.net
IP 172.16.1.84
services: mesos-slave
STEP 1. Prerequisites install golang and git
$ yum install go git
$ export GOPATH=$HOME/go $ export PATH=$PATH:$GOPATH/bin
$ go get github.com/tools/godep $ go get github.com/mesosphere/mesos-dns/logging $ go get github.com/mesosphere/mesos-dns/records $ go get github.com/mesosphere/mesos-dns/resolver
STEP 2. Clone the mesos-dns repository and build the mesos-dns binary.
$ git clone https://github.com/mesosphere/mesos-dns.git $ cd ./mesos-dns $ go build -o mesos-dns
After building mesos-dns you should have a mesos-dns binary file in your
./mesos-dns directory
STEP 3. In the ./mesos-dns directory there is a config.json.sample example file.
Copy this file and edit it for your own environment.
$ cp config.json.sample config.json
This link describes the each of the fields in the config.json file.
{
"zk": "zk://172.16.1.80:2181,172.16.1.81:2181,172.16.1.82:2181/mesos",
"masters": ["172.16.1.80:5050","172.16.1.81:5050","172.16.1.82:5050"],
"stateTimeoutSeconds": 300,
"refreshSeconds": 60,
"ttl": 60,
"domain": "mesos",
"ns": "ns1",
"port": 53,
"resolvers": ["172.16.1.21"],
"timeout": 5,
"listener": "0.0.0.0",
"SOAMname": "root.ns1.mesos",
"SOARname": "ns1.mesos",
"SOARefresh": 60,
"SOARetry": 600,
"SOAExpire": 86400,
"SOAMinttl": 60,
"dnson": true,
"httpon": true,
"httpport": 8123,
"externalon": true,
"recurseon": true,
"IPSources": ["mesos", "host"],
"EnforceRFC952": false
}
STEP 4. Run the mesos-dns with the config.json file to verify it is properly formatted.
$ ./mesos-dns -config=config.json
On the mesos slave create a directory for the config.json file. I have designated mesos04.lab.net as the mesos-dns server for my cluster.
$ mkdir /etc/mesos-dns
STEP 5. Copy the mesos-dns binary to the mesos slave which you have designated as the mesos-dns server. In this example I copy the mesos-dns service to mesos slave mesos04.
$ scp ./mesos-dns/mesos-dns root@mesos04.lab.net:/usr/local/bin/mesos-dns
STEP 6. Configure the constraints for the mesos-dns service. This essentially tells the marathon to constrain the mesos-dns service to host mesos04.lab.net. For example, you may want to designate two nodes in your cluster to run mesos-dns. The constrains directive ensures that mesos-dns does not try to run on other hosts.
Constraints: hostname:CLUSTER:mesos04.lab.net
STEP 7. Update the network-script file with IP address of the host running mesos-dns.
$ vim /etc/sysconfig/network-scripts/ifcfg-ens160
DNS1="172.16.1.83" DNS2="172.16.1.21"
STEP 8. After updating the network-script file restart the network service
systemctl restart network
STEP 9. If you have any applications running in marathon you should be able to look them up using mesos-dns. For example, I had a application named nodehello2. I was able to resolve the application using mesos-dns.
$ nslookup nodehello2.marathon.mesos Server: 172.16.1.83 Address: 172.16.1.83#53 Name: nodehello2.marathon.mesos Address: 172.16.1.84 Name: nodehello2.marathon.mesos Address: 172.16.1.83
STEP 10. Additional verification can be done by hitting the node hello world app end point using the application name http://nodehello2.marathon.mesos with curl.
[root@mesos04 mesos-dns]$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 2f6d8a4f99fd 172.16.1.60:5000/node_hello:2.0 "/bin/sh -c '/node/bi" 35 hours ago Up 35 hours 0.0.0.0:31495->8081/tcp mesos-a78b235a-8427-4743-9bcc-5d6aed338412-S3.3698d9f9-a25a-457a-8602-50d9c26e70a7 38ca56e041f3 172.16.1.60:5000/node_hello:1.0 "/bin/sh -c '/node/bi" 35 hours ago Up 35 hours 0.0.0.0:31884->8081/tcp mesos-a78b235a-8427-4743-9bcc-5d6aed338412-S3.e16a1e3e-a662-40da-b353-318de55178dc [root@mesos04 mesos-dns]$ curl http://nodehello2.marathon.mesos:31495 Version 2.0 Hello World [root@mesos04 mesos-dns]$ curl http://nodehello2.marathon.mesos:31884 Version 1.0 Hello World
STEP 11. You can also return the ports of the application. For example, nodehello2 is running on port 31472 on s2.marathon.slave.mesos and port 31495 on s3.marathon.slave.mesos.
[root@mesos04 mesos-dns]$ dig _nodehello2._tcp.marathon.mesos SRV ;; ANSWER SECTION: _nodehello2._tcp.marathon.mesos. 60 IN SRV 0 0 31472 nodehello2-uhq4s-s2.marathon.slave.mesos. _nodehello2._tcp.marathon.mesos. 60 IN SRV 0 0 31495 nodehello2-sbk5j-s3.marathon.slave.mesos.
Tater's Tech Blog 