Dashing on Openshift

January 18th, 2017

Disclaimer: I don’t really know how to code Ruby!!!

Playing around with dashing on Openshift. Dashing is pretty simple framework to create your own pretty dashboards. In its own words:

Dashing is a Sinatra based framework that lets you build beautiful dashboards.

Check out a demo over here. Here’s another one, optimized for 1080p screens.

Key features:

  • Use premade widgets, or fully create your own with scss, html, and coffeescript.
  • Widgets harness the power of data bindings to keep things DRY and simple. Powered by batman.js.
  • Use the API to push data to your dashboards, or make use of a simple ruby DSL for fetching data.
  • Drag & Drop interface for re-arranging your widgets.
  • Host your dashboards on Heroku in less than 30 seconds.

This project was created at Shopify for displaying custom dashboards on TVs around the office.

Great post on setting up dashing on a raspberry pi

Dashing on Openshift

Let’s get a simple quickstart up and running on OpenShift…

gem install dashing
dashing new basic-dashboard
cd basic-dashboard
bundle
dashing start

navigate to http://localhost:3030/sample to confirm the dashboard is running locally… then CTRL-C to stop it…

rm Gemfile.lock

# create a local and remote git repo and push...
git init
git remote add origin git@github.com:welshstew/basic-dashboard.git
git add * -f
git tag -a v1.0 -m "basic dashing quickstart"
git push origin master --all

Now we are ready to get this up and running on openshift. The following commands will kick off a build and make the application accessible via a route.

oc new-project dashing
oc new-app openshift/ruby:latest~https://github.com/welshstew/basic-dashboard#v1.0
oc expose svc/basic-dashboard

You can now visit http://basic-dashboard-dashing.rhel-cdk.10.1.2.2.xip.io/sample to see the dashboard running.

Querying Kubernetes/Openshift resources

In order for dashing to grab any http resources from openshift, it needs to be authenticated. Openshift exposes a token via /var/run/secrets/kubernetes.io/serviceaccount/token, which will need to be granted view access in order to be able to query those resources. The pod is currently running as the default user - so we will give it read access.

oc policy add-role-to-user view system:serviceaccount:dashing:default -n dashing

Grab extra widgets https://github.com/Shopify/dashing/wiki/Additional-Widgets

# Dashing table widget : https://gist.github.com/nick123pig/9c7cb3030f63ad10e517
dashing install 9c7cb3030f63ad10e517
bundle install

Update the buildconfig source ref to point to the tag in my git repo for the code:

oc edit bc/basic-dashboard
...

  runPolicy: Serial
  source:
    git:
      ref: v1.1
      uri: https://github.com/welshstew/basic-dashboard
    type: Git
  strategy:
    sourceStrategy:

Two means of grabbing data are shown in the following files:

  • pods - uses json and REST to get the data
  • kubepods - uses the kubeclient gem in order to invoke the kubernetes API

Should result in the tables being displayed:

Tables displayed

Grabbing stats from pods exposing Jolokia

Setup a simple spring boot camel application…

In order to demonstrate grabbing the jolokia stats exposed by openshift java pods, we will:

  1. Generate a spring boot application based on the FIS 2.0 documentation. Please note, in order to build this you’ll need to populate your maven settings file with the appropriate repositories.
  2. Create a git repo and push the code generated to that repo
# Part 1
mvn archetype:generate \
  -DarchetypeCatalog=https://maven.repository.redhat.com/earlyaccess/all/io/fabric8/archetypes/archetypes-catalog/2.2.180.redhat-000004/archetypes-catalog-2.2.180.redhat-000004-archetype-catalog.xml \
  -DarchetypeGroupId=org.jboss.fuse.fis.archetypes \
  -DarchetypeArtifactId=spring-boot-camel-archetype \
  -DarchetypeVersion=2.2.180.redhat-000004

# Part 2
git clone https://github.com/welshstew/camel-springboot-logger.git
oc import-image jboss-fuse-6-tech-preview/fis-java-openshift  --from=registry.access.redhat.com/jboss-fuse-6-tech-preview/fis-java-openshift -n openshift --confirm
oc new-build jboss-fuse-6-tech-preview/fis-java-openshift:latest~https://github.com/welshstew/camel-springboot-logger.git --allow-missing-images --strategy=source

# generate the openshift.yml file to create the openshift resources/objects
mvn clean fabric8:resource
oc create -f target/classes/META-INF/fabric8/openshift.yml

The camel springboot pod should now be up and running in the dashing namespace

Pods in Openshift

Grabbing the statistics

By navigating to the camel Java console we can see how to grab particular attributes from Jolokia. See the image to see what the URL is for the ExchangeCompleted value in “route1”.

Exchanges Completed

Java Console REST URL

The url is https://10.1.2.2:8443/api/v1/namespaces/dashing/pods/https:camel-springboot-logger-1-27p5q:8778/proxy/jolokia/exec/org.apache.camel:context=MyCamel,type=routes,name=%22route1%22/getExchangesCompleted(). We can go ahead and use POSTMAN to try this out.

Postman

Or use curl…

curl -X GET -H "Cache-Control: no-cache" -H "Postman-Token: 6c2281e8-c6a8-60c2-568c-88c77c2951dd" "https://10.1.2.2:8443/api/v1/namespaces/dashing/pods/https:camel-springboot-logger-1-27p5q:8778/proxy/jolokia/exec/org.apache.camel:context%3DMyCamel%2Ctype%3Droutes%2Cname%3D%22route1%22/getExchangesCompleted()" --insecure
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "User \"system:anonymous\" cannot get pods/unsafeproxy in project \"dashing\"",
  "reason": "Forbidden",
  "details": {
    "name": "https:camel-springboot-logger-1-27p5q:8778",
    "kind": "pods/unsafeproxy"
  },
  "code": 403
}

We are unauthorized, so we need a HTTP Authorization header containing the token of a user which has access to this namespace. We can try this with our admin token…

➜  camel-springboot-logger git:(master) oc whoami -t
sKxDTx5qvwQAKV9GJ8CxDn9nsM2kihTD6BTt_gWtf00

And try again…

curl -X GET -H "Cache-Control: no-cache" -H "Authorization: Bearer sKxDTx5qvwQAKV9GJ8CxDn9nsM2kihTD6BTt_gWtf00" -H "Postman-Token: 6c2281e8-c6a8-60c2-568c-88c77c2951dd" "https://10.1.2.2:8443/api/v1/namespaces/dashing/pods/https:camel-springboot-logger-1-27p5q:8778/proxy/jolokia/exec/org.apache.camel:context%3DMyCamel%2Ctype%3Droutes%2Cname%3D%22route1%22/getExchangesCompleted()" --insecure

{"request":{"mbean":"org.apache.camel:context=MyCamel,name=\"route1\",type=routes","type":"exec","operation":"getExchangesCompleted()"},"value":462,"timestamp":1484820093,"status":200}

It can be seen from the URL that the pod name forms part of the request…

➜  camel-springboot-logger git:(master) oc get pods | grep camel
camel-springboot-logger-1-27p5q   1/1       Running     0          44m
camel-springboot-logger-1-build   0/1       Completed   0          11h

Breaking down the structure of the URL we have :

  • Original URL : https://10.1.2.2:8443/api/v1/namespaces/dashing/pods/https:camel-springboot-logger-1-27p5q:8778/proxy/jolokia/exec/org.apache.camel:context%3DMyCamel%2Ctype%3Droutes%2Cname%3D%22route1%22/getExchangesCompleted()
  • Structure : https://SERVER_IP:SERVER_PORT/api/v1/namespaces/NAMESPACE_NAME/pods/https:POD_NAME:8778/proxy/jolokia/exec/MBEAN NAME - APP DEPENDANT

Trying this via Dashing

Determine the pod to select by selecting via the correct label.

➜  camel-springboot-logger git:(master) oc get pods --show-labels | grep camel
camel-springboot-logger-1-27p5q   1/1       Running     0          1h        deployment=camel-springboot-logger-1,deploymentconfig=camel-springboot-logger,group=com.nullendpoint,project=camel-springboot-logger,provider=fabric8,version=1.0-SNAPSHOT
camel-springboot-logger-1-build   0/1       Completed   0          12h       openshift.io/build.name=camel-springboot-logger-1

Changing the sample.erb:

    <li data-row="1" data-col="1" data-sizex="1" data-sizey="1">
      <div data-id="camel-springboot-logger-exchanges" data-view="Number" data-title="Exchanges Completed" data-moreinfo="from camel-springboot-logger" ></div>
    </li>

I added the camel-springboot-data.rb file, which makes 2 http calls..

  1. First call determines the name of the camel-springboot-logger pod
  2. Second call actually grabs a statistic from Jolokia of the camel-springboot-logger pod.
#call kube api to get the pod name
kubepods = client.get_pods(namespace: namespace, label_selector: label)

#####

# construct a RESTFUL API GET to grab the Jolokia stat
url = URI.parse("#{server}/api/v1/namespaces/#{namespace}/pods/https:#{podName}:8778/proxy/jolokia/exec/#{mbean}")
http = Net::HTTP.new(url.host, port)
http.use_ssl = (url.scheme == 'https')
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
response = http.request(Net::HTTP::Get.new(url.request_uri, {'Authorization' => "Bearer #{token}"}))
j = JSON[response.body]
current_exchanges_completed = j["value"]

Exchanges Completed Square

Grabbing multiple stats

If we want to get multiple results back from Jolokia, we need to form a HTTP POST request, and parse the response.

https://10.1.2.2:8443/api/v1/namespaces/dashing/pods/https:camel-springboot-logger-1-sgud6:8778/proxy/jolokia/

{
    "attribute": [
        "Uptime",
        "State",
        "MinProcessingTime",
        "MaxProcessingTime",
        "LastProcessingTime",
        "TotalRoutes",
        "TotalProcessingTime"
    ],
    "mbean": "org.apache.camel:context=MyCamel,type=context,name=\"MyCamel\"",
    "type": "read"
}

I added the camel-springboot-data-post.rb file to show how to grab a number of statistics from a single request.

Camel Multiple Stats

Setup the local hawkular metrics

We should get the metrics up and running by following the README on Github. This can take some time due to all the containers it needs to pull…!


git clone https://github.com/openshift/origin-metrics.git

oc create -f metrics-deployer-setup.yaml -n openshift-infra
oadm policy add-role-to-user edit system:serviceaccount:openshift-infra:metrics-deployer -n openshift-infra
oadm policy add-cluster-role-to-user cluster-reader system:serviceaccount:openshift-infra:heapster -n openshift-infra
oadm policy add-role-to-user view system:serviceaccount:openshift-infra:hawkular -n openshift-infra
oc secrets new metrics-deployer nothing=/dev/null -n openshift-infra

#Prepare to wait a while after running this command...!
oc process -f metrics.yaml -v HAWKULAR_METRICS_HOSTNAME=hawkular-metrics.rhel-cdk.10.1.2.2.xip.io,CASSANDRA_PV_SIZE=1Gi | oc create -f -


# if anything goes wrong with the metrics-deployer - do this and start over...

oc delete all,sa,templates,secrets,pvc --selector="metrics-infra"
oc delete sa,secret metrics-deployer


# Also be sure to remember to update the metricsPublicUrl in the master-config.yaml ( /var/lib/openshift/openshift.local.config/master/master-config.yaml )
vagrant ssh
su -
vi /var/lib/openshift/openshift.local.config/master/master-config.yaml

#  master-config.yaml snippet...
  masterPublicURL: https://10.1.2.2:8443
  metricsPublicURL: "https://hawkular-metrics.rhel-cdk.10.1.2.2.xip.io/hawkular/metrics"
  publicURL: https://10.1.2.2:8443/console/

If everything has worked, then we should see metrics on our pods in the OpenShift Console:

Metrics on Pod

Grabbing Metrics data from Hawkular

We can see that the Openshift console is now getting the CPU, and memory data from Hawkular. We can also grab that data from a REST request. To find out what these requests look like we can look at the network tab in the chrome developer tools, and of course, check out the Hawkular API documentation

#request

curl 'https://hawkular-metrics.rhel-cdk.10.1.2.2.xip.io/hawkular/metrics/gauges/data?stacked=true&tags=descriptor_name:memory%2Fusage,type:pod_container,labels:%5E%28%3F%3D.%2A%5Cbdeployment%3Acamel-springboot-logger-1%5Cb%29%28%3F%3D.%2A%5Cbdeploymentconfig%3Acamel-springboot-logger%5Cb%29%28%3F%3D.%2A%5Cbgroup%3Acom.nullendpoint%5Cb%29%28%3F%3D.%2A%5Cbproject%3Acamel-springboot-logger%5Cb%29%28%3F%3D.%2A%5Cbprovider%3Afabric8%5Cb%29.%2A%24&bucketDuration=60000ms&start=1484848650593' -H 'Hawkular-Tenant: dashing' -H 'Origin: https://10.1.2.2:8443' -H 'Accept-Encoding: gzip, deflate, sdch, br' -H 'Accept-Language: en-US,en;q=0.8' -H 'Authorization: Bearer s8vH-4lH277A07aBeKvlmtR267aXyauBF4NbXh8EXI0' -H 'Accept: application/json' -H 'Referer: https://10.1.2.2:8443/console/project/dashing/overview' -H 'User-Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36' -H 'Connection: keep-alive' --compressed --insecure


#response

[{"start":1484848650593,"end":1484848710593,"min":6.49949184E8,"avg":6.527563093333334E8,"median":6.52046336E8,"max":6.56273408E8,"sum":1.958268928E9,"samples":1,"empty":false},{"start":1484848710593,"end":1484848770593,"min":0.0,"avg":0.0,"median":0.0,"max":0.0,"sum":0.0,"empty":true}]

https://hawkular-metrics.rhel-cdk.10.1.2.2.xip.io/hawkular/metrics/gauges/data?stacked=true&tags=descriptor_name:memory/usage,type:pod_container,labels:^(?=.\bdeployment:camel-springboot-logger-1\b)(?=.\bdeploymentconfig:camel-springboot-logger\b)(?=.\bgroup:com.nullendpoint\b)(?=.\bproject:camel-springboot-logger\b)(?=.\bprovider:fabric8\b).$&bucketDuration=60000ms&start=1484848650593

codergists