Testing ActiveMQ on Openshift using the activemq-perf-plugin

August 24th, 2018

I have been meaning to write a blog post on this topic for a while. When on engagements and installing ActiveMQ on Openshift, I am often asked what is a quick way of testing out the install. Many times I would write a simple application using camel with a producer and consumer, however, this wouldn’t result in giving much on the statistics front.

In standalone installs of ActiveMQ I have used the perftest plugin which gives out some great statistics. In this article I will show how to quickly build an image on openshift in order to run jobs to test ActiveMQ.

The following will clone a git repo and create a new build in Openshift, this will create an image that has maven and the plugin dependencies pre-baked in that image. Once the build has finished it will be available for us to create Jobs using that image. The amq-perf-test-pvc is a shared persistent volume area where we will store the reports generated by the job.

# Clone this repository:
git clone https://github.com/welshstew/amq-perf-test-job.git

#Build the container with the amq-perf-plugin built-in
oc new-build fis-java-openshift:2.0~https://github.com/welshstew/amq-perf-test-job.git

#Create a shared PVC
oc create -f openshift/amq-perf-test-pvc.json

To create a job we can run the following, where 172.30.1.1 is my minishift registry ip oc get svc/docker-registry -n default -o jsonpath='{.spec.clusterIP}') and amq is the namespace where I have built the amq-perf-test-job image. This job creates both a consumer and a producer.

Please note that all of the -D options on the command are configured ot suit my environment (including username and password for the ActiveMQ instance). All of the configuration parameters for the activemq-perf-maven-plugin are documented at: http://activemq.apache.org/activemq-performance-module-users-manual.html

oc create -f - <<EOF
---
apiVersion: batch/v1
kind: Job
metadata:
  name: amq-perf-test-1
spec:
  parallelism: 1    
  completions: 1    
  template:         
    spec:
      containers:
      - name: perf-producer-1
        image: 172.30.1.1:5000/amq/amq-perf-test-job:latest
        command: ["/bin/bash", "-c", "mvn -f /tmp/src/pom.xml activemq-perf:producer -Dfactory.brokerURL=tcp://broker-amq-tcp:61616 -DsysTest.numClients=1 -Dconsumer.durable=true -Dfactory.userName=admin -Dfactory.password=admin -DsysTest.reportDir=/test/ -DsysTest.reportName=amq-perf-producer-1 -Dmaven.repo.local=/tmp/artifacts/m2"]
        resources:
          limits:
            cpu: 300m
            memory: 512Mi
          requests:
            cpu: 300m
            memory: 512Mi        
        volumeMounts:
        - mountPath: /test
          name: test
      - name: perf-consumer-1
        image: 172.30.1.1:5000/amq/amq-perf-test-job:latest
        command: ["/bin/bash", "-c", "mvn -f /tmp/src/pom.xml activemq-perf:consumer -Dfactory.brokerURL=tcp://broker-amq-tcp:61616 -DsysTest.numClients=1 -Dconsumer.durable=true -Dfactory.userName=admin -Dfactory.password=admin -DsysTest.reportDir=/test/ -DsysTest.reportName=amq-perf-consumer-1 -Dmaven.repo.local=/tmp/artifacts/m2"]
        resources:
          limits:
            cpu: 300m
            memory: 512Mi
          requests:
            cpu: 300m
            memory: 512Mi        
        volumeMounts:
        - mountPath: /test
          name: test          
      volumes:
      - name: test
        persistentVolumeClaim:
          claimName: perf-test-claim
      restartPolicy: Never
...
EOF

The results of the job are available from the pod logs:

oc logs jobs/amq-perf-test-1 -c perf-producer-1
...
########################################
####    SYSTEM CPU USAGE SUMMARY    ####
########################################
Total Blocks Received: 1615560
Ave Blocks Received: 6731.5
Total Blocks Sent: 1822740
Ave Blocks Sent: 7594.75
Total Context Switches: 1830966
Ave Context Switches: 7629.025
Total User Time: 7433
Ave User Time: 30.970833333333335
Total System Time: 5084
Ave System Time: 21.183333333333334
Total Idle Time: 10882
Ave Idle Time: 45.34166666666667
Total Wait Time: 277
Ave Wait Time: 1.1541666666666666
[INFO] Created performance report: /test/amq-perf-producer-1.xml
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
...

Also, the report will be saved in the shared persistent storage. Below I show how to get this from minishift.

[user@localhost amq-perf-test-job]$ oc get pv | grep perf-test-claim
pv0059    100Gi      RWO,ROX,RWX    Recycle          Bound       amq/perf-test-claim                             5d

We can export the PV and see where on the filesystem the storage actually is, in this case, it’s in /var/lib/minishift/openshift.local.pv/pv0059 inside the minishift instance.

[user@localhost amq-perf-test-job]$ oc get pv | grep perf-test-claim | awk '{print $1}' | xargs oc export pv
apiVersion: v1
kind: PersistentVolume
metadata:
  annotations:
    pv.kubernetes.io/bound-by-controller: "yes"
  creationTimestamp: null
  labels:
    volume: pv0059
  name: pv0059
spec:
  accessModes:
  - ReadWriteOnce
  - ReadWriteMany
  - ReadOnlyMany
  capacity:
    storage: 100Gi
  claimRef:
    apiVersion: v1
    kind: PersistentVolumeClaim
    name: perf-test-claim
    namespace: amq
    resourceVersion: "239776"
    uid: c7b56267-6801-11e8-bf9f-52540038bbcc
  hostPath:
    path: /var/lib/minishift/openshift.local.pv/pv0059
    type: ""
  persistentVolumeReclaimPolicy: Recycle
status: {}

We need to ssh into minishift to find the files:

[user@localhost amq-perf-test-job]$ minishift ssh
Last login: Mon Jun  4 12:21:20 2018 from 192.168.42.1
[docker@minishift ~]$ ll /var/lib/minishift/openshift.local.pv/pv0059
total 144
-rw-r--r--. 1 185 root 70638 Jun  4 12:11 amq-perf-consumer-1.xml
-rw-r--r--. 1 185 root 70649 Jun  4 12:11 amq-perf-producer-1.xml

All of the code/README for this blog post is available on github at: https://github.com/welshstew/amq-perf-test-job

codergists