MQTT Test Suite

Quickstart Installation Guide

Thank you for your interest in the IWL MQTT Test Suite. This is the Quickstart Installation Guide.

For complete information on running the IWL MQTT Test Suite, please refer to the MQTT Test Suite User Guide. Click on the "Help" button on the MQTT Test Suite web user interface to access the User Guide.

Follow this sequence of steps to get the MQTT Test Suite up and running:

  • Install Docker

  • Install the MQTT Test Suite Docker image file

  • Decide on a Cloud License or an On-Premise License:

    • Set up the Cloud License

    • Set up the On-Premise License Server and then the License File

  • Start the MQTT Test Suite

  • View the MQTT Test Results

  • Capture MQTT Test Messages

  • Stop the MQTT Test Suite

Installing Docker

The IWL MQTT Test Suite is distributed as a compressed Docker image file.

You can run the IWL MQTT Test Suite on Windows, Mac, or Linux where Docker is supported.

First, install Docker

https://docs.docker.com/get-docker/

Second, test that your installation works by running the hello-world Docker image:

   $ docker run hello-world

      Unable to find image 'hello-world:latest' locally
        latest: Pulling from library/hello-world
        ca4f61b1923c: Pull complete
        Digest: sha256:ca0eeb6fb05351dfc8759c20733c91def84cb8007aa89a5bf606bc8b315b9fc  

 Status: Downloaded newer image for hello-world:latest

     Hello from Docker!

The message above indicates the installation appears to be working correctly.

Third, execute the following command to list the downloaded image.

   $ docker image ls

Fourth, list the hello-world container (spawned by the image) which exits

   $ docker ps –all
    CONTAINER ID     IMAGE           COMMAND      CREATED         STATUS
    54f4984ed6a8     hello-world     "/hello"     20 seconds ago  Exited (0) 19 seconds ago

The recommended Docker version required is Docker 19.03.13 and later releases.
Execute the following command to check the version of Docker installed on your machine:

         $ docker --version

Installing the IWL MQTT Test Suite Docker Image

Login to https://download.iwl.com/

Navigate to download the IWL MQTT Test Suite Docker Image.

The name of the MQTT Test Suite image file is usually “com.iwl.pte_dist_mqtt.txz”.

  • Open a command line window

  • Go to the folder where the downloaded file "com.iwl.pte_dist_mqtt.txz" is stored

Enter the following command. (It may be necessary to give the full path to the PTE image file.):

 $ docker load --input com.iwl.pte_dist_mqtt.txz

You should receive a confirming message that the image has been loaded.

You can double check that it has been installed by typing the command:

 $ docker image ls

You should see a response that looks something like the following (there may be additional lines):

 REPOSITORY                      TAG       IMAGE ID        CREATED        SIZE
 iwl.com/com.iwl.pte_dist_mqtt   latest    1aab5c8be23c    3 days ago     355 MB

Now that the MQTT Test Suite is installed, you will need to "unlock" the Test Suite by installing a License Server and/or License File.

Executing the MQTT Test Suite requires a license file from IWL.

MQTT-Premises-Install-Diagram-1.png

Set up the On-Premises License

If the computer on which you plan to run the MQTT Test Suite does not have Internet access to the outside, then you need to install IWL License Server locally and obtain a customer premises license file from IWL to run the MQTT Test Suite.

The IWL License Server can be installed on Windows 10, macOS (10.13 High Sierra and above), or Linux (64 bit Fedora 7+, CentOS 6, RHEL 6, Ubuntu 14, etc.).

If you have purchased a single license of the IWL MQTT Test Suite, you can simply install IWL license server on the same computer where you installed IWL MQTT Test Suite Docker image.

If you have purchased multiple licenses of IWL MQTT Test Suite, you may choose to install IWL license server on a separate computer that would stay on constantly. This way all other computers on which the IWL MQTT Test Suites would be running can always access and check out licenses from it.

Log in https://download.iwl.com/ Navigate to download IWL License Server installation package for Windows, Linux or macOS.

Install the IWL License Server

  • To install the IWL Windows License Server, click on the downloaded self-executable installer to install.

  • To Install the IWL Linux and macOS License Server, extract the package from the downloaded gzipped tar file.

For example:

 $  tar -zvxf IWL-License-Server-2020.10.07-macos-64.tar.gz            

Get the hostID of your license server machine

Now move to the newly created directory. For example, on Windows:

 C:\  cd “C:/InterWorkingLabs/IWL FortLM License Server”

Then run start.bat on Windows or start.sh on Linux and macOS.

Running start.bat or start.sh shell scripts will print out your host ID.

For example:

 C:\InterWorkingLabs\IWL FortLM License Server>  start.bat  
  The hostid of this machine is 8022c723
  ...

Email this host ID to license@iwl.com and a license file will be generated and emailed to you.

Install the license file you received

First, open the file named fortlm_license.lic that was emailed to you in a text editor.

Next, edit the first line (SERVER= line):

 SERVER=hostname_or_IP; PORT=port_no; IP=v4; ANYIF=No;  

Where "hostname_or_IP" should be replaced with the host name or IP address of the license server computer.

Replace "port_no" with a free port number. You should pick a port number > 1024 to avoid using privileged/restricted ports.

For Example, after editing, if using an IP address, the first line may look like:

 SERVER=192.168.10.2; PORT=4545; IP=v4; ANYIF=No;

If using a host name, that line may look like:

 SERVER=calypso.myorg.com; PORT=4545; IP=v4; ANYIF=No;

Keep other parts unchanged and then save_ fortlm_license.lic _as a text format file in the top directory of the installed license server package.

On Windows, the default directory is “C:/InterWorkingLabs/IWL FortLM License Server”

Note: You need to change ‘IP=v4;’ to ‘IP=v6;’ if you are running on a IPv6 server. If you change ‘ANYIF=No;’ to ‘ANYIF=Yes;’ then the server will listen on any interface on this server computer.

Start the IWL license Server

To start the license server, from the top directory of the installed license server package, run start.bat on Windows or start.sh on Linux and macOS.

For example:

 C:\InterWorkingLabs\IWL FortLM License Server>  start.bat

If the installation was successful, you will see a message that indicates the license server has been started successfully.

Select Command Prompt - start.bat

c:\InterWorkingLabs\IWL FortLM License Server>start.bat

c:\InterWorkingLabs\IWL FortLM License Server>echo off
The hostid of this machine is
C4B301D38DCC

 Attempting to start license server
 NOTE: If the license server has been installed and started automatically as a system service when the computer starts then this will fail!

FortLM license server 2.7.10 x86 - Copyright (c) 2012 - 2020 by SegueSoft Inc. All rights reserved.
Found SERVER 192.168.10.126 and Port 5678 specified in c:\InterWorkingLabs\IWL FortLM License Server\fortlm_license.lic

1. validating license c:\InterWorkingLabs\IWL FortLM License Server\fortlm_license.lic ...
Remaining valid days: 720
License verification succeeded ...
For feature sets: MQTT
    Available seats => unreserved: 1

Attempt to start server on host=192.168.10.126 port=5678
Listening (IPv4) on port 5678 Ok ...

Loaded Feature sets and License Summary:
Feature sets: MQTT: licenses out: 0, remaining: 1 (public: 1, reserved: 0)
Server started on Fri Oct 23 12:27:11 2020

NOTE: Instead of running the start.bat or start.sh, the IWL License Server can be started by executing the following command:

 $ /path-to-server/flmserver -f /path-to-server/fortlm_license.lic /path-to-server/iwl_pubkey.pem

The ‘-f’ option is used to force starting a new instance of license server by removing active license server instance lock (if any).

For additional information on how to start the IWL license server automatically when the license server computer starts please see README_FORTLM.txt.

On macOS, you can add the start script to the Launch Daemon.

Starting the IWL MQTT Test Suite

First, start the MQTT Test Suite Docker container and then start the Web browser User Interface (UI).

Starting the IWL MQTT Test Suite Docker Container

To start the MQTT Test Suite, open a command line window and create a working directory. In this example we name it “mytest”.

If you are using TLS you can place your certificate files here as well.

 $ mkdir mytest
 $ cd mytest

Place a copy of the fortlm_license.lic in this working directory . To start the IWL MQTT Docker container on Linux or MacOS, enter the following command into a run.sh script file.

Note you need to change ‘full-path-to’ to match your settings and the command should be typed in one line.

The test results will be saved in the ‘mytest’ directory mapped on your local computer, since test results on Docker containers won’t persist.

The mapped port 35002 can be any free port on the host computer, but the docker container’s listening port must be 8080.

 $   docker run -it --rm --name mqtt1 -p 35002:8080 
      -v /full-path-to/mytest:/usr/local/pte/lib/test_results 
      -v /full-path-to/mytest/fortlm_license.lic:/usr/local/pte/lib/testsuites/fortlm_license.lic 
       iwl.com/com.iwl.pte_dist_mqtt

On Windows, enter the following command into a run.bat script file. Note you must specify the drive letter in the path in your run.bat, for example:

 C:\> docker run -it --rm --name mqtt1 -p 35002:8080 
     -v C:/Users/bob/mytest:/usr/local/pte/lib/test_results 
     -v C:/Users/bob/mytest/fortlm_license.lic:/usr/local/pte/lib/testsuites/fortlm_license.lic          
     iwl.com/com.iwl.pte_dist_mqtt

Note the command run.sh and run.bat mentioned above can be downloaded from IWL download site (run.zip) after logging in with your user name and password.

After unzipping the downloaded run.zip file, you will find a 'run.bat' and a ‘run.sh’ file.

On Linux and MacOS make sure the run.sh has proper exec permission set. The placing a copy of the fortlm_license.lic file into the same directory as your run.bat (or run.sh), open a terminal, and type run.bat on Windows or run.sh on Linux and MacOS. You should see the the IWL MQTT Test Suite Docker container start running.

You should see some output that looks something like the following:

#**********************************************************************
#                         PTE by IWL                                                                  
# Copyright (c) 2018 InterWorking Labs, All Rights Reserved                
#              RESTRICTED RIGHTS LEGEND                                                 
# This software is provided with RESTRICTED RIGHTS.                           
#**********************************************************************
Invocation:
/usr/local/pte/etc/pte_dev_start.sh -H ptehost.example.tld -P 1883
Environment:
HOSTNAME=28ac1f03493c
PTE_SVN_VERSION=244
…...............
Default Device Under Test (DUT) name: ptehost.example.tld:1883
PTE Startup
This would be where we run in background
/usr/local/pte/lib /
Starting PTE in PRODUCTION Mode
testResDir: /usr/local/pte/lib/test_results
Listening on \*:8080

Congratulations! You now have the MQTT Test Suite backend set up!

Next you will go to a web browser to bring up the graphical user interface for the MQTT Test Suite.

Starting the IWL MQTT Test Suite Web User Interface

The IWL MQTT Test Suite should now be visible via a Web browser at:

   http://Host-IP-Address:35002/

The IWL MQTT Test Suite Web User Interface (UI) can be accessed using a standard web browser (Chrome, Firefox or Safari). For example:

   http://192.168.10.133:35002/

You should see IWL MQTT Test Web UI come up.

MQTT GUI.png

Click on “MQTT Tests Setup” tab and fill in the following details:

MQTT Test Setup.png

Server / Broker : The name / IP address of the server (broker) to connect to

Port : The server listening port, default to 1883

Enable TLS : If checked, using TLS/SSL. You must enter CA cert, client cert and client key.

TLS/SSL Port: The server listening port, default to 8883 when TLS/SSL is used

Username : You may not need a user name if TLS/SSL is used or for MQTTV5

Password : You may not need a password if TLS/SSL is used or for MQTTV5

Socket Timeout: Socket connection timeout seconds

Test Wait Time : Seconds to wait for messages before terminating tests

If you are using TLS/SSL, then you just need to enter the appropriate values for the following:

  • CA cert File : Certification Authority Certificate File (PEM format, default extension .crt)

  • Client Cert: Client Certificate (PEM format, default extension .crt)

  • Client Privkey File: Client Private Key (PEM format, default extension .key)

Click on “Connect V3” to check if the MQTT V3.1.1 broker under test can be accessed using the supported credentials.

Click on “Connect V5” to check if the MQTT V5 broker under test can be accessed using the supported credentials.

Click on *Logging Option” tab to select logging level.

INFO: The default level for normal messages.

DEBUG: Used for debugging purposes. MQTT message data Provides a log of the traceback (the place in the code where the failure occurred)

Now click on the "Tests" tab to run some tests.

MQTT Run Tests.png

Click on any cell in the Test Name column to run a single test.

Click on any cell in the Test Description and Result column to view the test description.

Select tests and click on the "Run Selected Tests" icon to run multiple tests:

Run Selected Tests Icon.png

Viewing the MQTT Test Suite Results

After the tests have completed, to view test results, click on one of these icons:

View Test Results Icon.png

The first will give you the last test report, and the second will give you the last test log.

The third will let you view test results saved in XML and HTML test reports for all previous test runs.

The last lets you delete test results.

Note the test results will be saved in the directory ‘myresults’ under your working directory ‘mytest’ on your local computer.

Each time you run tests, it will generate a new copy of report.html, log.html and output.xml for the last test run.

The results of different test runs are stored in a timestamped directory under the directory 'myresults'

Capturing MQTT Test Messages

You may want to capture the MQTT Traffic while a test is running.

IWL MQTT Test Suite runs in a Docker container that uses a network bridge for all traffic. By default it will be using bridge named docker0.

You can then use Wireshark to capture traffic on the interface docker0.

If you are not using the default setting, you can use commands such as ip addr show or ipconfig on Windows to find out the correct interface for your container.

Stopping the IWL MQTT Test Suite

The following command lists all launched instances of the IWL MQTT Test Suite:

docker ps -a

You should see a response that looks something like the following:

CONTAINER ID  IMAGE                              ...  STATUS  ...  NAMES
7fa0a97f7ebc  iwl.com/com.iwl.pte_dist_mqtt      ...  Up        ...  mqtt1

Stop the instance running above -- "pte_dist_mqtt" -- using the command:

$ docker stop mqtt1

Remove a stopped container if we haven't used the '--rm' option when launching the MQTT Test Suite. Otherwise it will be removed automatically when stopped.:

$  docker rm mqtt1