Squish/Load Testing

Load testing with Squish

Let's assume you have an application that communicates with a separate server, let it be a web server, a Jabber server or a custom developed server only for your application. Using Squish, you can load test this server with your application as the entry point. This is done by running Squish multiple times in parallel on multiple instances of your application working against the same server.

To accomplish the load testing, you need two different scripts: the actual test script that will run on each application instance and a launch script that will start off the requested number of Squish instances.

1) The test script

The content of the test script is of course very dependent of the application you are testing, but one general concept is that you put the interactions in a loop doing a pre-defined number of iterations. For the sake of this example, we will use the fortune client and threaded fortune server available in your Qt installation. The test will simply be repetetive clicking of the "Get Fortune" button, verifying that the fortune text is updated within a specific period of time, in the below case 20 milliseconds. The script assumes the fortune server is already running.

def main():
 startApplication("fortuneclient")
 type(waitForObject(":Server port:_QLineEdit"), "41868")

labelName = "{type='QLabel' unnamed='1' visible='1' window=':Fortune Client_Client' occurrence='3'}"
 waitFor("object.exists(labelName)", 20000)
 label = findObject(labelName)

prevText = label.text

for i in range(100):
 clickButton(waitForObject(":Fortune Client.Get Fortune_QPushButton"))

time = QTime()
 time.start()
 waitFor("label.text != prevText")
 elapsed = time.elapsed()
 test.verify(elapsed < 20)

prevText = label.text

clickButton(waitForObject(":Fortune Client.Quit_QPushButton"))

2) The launch script

This script can be written in any scripting language of your choice. The main purpose of it though is to launch ONE Squish server and an arbitrary number of Squish runners all running the script above. The script below does that, in Python, launching 50 Squish runners.

#!/usr/bin/env python

import subprocess

SQUISHDIR="/path/to/squish-4.1.0-qt-src"
TESTSUITEDIR="/path/to/suite_loadTesting"
TESTCASE="tst_case1"

subprocess.Popen(["s/bin/squishserver" SQUISHDIR, "—daemon"])

for i in range(50):
 subprocess.Popen(["s/bin/squishrunner" SQUISHDIR, "—testsuite", TESTSUITEDIR, "—testcase", TESTCASE])

Discussion

One problem with the above approach is that the test script will launch immediately when the Squish runner is launched, which is normally what you want. However, this means some of the application instances are finished by the time other start. This can be worked around by having the test script wait for a specific file to exist before entering the loop, and having the launch script create this file after all Squish runners has started. That way, the actual load testing won't start until all the clients are ready.

You may also want to introduce random snoozing between each iteration in the test script, to even out the load on the client machine and also making the load on the server machine more "human".

If you want to run more application instances than what is feasable on a single host, using Squish's remote testing you can execute multiple application instances on multiple different hosts. This of course still means running the Squish runners on one machine, but at least you reduce the load introduced by the application you are testing. An alternative of course is to execute the Squish runners remotely over e.g. SSH on multiple machines to spread the load even more.

Technically, you could be using multiple Squish servers (e.g. one per application instance), but as a single server is handling multiple Squish runners quite well, there is really no need for it, and it will actually increase system load on the client machine.