Using Selenium for Performance Automation
Coding automated test suites for Performance using freeware tools is frequently frightening,
but in certain occasions one can accomplish the feat and walk back home unharmed. We like to think that the case covered in this post is a good example of such.
The AUT that demanded Performance Testing is dedicated to manage patient surveys from Medical Institutes and as part of that goal, it provides the ability to search the surveys filtering and organize them based on diverse criteria.
The overall challenge we faced was presented by the client need itself. They needed to measure “Performance” for the web tool. Therefore, we needed to define “Performance” in a context (i.e: request-response? Page load times? For how many users? Against which baseline?). Given that they already had a system to measure times for request-response on the Database and Server sides, we soon found that what they were not able to tell yet was how/when the data and graphs were actually displayed in the page and ready for the user to interact with them, either by filtering or exporting the results. We then decided to go with tools that do better in interacting with websites themselves, rather than triggering and handling requests.
Before we could shortlist the candidate tools, we went on with defining the number of users needed for this Performance test – as you probably know, depending on the amount of threads you are willing to run, some tools may be more work than others. Luckily, the pool of active users was very limited, and we could live decently above the standard level by running 25 threads simultaneously.
Our initial shot was proposing solutions that combined both Performance and User Interface tools, such as HP Load Runner and HP Quick Test Pro. As usual, the pricing of licenses for such applications was a deal-breaker. Therefore, we decided to research on a way to build a solution for our client’s need by just using Open-Source tools.
Based on the experience and the nature of the client’s site (developed with WebLogic), the selected tool was Selenium Web Driver and its Page Object Pattern.
The first stage of our development process was defined and then focused on:
- Generating the workflow
- Triggering responses with actual data to avoid empty searches
- Identifying the pages as objects of the workflow
- Locating the Objects inside the Page Object Pattern
Our first challenge during development arose when we noticed that WebLogic started to nest iframes as part of the standard behavior in the AUT. The problem was that Web Driver usually identifies an iframe as a totally independent web page, no matter where it is located. As a result of this issue we had between 5 and 8 page objects for each actual page. Getting the focus on different iframes implied that we had to abstract from the model, and dig iframe by iframe in order to get to the specific element we wanted to interact with. Specific questions on how? Please ask in the comments, we don’t want to swamp this post with code!
The second challenge popped up when we asked whether there were “standard” reports that were known to be either frequently used OR putting pressure over the database, we realized that there were no statistics on the type of queries yet. We then decided to pick a wide range of options to cover all the available clients and filters so as to be able to find whether certain combinations might be more demanding than others. Once those criteria were agreed upon, we decided to implement them in a way that would provide flexibility to our solution, allowing the client to edit, copy and replicate the input data. Such a goal was accomplished by developing an input XML file with a defined structure. Such XML file not only provides input data, but also specifies run-time parameters, such as which environment the test is going to run against. Given that the structure is simple and straightforward (but still a bit complex to manually replicate it), a standalone application capable of turning input test criteria into said structure is scope for future releases.
After solving these two issues, the script was capable of making searches and sort the results by provided criteria through the input file (XML).
Our third challenge was how we capture and transmit those results to provide visibility to our client.
Our choice was implementing a simple CSV file (comma separated values) in which we recorded the time elapsed for the different stages of the workflow (login, open search window, set criteria, sort results, export to PDF). With the help of any spreadsheet processing program, we added any calculations proposed by the client for them to get the basics, such as average/maximum/minimum/90% times for each action, as well as the scenarios associated with them. Note that fancy line and bar graphics are easy to add once all the information is compiled.
The structure stood the frequent scope changes that were done in the User Interface. Whenever a new feature or behavior was introduced, we tried not to be too concerned about the issue itself, but more on expanding the whole page object structure to handle more levels/filters/user types. Therefore, if a new tab was added, we would be able to inspect the whole page at a high level, and just worry on how to get to interact with the new object inside it – as well as verifying that the other objects in the same layer were still accessible. This way, our Page Object remained uncorrupted and reliable. This confirmed that the more reliable your page object is, the easier is to face an eventuality in the future.
Finally, during runtime, we compiled the whole code in standalone *.jar files that would run on as many different Virtual Machines as user threads you would like. As you can imagine, this was a task that demanded some time and a decent amount of human intervention, but well worth the price saved in multiple licenses.
So, if you ever need to build Automation Frameworks for sites developed with WebLogic using freeware tools, Selenium is a big option for you to consider, as long as you devote the necessary time to narrowing the objectives and designing the Page Object Pattern you are going to rely on.