What’s new in open source software testing frameworks? That’s a question you may find yourself asking occasionally. It isn’t always easy to keep up with the changes in software development tools these days. The nature of continuous delivery requires developers to focus on rapid iteration updates to software, with correspondingly less emphasis on careful documentation of earlier versions and very possibly obsolete older features. The result is that, while it’s generally easy to find out about the current capabilities of an application, it isn’t always easy to find out which features are new, and which remain unchanged from previous versions.
With that in mind, here is a survey of the latest (and, arguably, the greatest) developments in three of the key open source software testing frameworks:
SELENIUM
Selenium has been and remains one of the most important software testing frameworks. Its WebDriver API serves, in fact, not just as a framework for automating browser-based tests, but also for automating browser administration in general. Many other open source testing frameworks either use WebDriver, or take it as a model for their own APIs.
For software developers and testers concerned about Firefox compatibility issues, the good news is that the Selenium IDE (for recording, editing, and debugging browser-based tests) has been completely rewritten and is now compatible with current versions of Firefox and Chrome.
The GitHub project describes the new IDE as a work in progress, and Mozilla lists the plugin as “Experimental,” so it may have a few rough edges, but it does bring Selenium IDE functionality to the contemporary Firefox universe.
SELENIUM 3.0
Selenium released version 3.0 in late 2016; version 3.13.0 is currently available for download. Selenium 3.x, unlike earlier versions, does not include Selenium Core (the original Selenium implementation). Instead, it relies entirely on the WebDriver API. (This change largely affects developers and testers who have been using Selenium RC; the RC APIs are not included in the default Selenium 3.x download, and Selenium urges all users to adhere to the WebDriver APIs, although the RC APIs are still available in a legacy package.)
WEBDRIVER AS A STANDARD
The move to WebDriver-only core functionality does much to consolidate Selenium’s position as the de facto standard for open source browser-based testing. Since WebDriver is in fact a driver-based model (making use of each browser’s own, built-in automation support), it overcomes the problems of non-driver automation methods, including the earlier Selenium-RC method, which injected JavaScript into the target browser.
Selenium project members are actively involved in the W3C Open Source WebDriver specifications, which are currently in the recommendation phase. When the specifications are complete, they should provide a standard, driver-based protocol for browser automation, making it even easier to develop and implement browser-based automated tests.
APPIUM
Appium has been a JS Foundation project since 2016; Sauce Labs donated it to the non-profit foundation in order to provide it with a home and support infrastructure which would allow it to continue as an active open source project. Appium originally brought the WebDriver model to mobile app testing, doing a great deal to make the mobile testing process easier, and at the same time, further establishing the WebDriver model as a test framework standard.
The most recent versions of Appium have moved away from the original mobile test drivers, replacing them with drivers which can take advantage of modern test interfaces. This is particularly important in mobile app testing, since the basic testing interfaces are under the control of Apple and Google, and are closely integrated with the current versions of the operating systems which they are designed to test.
THE LATEST DRIVERS
For iOS, this means that the original UIAutomation Driver has been deprecated in favor of the XCUITest Driver, which uses Apple’s XCUITest libraries. For Android, both the first UiAutomator driver and the Selendroid driver (used originally for legacy versions of Android) are deprecated in favor of the UiAutomator2 driver and the new (beta) Espresso driver, which uses Android’s Espresso automated testing technology.
Espresso is a compact, streamlined whitebox test system designed for integrating testing into the development cycle; the inclusion of the Espresso driver adds greater flexibility to the already-flexible Android interface for Appium.
SELENIFIED
Selenified (formerly SecureCI) extends Selenium’s capabilities in several directions, providing a web/API-level Java-based test framework, detailed reporting, and improved slow/missing page handling, among other things. The 3.0 release, in August 2017, was the first version of Selenified to be fully object-oriented, including support for the Page Object Model. This allowed Selenified to make more effective use of the WebDriver API.
SELENIFIED: UP AND COMING
The 3.0.1 release, in May of this year, includes some major updates, the most important of which is support for Selenium 3.x. Version 3.0.1 also includes support for multi-branch testing (via a Jenkins file), and support for PhantomJS. Along with cleaning up several minor issues, 3.0.1 also adds a variety of features, including is/waitFor location methods; findChild method for Elements; support for custom headers in API calls; the ability to retrieve a programmatically determined xpath for Elements, and the ability to run tests using Firefox and Chrome headlessly.
The initial releases of Selenified concentrated on core functionality. Upcoming releases are likely to significantly extend its capabilities, making it a test framework to watch.
DRIVER-BASED ACCESS
For all three of these frameworks, driver-based access is important. For Selenium and Selenified, drivers provide full browser integration; for Appium, they provide access to the built-in testing features of iOS and Android. In many software environments, the use of drivers leads quite naturally to a standardized, universal, driver-based interface, and this process appears to be well under way in the world of mobile and browser-based testing.
