What are the limitations of Selenium? (Ryo Chikazawa)

Selenium is an industry-standard test automation tool many are familiar with. Selenium is open-source software that offers various tools for different use cases. Those tools are WebDriver, IDE, and Grid. WebDriver is intended for advanced users for writing test scripts. IDE offers an easy-to-use interface to record users’ interactions with the web browser and playback test scripts. Grid offers virtual and physical device support for users to run simultaneous tests at once. With all of these toolsets, the software does have its drawbacks. Developers have repeatedly expressed frustration with the limitations of Selenium.

Takahiro Yasuda, an Executive Officer at SMN Corporation shared with us his experiences with Selenium. He told us his company tried to implement the open-source automation testing tool Selenium about five years ago. “We got to the point where we could start using it, but keeping up with subsequent development and continual maintenance became too labor-intensive,” explains Yasuda. He continues, “We were releasing once or twice a month at the time. The time it took to maintain Selenium and the cost of allocating skilled staff didn’t make financial sense. Of course, I’m not saying Selenium isn’t a good tool for test automation. It’s that we had a negative experience with it.”

Tomokazu Manabe, an engineer on the quality assurance team at Carview Inc. explains his team’s frustration with the tool. Citing, “development and maintenance were very time-consuming. Maintenance would take ages when a program or screen changed because the automated test needed to be edited as well.”

Seiji Tsukioka of ZOZO Technologies explained to us, “every time there was a modification in the program, Selenium had to be modified as well.” When we asked about how this associated with man-hours he explained, “management and operation costs are quite high, such as server maintenance of Selenium’s operating environment, and there were only a few people who could do these tasks.” He further elaborated, “In other words, the major issue was that the test environment was becoming more and more specialized, and I had always wanted to solve this issue.”

If you are a developer using Selenium, we are sure you can attest to the same frustrations. We have compiled a list of most of these hindrances, plus offer solutions to get around these restrictions with the popular tool.

What are the limitations of Selenium?

No desktop support, only web browser support. Selenium cannot automate testing on desktop apps, yet limited to web apps. You cannot operate outside of the browser. For example, it is quite frustrating if you are testing a feature to open a local file and upload it.

Veronica Miller is a Cybersecurity Expert at VPNoverview explains, “Selenium does not have support for desktop application research.” Eric McGee is a Senior Network Engineer at TRG Datacenters elaborates, “Windows programs are not supported by Selenium. Selenium only allows web-based software, implying that it can only be used to research websites.”

Selenium is also limited from testing mobile apps too. “Selenium can be used to test on any operating system and browser on the desktop, but it cannot be used to test on handheld devices,” describes Eric McGee.

Coding required. Selenium WebDriver requires coding to “write” test scripts that simulate real user interaction with the web browser. This raises the barrier of entry for non-engineering testers. It also places more burden on developers to code for testing when their efforts can be channeled towards new features and innovation. Cybersecurity Expert Veronica Miller concurs, “Selenium needs the team’s experience as well as money to handle.”

Lack of built-in image comparison. By default, Selenium doesn’t have a built-in image comparison solution. You must seek third-party solutions for this feature.

Lack of tech support. Being that it is open-source, there is no dedicated support ticketing system. You must resort to forums, chat rooms, or support via Slack. In its defense, there is a large community to tap into for help… just not directly from the makers.

Lack of reporting capabilities. You must seek third-party add-ons to gain reporting functionality. It is not built directly into the automation tool.

Maintenance nightmares. One of the greatest frustrations many developers gripe about while using Selenium is maintenance. At scale, it can be cumbersome. Luckily there is help to avoid Selenium maintenance headaches.

Even more limitations…

Those are the major issues most developers have with Selenium. However, there are more nuisances as you dive deeper into use cases with the tool…

• No native support for testing mobile apps. However, you can use Appium and Selendroid to test native, hybrid, or mobile web apps.
• Lack of handling dynamic elements. Since applications are dynamic and can change, relying on web elements such as IDs can prove problematic with automated testing. It’s best not to rely on IDs, here’s why.
• Handling dynamic page loading. There may be a form on the page that changes based on a previous action. For example, if selecting a country from a drop-down list then the next drop-down lists states or providences associated with the country. To resolve this, you must add wait conditions in Selenium as the initial script may not load the dynamic condition.
• Lack of image locating. You must add-on Sikuli to locate an element of an image.
• No native support for Captcha and OTP (one-time password.)
• No support for REST and SOAP platforms.
• Limited support for handling operating system popups.
• Cannot modify HTTP headers.

Developers have shared some real-world examples of how Selenium has limited testing of its applications. Artur Shaikhutdinov is a QA Automation Engineer at Deutsche Bank (Moscow.) He shared, “It’s slow. It will start clearing when the browser finishes loading the site. You will be wasting time and internet bandwidth on unnecessary CSS and JS files.” Also, if browser settings are changed, “It cannot be guaranteed that browser settings will always remain the same after a clean install. For example, you might accidentally change the proxy server address or security settings. This can lead to the failure of previously running tests.”

Senior Network Engineer Eric McGee, shares “The design of certain web elements is complex. It will be difficult to handle this in the usual way if an element’s ID changes on every page load. The dynamic elements must be handled using dynamic XPath or dynamic CSS selectors. Functions such as starts-with, includes, ends-with, and others are useful for dealing with dynamic objects.”

How to solve Selenium limitations?

According to research, DevOps QA teams are not satisfied with their tooling and search for better software multiple times per year. And based on the lengthy laundry list we’ve checked above- it’s apparent why they’re simply not satisfied.

Users have found Autify to be the best alternative to Selenium.

Although it does not support testing desktop applications, regarding web and mobile apps, it is superior in many ways. First, it is a codeless test automation platform. Therefore, the learning curve is far lower because a tester simply uses the browser to test their application. Autify records their interactions and automatically “writes” test scripts on behalf of the user. And if they need expanded functionality, they can append screen recordings with Javascript.

Second, the testing tool automatically saves screenshots of each step by default. It relies on visual testing for end-users rather than code-only presentation layers. And third, since it is a freemium solution, you gain premium technical support with the full subscription.

Reporting is not an afterthought, it is built-in. The platform’s reporting feature also presents side-by-side comparisons of failed tests for faster detection of bugs. Since it is powered by artificial intelligence, the tool can adapt to frequent user interface changes. Maintenance, too, is handled by AI. This means the plat