<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=1195114&amp;fmt=gif">

Mendix ATS 
Webinar series

Answers and questions of
the live Q&A's

We're happy to share with you the most frequently asked questions during the virtual training series. Of course, we are happy to answer more questions, so don't be shy to reach out.

Frequently Asked Questions

Can we deploy ATS in Azure cloud?

ATS is a SaaS tool. You do not have to deploy ATS anywhere it’s already running in the Mendix cloud and all you need to do is log in with your existing Mendix account.

Does ATS yet support testing of native mobile?

No. ATS supports testing of responsive web apps on real mobile devices. Testing of hybrid or native mobile apps is not supported. We recommend using Appium or other mobile specific testing technology.

Did the PO say he was going to use ATS to test in his production environment?

Yes. The reasons for this are to do availability and smoke testing. Many Mendix apps that use ATS have a test case or suite that every 10 minutes checks that the main user flows in their app are working as expected on production. So definitely there is a case running a test case on production and this is different from running a functional or regression test on the TEST environment.

Can tests be performed in parallel?

Yes, ATS supports grouping multiple tests together in a test suite. The tests in the test suite can be set to execute in parallel. The number of parallel sessions depends on your selenium provider of choice.

I did not see you (the presenter) ending the recording. How does that work?

The presenter started and stopped the recording from ATS by clicking on the respective buttons.

Is it possible to upload files via ATS?

Yes. Uploading a file is as simple as setting the name of the file in the file choose dialog. The tricky part lies in getting the file on the selenium server on which the test is running. Support for this varies across the different selenium providers, we refer you to check their documentation for exact details.

Is there a mechanism to clean up test-data?

Yes. Test cases support a tear-down phase where data created during the test can be cleaned. Tear down phase will always execute regardless if the test passes or fails.

How many hours of recording ATS supports?

There is no limit per se. Video recordings are kept for a duration depending on your chosen provider, and they can also be downloaded for long term storage.

Can you schedule all test cases at once?

Yes you can schedule as many test cases as you want as often as you want. Test cases can also be grouped in suites which can then be run/scheduled as a group.

Can you export Selenium scripts?

No. Similarly to how it is not possible to export java code from Mendix apps it is not possible to export selenium scripts from ATS. ATS is model driven and the logic is runtime and context dependent. In fact, most of the power of ATS comes from the awareness of the execution context in which the test is running. However, it is possible to export a complete definition of the test case in XML format, similarly to how you can use the Mendix SDK to export a json model of a Mendix app. Theoretically you could then transform this XML into an executable.

How do I connect an app to the test suite?

After you purchase an ATS license for a specific Mendix app that app will be onboarded to ATS automatically. All you need to do is log in to ATS with your existing Mendix account. Then you will be able to see your app, write or record test cases and execute them.

When using datasets is it possible to only run specific rows from the dataset, e.g. row 2 and 4?

All the rows have to run. If you want to use only some rows you can create another dataset with only those rows.

Sometimes I see the video of the test does not load. Do you know what causes this?

The live view is only available with the built-in selenium (ATSelenium).
It is only offered as a best-effort service due to the complexities involved in streaming live video over IP at scale.

Can get outcomes/values from 1 test case and use this value in the next test case?

No. Tests are designed to be standalone, which is a common testing practice. This lets us run test independently of each other and in parallel if desired. It is possible to return a value when using a custom action. Custom actions fulfil a similar role to sub-microflows in Mendix.

Can you schedule a test case/suite to runs on repeat (instead of once)?

Yes. When setting up a schedule you can configure if it should run once or repeat for e.g. every day or at other time intervals.

Sometimes mx-name is not unique (multiple buttons with the same name on 1 page). Is there a way to tell ATS which one to pick, without giving the widget a unique name?

Yes, this can be done by using a unique path of mx-names. All ATS functions that support a Widget Name allow for multiple widget names to be entered separated by a space e.g. “headerContainer nameText”. ATS will then look for a widget with the name “headerContainer” and for a widget “nameText” inside it.

Consider there are two buttons with the same name in the page or popup. I would like to click the second button using ATS instead of first one. How to achieve?

You have multiple options to deal with this. One option is to find a unique path to that widget. ATS recorder normally takes care of this for you. It might also be good to know that when a dialog(s) is open ATS will always prioritize elements in the active dialog. Alternatively you can use a search context or a sizzle selector. If you are using a sizzle selector you can also specify which occurrence to pick. And the most obvious way to fix this would be to change the mx-name in the modeler or add a unique class that you can use in a selector.

Does ATS run on its own server? Or on the Mendix Cloud? Can it run on-premise?

ATS runs on the Mendix cloud and follows the SaaS model. That means that you do not have to provision a server or a VM and install anything. You can log in with your standard Mendix account and start using ATS.
For On-Premise options please contact our sales dept at sales@mansystems.com.

Will ATS support different browser versions in the future?

We are considering adding support for a few different browser versions. However, keep in mind that the goal with ATS tests is to do functional testing on your business logic. Mendix as a platform is responsible to ensure that all standard components work across different browsers and browser versions.

Will ATS support Safari Browser in the future?

There are currently no plans to extend support for any more browsers. Again the goal with ATS tests is to do functional testing on your business logic. Mendix as a platform is responsible to ensure that all standard components work across different browsers and browser versions.

When you extract an action can you also cancel that?

No, the extraction happens as soon as you click on Extract. The dialog only lets you specify a user-friendly name for the action.

Can I receive alerts when a test case fails?

Yes. You can subscribe to alerts in the schedules page. For each schedule you can also configure whether it triggers alerts.

What if the building blocks in ATS do not support my use case?

ATS offers two alternatives for such cases. 1) ATS exposes many low-level selenium functions as building blocks, these can be combined in a low code way to support your use-case. If even this is not enough ATS supports javascript as a scripting language with special javascript actions.

Does ATS support API testing?

No. The primary focus of ATS is on UI testing. However, in some cases we have seen users exposing the API forms in the UI and testing them with ATS. We recommend using an API specific testing technology such as SoapUI for API testing.

How many tests can ATS run concurrently?

This is normally limited by the selenium provider. By default, the built-in selenium, BrowserStack and SauceLabs come with a single parallel session. You can arrange more sessions with the respective provider, but the price for this goes up fast. To run a lot of parallel tests ATS will soon release a load and performance testing feature, where many tests can be executed in parallel with the goal of checking the performance of your application. This will be done on a scalable Kubernetes cluster provided by Mansystems.

Can I test for the absence of a widget or a row in a data grid?

Yes, this is achieved with the negate option. Many assert functions support this option, which negates the assert condition. So Find Widget would normally fail if a widget is not found, a negated version would only pass if the widget is not found.

Is it possible to check the font (size), positions of widgets in the design/colors?

Yes, this is technically possible. ATS exposes many selenium functions such as Get Property Value to check the various properties of web elements on the page. That being said, the focus with ATS is on functional testing of the business logic of your app. If UI position and style is important to you, it might be useful to pair ATS with a visual testing tool such as Applitools.

Can I get the value of the message in a dialog?

Yes, you can use the ATS function Get Dialog Message Text for this exact purpose. This function returns the entire text of the dialog which you can then use in subsequent messages to assert against or to perform string manipulation.

Do I need CSS knowledge to test the custom widget?

Not necessarily. Depending on the complexity of the custom widget it might be possible to test without any in-depth css knowledge, by using the available ATS functions. However, for some complex widgets, e.g. calendars, or kanban boards resorting to sizzle or XPath selectors might be the only option. In that case knowledge of CSS selectors is desirable.

Does ATS support usage of functions to generate input data on the fly? ex: make use of today’s date or generate random numbers?

Yes. This can be done with the functions in the Generator category. You can use these to generate random strings, numbers and dates.