Getting started with XCUITest framework for testing iOS apps.

A Software Tester at Novoda who does his best to deliver highest quality iOS and Andriod applications.

Xcode comes with the XCUITest framework which allows you to easily develop UI tests that reflect users' interaction with the application. Find out how to kick off with the XCUITest framework to grow your automated UI test suite.

Background

In November 2016, I joined my first iOS project as a tester. It was a time when my project team had upgraded to Xcode 8, and started with the refactoring from Swift 2 to Swift 3.

With Xcode 8, Apple has deprecated UIAutomation in favour of the XCUITest framework. What's more, at that time Appium's support with Xcode 8 was far from ideal and required some workarounds to inspect elements of the iOS application.

Looking at the above, I decided to give the XCUITest framework a try and below you can find my findings.

What is UI testing?

In simple terms. UI testing is interacting with an app's UI elements by tapping, swiping, scrolling and verifying behaviour.

Automating UI tests can save you plenty of time, especially during regression testing, but at the same time automation testing needs to simulate the human's behaviour which is not always straightforward.

Thankfully, Xcode provides you with the XCUIElement class which allows gestural interactions such as:

func tap()  
func double​Tap()  
func two​Finger​Tap()  
func tap(with​Number​Of​Taps:​ UInt, number​Of​Touches:​ UInt)  
func press(for​Duration:​ Time​Interval)  
func press(for​Duration:​ Time​Interval, then​Drag​To:​ XCUIElement)  
func swipe​Left()  
func swipe​Right()  
func swipe​Up()  
func swipe​Down()  
func pinch(with​Scale:​ CGFloat, velocity:​ CGFloat)  
func rotate(CGFloat, with​Velocity:​ CGFloat)  

Xcode setup for UI testing

Create UI testing target.
If you have an existing project and would like to add automated UI tests to it, first you need to create iOS UI testing target. This is how you do it.

  1. Open your Xcode project.
  2. Go to: File -> New -> Target
  3. From the window Choose a template for your new target: select iOS UI Testing Bundle and hit Next:
  4. From the window Choose options for your new target: select your Team and Target to be tested
  5. Select Finish button and new test target has been created.

Create UI test file.
1. Pick the location in your Project navigator where would you like your test file to be created.
2. Right-click and select New File...
3. From the window Choose a template for your new file select UI Test Case Class and hit Next button.
4.In the Choose options for your new file: window provide class name and hit Next button.
5. Select the location where you want the file to be created and hit Create button.
6. You have just created your UI test class with setUp, tearDown and testExample methods.

Remember: Your test methods always have to start with the "test" word, otherwise you won't be able to run them.

UI Recorder

Once you are set up with the target and have created your test method you can start using the handy UI recorder which will generate the test script and identify the UI elements for you.

Recorder works with both simulators and physical devices, although you might sometimes find it easier to identify the UI element to use one or the other.

To kick off recording the tests go to your test method and hit the red dot button placed next to the Debug area.

Once you press the button, your application should be launched on simulator or device depending which one you have selected in Xcode. From now on, every interaction with the application will be recorded and the test script will be generated in your test method including your UI elements' identifiers.

Here’s an example of a test where user taps on the email field and types their username to login to the app:

let emailAddressTextField = application.collectionViews.scrollViews.otherElements.textFields["Email address"]  
        emailAddressTextField.tap()
        emailAddressTextField.typeText("bart@novoda.com")

Xcode alert.
While recording your tests and interacting with the app I experienced the Xcode bug saying "Timestamped Event Manager Error: Failed to find matching element".

That error means that the Xcode has not found the UI element that we interacted with. At that point the recording stopped automatically. For me the solution was to simply start recording again and interact with the element.

Run your tests

There are couple of ways to run your tests.

1.In Xcode go to Product -> Test (cmd+u). This will run all the tests in the project including unit tests.

2.If you want to run only the UI tests go to Test navigator. If your UI tests are located in one folder you can run all of them by pressing the play icon next to the folder where your tests are located.

3.There might be a situation in which you won't want to run all your tests. If you would like to run them individually go to the test class. Next to your test method there should be a diamond icon visible. When you click on it, your test will start.

Make your tests more reliable and stable

Every test has its expected output which should be verified by the test itself. As the UI tests are extend the already existing XCTest class we have the access to all the normal assertions e.g. XCTAssertEqual, XCTAssertTrue, XCTAssertNotEqual and many more.

Assert if an element exists. This only takes these two lines of code.
Let's say you want to make sure that the Login button exists on the login screen.

let loginButton = app.staticTexts["Login"]  
XCTAssertEqual(loginButton.exists, true)  

Assert if an error message displays the correct value.
Let's say you want to make sure that when the user enters the wrong password, the correct message will display.

let tooShortPasswordMessage = app.staticTexts["Password must exceed 3 characters"]  
XCTAssertEqual(tooShortPasswordMessage.label, "Password must exceed 3 characters")  

Wait for an element to appear on the screen.
Some UI elements take time to appear on the screen. That may depend for example on the API response or WiFi/3G speed. That is why it is necessary to have a solution that waits for those elements, performs the interaction on them and continue with executing the next steps of the tests.

Let's say that after tapping on 'Create account' button, you expect the application to show 'Account successfully created!', but this message isn't displayed immediately. That's the perfect situation to apply this waitForExpectations solution:

let accountSuccessfullyCreatedMessage = self.app.staticTexts["Account successfully created!"]  
let exists = NSPredicate(format: "exists == true")  
expectation(for: exists, evaluatedWithObject: accountSuccessfullyCreatedMessage, handler: nil)

app.buttons["Create Button"].tap()

waitForExpectations(timeout: 10, handler: nil)  
XCTAssert(accountSuccessfullyCreatedMessage.exists)  
XCTAssert(accountSuccessfullyCreatedMessage.label, "Account successfully created!")  

The above code waits for the text 'Account successfully created!' to appear on the screen after pressing on 'Create account' button.

The predicate matches when element exists NSPredicate(format: "exists == true"). Additionally we are setting out the expectation for the object to exist.

After tapping on the button, the waitForExpectations is going to be executed and wait for the element to appear on the screen with the timeout given.

Test failures

When your tests fail you want to know why. There are a couple of ways of debugging the tests.

Printing.
1.First one is printing the accessibility hierarchy, which allows us to find out how our tests see the app's interface. To print out the accessibility hierarchy use:

print(app.debugDescription)  

Do not forget to set the breakpoint after printing so that you will be able to verify the hierarchy in the debug window.

2.If you want to reduce the information displayed in the debug window you can also specify the UI element of which you want to print the accessibility hierarchy. To do so, use:

let accountSuccessfullyCreatedMessage = self.app.staticTexts["Account successfully created!"]  
print(accountSuccessfullyCreatedMessage.debugDescription)  

Jump to report.
When your tests fail, go to the Test navigator:
In that view you can see which of your tests failed, as they are marked with the red exclamation mark. Right click on the test and select Jump to report. Expand the view with the arrows given which will lead you to the point where your test failed.

Example view:

The report shows that test did not find the Home button on the screen.

Sum up

After reading this article you will be able to:
1. Explain what UI tests are.
2. Set up the Xcode project to start working with XCUITest framweork.
3. Use UI recorder and identify elements.
4. Perform basic interactions on the UI of the app.
5. Run and debug tests.

What is the next step?

Launch Xcode, configure the project and start writing your tests. Happy testing!

About Novoda

We plan, design, and develop the world’s most desirable software products. Our team’s expertise helps brands like Sony, Motorola, Tesco, Channel4, BBC, and News Corp build fully customized Android devices or simply make their mobile experiences the best on the market. Since 2008, our full in-house teams work from London, Liverpool, Berlin, Barcelona, and NYC.

Let’s get in contact