How to test Windows applications

Foreword

This tutorial mainly explains how to use AirtestIDE for automated testing of Windows applications (taking NetEase Cloud Music as a case).

By reading this article, you will learn:

• Using AirtestIDE to script a Windows software window
• How to call Windows operation interface
• How to specify a Windows window in the script and command line

1. Basic principle

AirtestIDE's test support for common Windows applications mainly relies on the image recognition framework (Airtest) for location positioning. Then it uses the operation interface of pywinauto for simulation operations.

Poco's support for Windows has not yet been launched, and we will launch this feature as soon as possible, so stay tuned.

2. Connect Windows windows in AirtestIDE

To use AirtestIDE to test a Windows program, you must first connect the Windows window to be tested with AirtestIDE, which is actually telling AirtestIDE the handle of the Windows window.After successfully connecting the window, Airtest can use the window as a device . Next, you can perform image recognition and click operations on the Windows window just like operating an Android device.

Note: After connecting a Windows window, you will rely on this particular window handle when operating and running scripts in AirtestIDE.Therefore, please do not close it after selecting the window. Otherwise, it will fail to find the corresponding window and cause an error.

General connection method: one-click embedding

AirtestIDE provides a one-click embedded Windows window function, as shown below:

Click the Selected Window button of AirtestIDE, you will be able to select a Windows application that has been launched on the desktop (a green box will appear at the edge of the window at this time) and embed it into AirtestIDE.After embedding the window, you can easily manipulate the window, record statements, and run scripts in AirtestIDE.

Another alternative connection method

The underlying implementation of Windows applications varies. Sometimes the default one-click embedding method is used to connect to some application windows. After embedding them in AirtestIDE, you will encounter some problems (such as the inability to use the mouse after embedding, the inability to embed normally, the inability to display images after embedding, etc.)

In response to these problems, we provide a solution without embedded connections. Please find Device-Windows Embed Backup Method in the settings panel of AirtestIDE, then check this option. Finally use this alternative solution to connect the windows.

The next connection method is the same as the conventional method. Click Select Window and select the window program to be tested:

As shown in the figure, this window connection method does not embed Windows windows into AirtestIDE. Although it is not as simple and easy to use as the default embedding method during script recording, it can avoid some problems caused by window embedding.

Desktop mode

If you want to test more than one window (with multiple different window handles), embedding a single window may not meet the testing requirements. At this time, we also provide a third solution: desktop mode.

Click the desktop mode button. At this time, the device window on the right side of AirtestIDE will be completely hidden, so that you will enter the Windows desktop mode.

In this mode, there is no difference between recording and running scripts. It's just that the screenshot will be recognized on the entire desktop during execution, and even the screenshot statement in the code window of AirtestIDE can be recognized. Therefore, when executing the script, please minimize the window of AirtestIDE to avoid the screenshot in the script interface disturbing the execution result.

Considerations for connection

• When you select a window, there may be many other windows open on the desktop at the same time, so the green box when selecting may not be able to accurately select the application under test. Before you click the Selected Window button, please try to minimize all other unrelated windows as much as possible. This will make the window selection more accurate.
• To prevent abnormal situations, please do not choose your own desktop or AirtestIDE software.Because the principle of window embedding function is to set a Windows window as a child window of the IDE, if you try to connect to the desktop, it may cause unexpected behavior.If you just want to test all the windows on your desktop, you can choose our desktop mode to record the script.
• Windows windows need to record resolution when taking screenshots, so the size will be fixed after the window is embedded in the IDE and cannot be modified. If you think the window is too large after embedding, you can adjust the window to a suitable size before embedding.

Exit Windows connection

• Close AirtestIDE directly to exit this Windows connection and pop up the previously embedded window automatically
• You can also use the Disconnect current device button in the upper right corner to exit the window connection

3. Record and write Airtest scripts

After successfully connecting to the Windows window, it is equivalent to having connected a device, and then you can record and play back the script.

Generate screenshot statement

First, statements such as touch /wait / exists /assert_exists / assert_not_exists with screenshots are not much different from recording on Android / iOS devices:

However, it should be noted that after using the mouse to drag the box to select the position where you want to take a screenshot, you need to double-click the screenshot area to complete the screenshot. It does not automatically complete the screenshot when the mouse is released, which is the biggest difference between screenshots under Windows.

So the steps for the screenshot are:

• Drag the mouse frame to select the appropriate area
• Double-click to complete the screenshot
• Click the right mouse button to reselect the box
• Press the Esc button to exit this screenshot operation

When recording a swipe statement, you need to click the sliding end point to complete the statement recording after completing the selection of the screenshot area.

Call the Windows interface

Like Android / iOS, Airtest also encapsulates common operations under Windows. The underlying operation interface uses the pywinauto library.

Therefore, when writing a test script for a Windows application, we may need to consult the following API documents:

• Airtest's cross-platform API, All API here can be called directly in the script
• Airtest's Windows-specific APIPlease refer to the code examples provided below to learn how to call the proprietary interface of a Windows device
• If you need more complicated operations, you may need to check the API provided by pywinauto

A simple example

For the sake of demonstration, we assume that this script does not pass parameters in the command line, but uses the connect_device interface in the script to connect to a window with a handle of 123456 and perform some operations on it:

from airtest.core.api import *
dev = connect_device("Windows:///123456")
# The calling method of general interface is the same as other platforms:
touch(image)

If we want to be able to call some Windows-specific operations, for example, after consulting the Airtest's Windows-specific API document, we find that there are some operations that are only available in Windows windows:

# Call Windows-specific interfaces, such as getting the title content of the current window
print(dev.get_title())
# Move the window to a certain coordinate position
dev.move((100, 200))

When we want to use the mouse wheel, we need to check Airtest's API. But we found that the Windows module of Airtest did not encapsulate the function of the mouse wheel. At this point we can further query the documentation of pywinauto and find the relevant chapter of mouse, then we know how to call the mouse wheel interface:

dev.mouse.scroll(coords=(80, 100), wheel_dist=1)

Enter keyevent

In Android, we can use the keyevent ("HOME") to implement the operation of pressing the HOME key. In Windows, we can also send some key responses through the keyevent interface.Android key codes are based on ADB, while Airtest's Windows module uses key codes supported by pywinauto. Please refer to the content of the pywinauto.keyboard document to write the keyevent interface parameters under Windows:

# In pywinauto, the symbol ^ also represents the CTRL key, so ^ a means select all (Ctrl + A)
keyevent("^a")
# For example, this is the delete key input method, you need to add brackets {}    
keyevent("{DEL}")

Please refer to the document according to the actual needs, and then write the key response code.

4. How to specify a window when running a script

In AirtestIDE's Windows mode, running scripts and viewing reports is no different than other platforms:

However, the most important thing to note is that the windows connected in AirtestIDE are all connected using window handle.The window handle is a unique 32-bit unsigned integer owned by each Windows window object, and this value changes every time the window is opened.

This means that if we write a script with AirtestIDE embedded in a window. Although it can be run directly this time, there is no guarantee that it will run next time by copying the script command line in AirtestIDE.Because the command line in AirtestIDE will have such parameter content --device Windows: /// handle, and the next time you open the window, the handle may have changed.

Therefore, instead of using handles, we can connect windows in other ways.Airtest's Windows module uses the connect interface of pywinauto to connect windows, so in addition to the handle, we can connect by the window title:

# Connect a Windows window with a window handle of 123456
Windows:///123456
# Connect a Windows window, the window name matches a regular expression   
Windows:///?title_re=Unity.*
# Connect to the windows desktop without specifying any windows, corresponding to the desktop mode of the IDE   
Windows:///

If you want to use the window name to connect on the command line, you don't need to add symbols such as quotation marks.

airtest run test.air --device Windows:///?title_re=Unity.*

5. Poco support

Windows window connected to Poco-SDK

Some Windows-apps have access to Poco (such as Unity game window), we can also use AirtestIDE to easily view and select UI nodes.But before using this function, you need to tell AirtestIDE in advance where the Windows window is and its size.

In "Options"-"Settings"-"Poco", select "Windows Content Area Rect". Then select the main window of the window and double-click to confirm (right click / esc to cancel), and then you can start using Poco related functions, as shown in the figure:

Poco-Windows support

We are improving the documentation and features for Poco support for Windows windows, so stay tuned.