Testing is critical for all applications including the Power Platform and is a key part of the software development life cycle
Testing identifies issues or defects early in the release process and provides an opportunity to fix these issues to make the app more reliable before releasing changes into production
Manual & Automated Testing
Depending on the size and usage of the app, manual testing of new changes might be enough. However, as the app grows in complexity and usage, you might need to consider a test strategy that includes automated testing, instead of purely manual testing. Automated testing is particularly useful when the same test has to be repeated a number of times
With new features, often the best way is to test manually, particularly if those features are being refined or the business processes may change. Regression testing is the testing to ensure that new features haven’t inadvertently impacted existing functionality
As an app evolves, many of the regression tests have to be repeated with each release. Soon regression testing might take longer than the time spent to develop new features! In an environment of fast-paced development, thoroughly testing each existing feature in an app becomes a bottleneck to releasing software updates. This is where test automation reduces the time and effort, and can be much more effective at identifying critical issues than testing manually
Power Apps Test Studio
Power Apps Test Studio is a low-code solution to write, organize, and automate tests for canvas apps. In Test Studio, you can write tests using Power FX or use a recorder to automatically generate the expressions while using the app. You can play written tests back within Test Studio or they can be run in a web browser.
Not only that, the automated tests can be added to the automated ALM process so the whole process of testing and deploying new releases can be integrated
Microsoft: Working With Test Studio
Test Cases, Suites & Assertions
In Test Studio, test cases are executed to validate that specific features in the app, are working as expected. For example, in an Expense app, you would like to ensure that only expenses with an associated actual cost can be submitted. A test case can help verify that this condition or requirement is always met. Test cases are made up of a series of instructions or actions, called test steps and each of the test steps are written using Power FX
Test suites are used to organize or group associated test cases together. For example, you might have one test suite with test cases to validate expense report submissions and another test suite that focuses just on expense approvals
Every test case should have an expected result. To validate the expected result of a test against the actual result of your test, you can write test assertions. An assertion is an expression that evaluates to true or false in the test. If the expression returns false, the test case will fail.
In the example of an expense app, an assertion would be written to validate whether an expense report can be created with an expense line item having a zero cost. If so, the test would fail
Best Practises
When using Test Studio, follow these best practices:
Determine which test cases should be automated
It’s impractical to completely rely on test automation. Manual testing should be performed in addition to test automation, particularly for new features
Tests best suited to automation are:
- Repetitive tests
- High business impact functionality tests
- Features that are stable and not undergoing significant change
- Features that require multiple data sets
- Manual testing that takes significant time and effort
Keep test cases small
Avoid large test cases. Instead divide them into multiple smaller test cases with each testing a specific feature or item of functionality in the app. A failed assertion in a large test case might cause other functionality to remain untested. Using multiple test cases allows other functionality to get tested regardless of whether a previous test case failed making it easier to isolate test failures
Keep expressions to a single test action
A test action can contain multiple expressions. Large multi-action test expressions for a single step might impact your ability to debug and isolate any test failures. Divide a test step with multiple actions into more test steps of single actions to identify issues faster
Every test case should have an expected result
Each test case should have one or more expected results. Test assertions should be used to validate the expected outcomes of your test against the actual outcomes. Multiple assertions can be written for a single test case.
Use test suites.
Group or categorize similar test cases together with a description for the purpose and expected results
Limitations
The following cannot currently best tested in Test Studio:
- Components
- Code components written in the Power Apps Component (PCF) Framework
- Nested galleries
- Media controls
- Support for controls not listed in the Selectand SetProperty functions
- Person-type columns
Power FX ‘stubs’ can be added to Test Studio to replace content that would be return by these objects so that the automated test can execute
Also, bear in mind that the formula-level error management experimental feature needs to be turned on for the app in order to use Test Studio
DevOps Integration
Canvas app tests built in Test Studio can be further automated by integrating them with Azure Pipelines to automate deployments
- Microsoft: Automate Tests with Azure Pipelines
- Microsoft: Automated Tests with Azure Pipelines using YAML
Where Next for Automation
In October 2022, Microsoft announced the initial preview of Test Engine. Test Engine builds upon Test Studio, but takes it in a new, powerful direction through open source collaboration and use of the Playwright browser testing platform. The ultimate goals of Test Engine are to provide a robust testing platform for Model Driven Apps as well as Canvas Apps, and to make it easy to integrate automated testing into app development
- Microsoft: Introducing Test Engine
- Microsoft: Test Engine on GitHub
Summary
As with all software, effective testing of the Power Platform is important to ensure a smooth transition in to production
It can take some additional time to set up text cases in Test Studio, but for larger apps, particularly those that will receive regular updates, it can be a really useful tool
When it comes to regression testing, test automation can save a lot of time avoiding manual repetition of the same tests