Skip to main content

Test Plan for humuus

info

This document is still a draft.

Introduction​

Purpose of the Document​

This document defines the testing strategy for the humuus web application. It aims to provide a structured overview of the testing activities, the test environment, and the associated workload.

Test Object​

humuus is a web-based application that enables interactive workshop design through various quizzes, slides, and many other interactive elements. The application's architecture includes a frontend built with TypeScript using React and Next.js. The backend is also written in TypeScript and runs on a Nakama Game Server.

Test Objectives​

Preventive Error Detection​

By integrating tests into the CI/CD pipeline using GitHub Actions, pull requests can be automatically validated when merged into the main or dev branches. This ensures code quality and prevents errors from reaching production.

Test Coverage​

The testing strategy aims for comprehensive code coverage to ensure a consistently good user experience. Function-critical areas such as login, match creation and joining, and user interface accessibility should be prioritized. Therefore, test coverage cannot be fully achieved through unit tests alone, which is why different test levels should be considered.

Test Risks​

Personnel resources for test management are limited, which brings the following risks:

  • Possible gaps in the test plan
  • Limited expertise in specialized testing areas
  • Time restrictions

Therefore, test activities should be prioritized to ensure system-critical tests are in place.

Test Strategies​

TestPyramid

To achieve a good balance between test coverage and execution effort, it is proposed to use unit tests as the foundation, followed by integration tests and end-to-end tests. Unit tests enable quick detection and prevention of errors in individual functions. Integration and end-to-end tests, on the other hand, provide an ideal way to test code dependencies. If a test fails here, a somewhat longer error analysis and resolution may follow. Therefore, it makes sense to identify errors at a modular level in an isolated environment.

Unit Tests​

The fundamental tests should be unit tests - tests that validate the functionality of individual components such as functions or classes. Jest allows required dependencies in functions/classes to be simulated through mocks, enabling independent and modular test execution. Unit tests provide quick error identification at a granular level.

Integration Tests​

Integration tests can test relationships and processes to validate the interaction of different components and thus ensure the validity of communication between system parts. They cover more complex processes and verify that modules interact as expected. Jest could also be used here to ensure functionality flows in an encapsulated manner.

End-to-End Tests​

E2E tests then simulate complete workflows that a user goes through on the platform in a realistic browser environment. These tests thus validate the application from a user's perspective. They also require a high understanding of the application's processes. This way, problems that only become visible in the interaction between browser and system components can be identified. After analyzing possible frameworks, Playwright was chosen. More on this in the next chapter.

Test Frameworks​

Jest​

Jest was chosen as the primary testing framework for unit and integration tests. It is widely used among JavaScript applications and was selected here because it can be ideally integrated into the TypeScript and React environment of humuus through versatile React testing libraries. Jest offers the ability to mock modules, functions, API calls, etc., and thus create isolated unit and integration tests for humuus. The spy functionality of Jest also allows function calls to be tracked and verified whether data was successfully passed on. Jest can also be seamlessly integrated into a CI/CD pipeline through GitHub Actions.

Playwright​

For end-to-end tests, Playwright was selected after thorough research. Other options were Cypress, Puppeteer, and WebdriverIO. One of the most important factors in selecting the right framework/tool was good cross-browser support. Since humuus is used by many different participants, compatibility with Safari, Chrome, Firefox, etc. is a very important requirement to ensure user experience. However, Puppeteer and Cypress only offer limited support for Safari. Setting up Playwright happens automatically and according to experience reports, it doesn't require much effort for configuration. The framework offers automatic waiting for elements, context isolation, and multi-tab/multi-window support. Playwright is scalable and should run performantly even with large test cases. Similar to Jest, it can be easily integrated into existing GitHub Actions. Additionally, visual testing is also possible through an extension with Playwright. This would serve as an alternative to Storybook.

Test Cases​

The test cases are structured into three categories according to the test pyramid:

Unit Test Cases (Jest)​

Unit test cases mainly include smaller test cases. They form the foundation of the testing strategy and focus on the isolated verification of individual functions, classes, and React components. It is checked whether React components render correctly. In some cases, it is verified whether buttons fulfill their function or whether specified texts are visible to the user. Function error handling is also checked to ensure all function calls are made correctly and the correct values are returned. In the backend, element functionalities are also tested, such as the vital RPC functions. Here it is also checked whether all redirects, errors, and values are as expected - all independent of other components and also independent of Nakama.

Integration Test Cases (Jest)​

Integration tests verify the interaction of multiple components and ensure that the various parts of the application communicate correctly with each other. Critical test cases such as the flow for joining a match, the flow in a quiz (display question, select answer, resolution), and others should be covered here.

End-to-End Test Cases (Playwright)​

End-to-end tests then cover critical user flows that take place in an isolated browser environment through Playwright. One of the most important test cases will be user authentication, which can be replicated through Playwright. Here, for example, it can be checked whether successful and unsuccessful login attempts produce the desired behavior patterns of the website. The flow of a complete workshop should also be run through from both the host's and a guest's perspective to test whether answers, questions, etc. are displayed correctly and all buttons fulfill their desired function. Error scenarios such as connection interruptions and entering an incorrect workshop code should also be checked. All end-to-end tests should be tested at least in Chrome, Firefox, and Safari to increase usability and ensure user experience for users on different systems.

Acceptance Criteria​

To achieve quality goals, unit tests should demonstrate significant code coverage. Additionally, E2E tests should at least cover the most critical user flows such as authentication. Tests should also be integrated into the repository as CI/CD integration. This is enabled through GitHub Actions, which allows test automation and protection against erroneous changes in production.

Sources for Further Reading​