Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
224 changes: 180 additions & 44 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ web/

### Jest Unit Testing Guide

This guide provides guidelines and best practices for writing unit tests using Jest in this project. Following these
These are the guidelines and best practices we follow when writing unit tests using Jest in this project. Following these
standards ensures consistency, maintainability, and comprehensive test coverage.

#### Best Practices
Expand Down Expand Up @@ -348,7 +348,6 @@ jest.mock('path/to/module')
* This creates an object containing all named exports from ./path/to/module
*/
import * as mockModule from 'path/to/module'

;(mockModule.module as jest.Mock).mockResolvedValue(mockReturnValue)
```

Expand Down Expand Up @@ -493,14 +492,28 @@ jest.spyOn(Array.prototype, 'includes').mockImplementation(function (value) {

# Playwright (E2E) Testing Guide

E2E tests use [Playwright](https://playwright.dev/) and run against a fully isolated local stack:
These are the guidelines and best practices we follow when writing E2E tests using [Playwright](https://playwright.dev/) in this project. Following these
standards ensures consistency, maintainability, and comprehensive test coverage.

E2E tests live in `tests/e2e/` and follow the `*.spec.ts` naming convention, when executed they are run against a fully isolated local stack:

- **Supabase** (Postgres) via `npx supabase start`
- **Firebase** (Auth and Storage) via `firebase emulators:start`
- **Supabase** (Postgres)
- **Firebase** (Auth and Storage)
- **Backend API** (`backend/api`)
- **Next.js frontend** (`web`)

Tests live in `tests/e2e/` and follow the `*.e2e.spec.ts` naming convention.
### Best Practices

1. Test one scenario per test - Each test should verify a single behavior.
2. Keep tests independent - Each test should be fully independent.
3. Use locators that reflect how users will interact with the application - Outlined in the [Component Selection Hierarchy](###component-selection-hierarchy) below.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix invalid markdown link fragment.

The link uses ### in the fragment, but markdown internal links should use a single # followed by the heading slug.

🔗 Proposed fix
-3. Use locators that reflect how users will interact with the application - Outlined in the [Component Selection Hierarchy](###component-selection-hierarchy) below.
+3. Use locators that reflect how users will interact with the application - Outlined in the [Component Selection Hierarchy](`#component-selection-hierarchy`) below.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
3. Use locators that reflect how users will interact with the application - Outlined in the [Component Selection Hierarchy](###component-selection-hierarchy) below.
3. Use locators that reflect how users will interact with the application - Outlined in the [Component Selection Hierarchy](`#component-selection-hierarchy`) below.
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)

[warning] 509-509: Link fragments should be valid

(MD051, link-fragments)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/testing.md` at line 509, The markdown link fragment
"###component-selection-hierarchy" is invalid; replace it with a proper internal
link fragment using a single '#' and the correct heading slug (e.g.,
"`#component-selection-hierarchy`"), and verify the heading named "Component
Selection Hierarchy" is converted to the matching kebab-case lowercase slug so
the anchor works.

4. Use the Page Object Model (POM) - Keeps the test files lightweight and easily readable at a glance.
5. Store authentication state - Persist login state via `storageState` and reuse it across tests where needed.
6. Use fixtures for setup and teardown - Keeps test files lightweight.
7. Use web-first assertions - Use Playwright's built-in `expect` assertions (e.g., `toBeVisible`, `toHaveText`) which auto-retry until the condition is met.
8. Use environment variables for config - Store credentials, base URLs, and environment-specific settings in env vars rather than hardcoding them in test files.
9. Organise tests with a clear folder structure - Separate test files, page objects, fixtures, and helpers into distinct directories for scalability and maintainability.
10. Integrate with CI/CD - Run Playwright tests in headless mode in your pipeline, alerting you of possible issues early.

---

Expand Down Expand Up @@ -603,9 +616,15 @@ This opens a visual browser interface where you can:
- 🔄 Re-run tests without restarting anything
- 🕵️ Time-travel debug through test steps

Alternatively if you only want to open the Playwright UI you can use:

```bash
npx playwright test --ui
```

### 3. Edit tests and re-run

Edit your `*.e2e.spec.ts` file, save, then click **Run** in the Playwright UI.
Edit your `*.spec.ts` file, save, then click **Run** in the Playwright UI.
No restart needed for test file changes.

### 4. Reset data when needed
Expand Down Expand Up @@ -672,10 +691,10 @@ tests/
└── e2e/
├── web/
│ └── specs/
│ └── auth.e2e.spec.ts
│ └── auth.spec.ts
└── backend/
└── specs/
└── api.e2e.spec.ts
└── api.spec.ts
```

### Component Selection Hierarchy
Expand Down Expand Up @@ -712,23 +731,62 @@ This hierarchy mirrors how users actually interact with your application, making
Tests often receive multiple page objects as fixtures (e.g. `homePage`, `authPage`, `profilePage`). This is the **Page
Object Model** pattern — a way to organize selectors and actions by the area of the app they belong to.

**Page objects are not separate browser tabs.** They are all wrappers around the same underlying `page` instance. Each
class simply encapsulates the selectors and actions relevant to one part of the UI:
**Page objects** are all wrappers around the same underlying `page` instance. Each
class simply encapsulates the selectors and actions relevant to an entire page of the application.

The `app.ts` file improves scalability by acting as a central hub for page objects and shared modules. Instead of importing 40 different pages into a test, modules can be accessed through `app.ts`, making tests cleaner and easier to maintain. It also supports functionality that spans multiple pages.

```typescript
class ProfilePage {
constructor(private page: Page) {}
//profilePage.ts

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Use PascalCase for page object filenames to match coding guidelines.

The example filenames use camelCase (profilePage.ts, settingsPage.ts, app.ts), but the coding guidelines require PascalCase: "File and class names must use PascalCase (e.g., CompatibilityPage.ts / class CompatibilityPage)".

📝 Proposed fix
-//profilePage.ts
+//ProfilePage.ts
 import {expect, Locator, Page} from '`@playwright/test`'
 
 export class ProfilePage {
-//settingsPage.ts
+//SettingsPage.ts
 import {expect, Locator, Page} from '`@playwright/test`'
 
 export class SettingsPage {
-//app.ts
-import {ProfilePage} from './profilePage'
-import {SettingsPage} from './settingsPage'
+//App.ts
+import {ProfilePage} from './ProfilePage'
+import {SettingsPage} from './SettingsPage'
 
 export class App {

As per coding guidelines, Playwright E2E test guidelines specify: "File and class names must use PascalCase (e.g., CompatibilityPage.ts / class CompatibilityPage)."

Also applies to: 756-756, 772-772

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/testing.md` at line 740, Update the example filenames and corresponding
class names from camelCase to PascalCase: rename profilePage.ts ->
ProfilePage.ts, settingsPage.ts -> SettingsPage.ts, app.ts -> App.ts and update
any example class declarations (e.g., class profilePage -> class ProfilePage)
and all references to those symbols in the document (including the other
mentioned occurrences) so examples follow the "File and class names must use
PascalCase" guideline.

Source: Coding guidelines

import {expect, Locator, Page} from '@playwright/test'

export class ProfilePage {
private readonly displayName: Locator

constructor(public readonly page: Page) {
this.displayName = page.getByTestId('display-name')
}

async verifyDisplayName(name: string) {
await expect(this.page.getByTestId('display-name')).toHaveText(name)
await expect(this.displayName).toBeVisible()
await expect(this.displayName).toHaveText(name)
}
}

class SettingsPage {
constructor(private page: Page) {} // same page instance
//settingsPage.ts
import {expect, Locator, Page} from '@playwright/test'

export class SettingsPage {
private readonly deleteAccountButton: Locator

constructor(public readonly page: Page) {
this.deleteAccountButton = page.getByRole('button', {name: 'Delete account'})
}

async deleteAccount() {
await this.page.getByRole('button', {name: 'Delete account'}).click()
await expect(this.deleteAccountButton).toBeVisible()
await this.deleteAccountButton.click()
}
}

//app.ts
import {ProfilePage} from './profilePage'
import {SettingsPage} from './settingsPage'

export class App {
readonly profile: ProfilePage
readonly settings: SettingsPage

constructor(public readonly page: Page) {
this.profile = new ProfilePage(page)
this.settings = new SettingsPage(page)
}

//Methods that span multiple pages can be outlined here
async verifyAccountThenDelete(name: string) {
this.profile.verifyDisplayName(name)
//navigation to the settings page
this.settings.deleteAccount()
}
}
```
Expand All @@ -747,9 +805,9 @@ await page.locator('[data-testid="skip-onboarding"]').click()
// ...50 more lines of noise

// ✅ With POM — readable and maintainable
await registerWithEmail(homePage, authPage, fakerAccount)
await skipOnboardingHeadToProfile(onboardingPage, signUpPage, profilePage, fakerAccount)
await profilePage.verifyDisplayName(fakerAccount.display_name)
await app.registerWithEmail(fakerAccount)
await app.skipOnboardingHeadToProfile(fakerAccount)
await app.profile.verifyDisplayName(fakerAccount)
```

**What happens if you call a method on the "wrong" page object?**
Expand All @@ -761,20 +819,93 @@ won't find its element and the test will **time out**.

```typescript
// ⚠️ This fails at runtime if navigation hasn't happened yet
await settingsPage.deleteAccount() // navigates away from profile
await profilePage.verifyDisplayName(name) // locator not found → timeout
await app.settings.deleteAccount() // navigates away from profile
await app.profile.verifyDisplayName(name) // locator not found → timeout
```

Always ensure navigation has completed before calling methods that depend on a specific screen being visible.

### Fixtures

To further improve readability, and simplify the creation/implimentation of tests, fixtures are used for test case setup and teardown where appropriate.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix typo.

"implimentation" should be "implementation".

✏️ Proposed fix
-To further improve readability, and simplify the creation/implimentation of tests, fixtures are used for test case setup and teardown where appropriate.
+To further improve readability, and simplify the creation/implementation of tests, fixtures are used for test case setup and teardown where appropriate.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
To further improve readability, and simplify the creation/implimentation of tests, fixtures are used for test case setup and teardown where appropriate.
To further improve readability, and simplify the creation/implementation of tests, fixtures are used for test case setup and teardown where appropriate.
🧰 Tools
🪛 LanguageTool

[grammar] ~830-~830: Ensure spelling is correct
Context: ... readability, and simplify the creation/implimentation of tests, fixtures are used for test ca...

(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/testing.md` at line 830, Typo: change the word "implimentation" to
"implementation" in the sentence "To further improve readability, and simplify
the creation/implimentation of tests, fixtures are used for test case setup and
teardown where appropriate." Locate that sentence in docs/testing.md (the line
that contains "creation/implimentation of tests") and replace "implimentation"
with "implementation" to correct spelling.


```typescript
//baseFixture.ts
import {test as base} from '@playwright/test'
import {App} from './app.ts'
import {testAccounts, UserAccountInformation} from './accountInformation'
import {deleteUser} from './deleteUser'
import {seedUser} from './seedDatabase'

export const test = base.extend<{
app: App
signedInAccount: UserAccountInformation
}>({
//This gives access to the entire POM structure for the application
app: async ({page}, use) => {
const appPage = new App(page)
await use(appPage)
},
/**
* This generates a test account
* Seeds the database with the user
* Signs the user into the app
* Executes the test
* Deletes the user after execution is complete
*/
signedInAccount: async ({app}: {app: App}, use) => {
const account = testAccounts.faker_account()
await seedUser(account.email, account.password)
await app.signinWithEmail(account)
await use(account)
await deleteUser(account)
},
})

export {expect} from '@playwright/test'
```

```typescript
//test.spec.ts
//This is an example of how the above fixture would be used in a test
import {expect, test} from './fixtures/baseFixture'

test.describe('when given valid input', () => {
test('should already be signed into the correct account', async ({
signedInAccount, //This is the fixture that contains the account information and is already signed in
app, //This is the fixture that gives access to the entire POM structure
}) => {
await app.home.goToProfilePage()
await app.profile.verifyDisplayName(signedInAccount.display_name)
})
})

//This is how the test would look without the fixture
import {test, expect} from '@playwright/test'
import {App} from './app.ts'
import {testAccounts, UserAccountInformation} from './accountInformation'
import {deleteUser} from './deleteUser'

test.describe('when given valid input', () => {
test('should already be signed into the correct account', async ({page}) => {
const app = new App(page)
const account = testAccounts.faker_account()

await app.auth.signUpWithEmail(account)
await app.home.goToProfilePage()
await app.profile.verifyDisplayName(signedInAccount.display_name)
await deleteUser(account)
})
})
```

### Setting up test data

Since the tests run in parallel (i.e., at the same time) and share the same database and Firebase emulator, it can
create issues where one tests edits or deletes data that another test is using, hence breaking that test.

The standard solution for shared data is **test isolation via unique data per test**. Each test generates its own unique
identifiers so
they never touch each other's data.
identifiers so they never touch each other's data.

**1. Use unique emails/username/IDs per test**

Expand All @@ -792,16 +923,8 @@ This way no two tests share the same user, so deletes/reads never conflict.

**2. Cleanup only your own data**

Each test must fully attend to their own (and only their own) garden, by tracking what it created and cleaning up only
that:

```js
afterEach(async () => {
await deleteUser(email, password) // only the one this test created
})
```

Avoid `deleteAllUsers()` or broad wipes in parallel tests — that's what causes race conditions.
Each test must fully attend to their own (and only their own) data, by tracking what it created and cleaning up only
that, correct use of fixtures helps data cleanup.

**3. If you must share fixtures, use read-only shared data**

Expand All @@ -825,21 +948,13 @@ This gives each parallel worker its own emulator namespace, so even aggressive c

This is not implemented yet, but it will be very userful as the playwright test suite grows.

**Recommended approach in practice:**

- Unique email/username/ID per test → no sharing, no conflict
- `afterEach` cleans up only own data
- `beforeAll` seeds any read-only shared fixtures once

This eliminates race conditions without needing locks or sequencing.

### Example test

```typescript
import {test, expect} from '@playwright/test'

test.describe('Authentication', () => {
test('should login successfully', async ({page}) => {
test.describe('When given valid input', () => {
test('this should login successfully', async ({page}) => {
await page.goto('/')
await page.getByRole('button', {name: 'Sign In'}).click()
await page.getByLabel('Email').fill('test@example.com')
Expand Down Expand Up @@ -868,6 +983,27 @@ For comprehensive troubleshooting guidance beyond testing-specific issues, see
the [Troubleshooting Guide](troubleshooting.md) which covers development environment setup, database and emulator
issues, API problems, and more.

### Supabase emulator not working

There might be compatability issues with the Supabase emulator and your setup this can cause a `Runtime error` on the app pointing to an issue in the `supabase/utils.ts (69: 17)` file, and the Supabase emulator showing a generic `site can't be reached` browser error.

The workaround for this is to use a remote db and the local firebase emulator

Install DBeaver (contact the main maintainer for the postgres db connection info) to view and edit the database

```bash
# Comment out "Object.assign(process.env, supabaseEnv)" in playwright.config.ts

# This launches the Firebase Emulator
yarn emulate

# This launches the app
yarn dev

# Launch Playwright
npx playwright test --ui
```

### Port already in use

```bash
Expand Down
Loading
Loading