Testing¶

Metaseed uses pytest for testing with a multi-layered approach covering unit tests, integration tests, and end-to-end UI tests.

Running Tests¶

# Run all tests
uv run pytest

# Run tests with coverage
uv run pytest --cov=metaseed

# Run only UI tests
uv run pytest tests/test_ui/ -v

# Run Selenium tests (visible browser)
uv run pytest tests/test_ui/test_selenium.py -v

Test Structure¶

tests/
├── test_api/           # REST API endpoint tests
├── test_cli/           # CLI command tests
├── test_facade.py      # ProfileFacade tests
├── test_models/        # Pydantic model factory tests
├── test_specs/         # YAML spec loading tests
├── test_storage/       # JSON/YAML storage tests
├── test_validators/    # Validation rule tests
├── test_ui/            # UI tests
│   ├── test_htmx.py    # HTMX route tests (FastAPI TestClient)
│   └── test_selenium.py # Selenium E2E tests (visible browser)
└── test_version.py

A test never touches the user's own data¶

The suite runs on the machine a person also uses metaseed on, so anything it writes to a default location lands in their real work. Two directories are therefore redirected for every test by an autouse fixture in tests/conftest.py:

Directory	Redirected by	Default it must never reach
Datasets	`METASEED_DATASETS_DIR`	`~/.local/share/metaseed/datasets`
Specifications	`XDG_DATA_HOME`	`~/.local/share/metaseed/specs`

Specifications resolve under the data directory rather than through an override of their own, so redirecting XDG_DATA_HOME covers them and anything else that directory grows. A test that sets XDG_DATA_HOME itself still wins, because its own monkeypatch runs after the fixture.

Both were learned rather than designed. A test that saved through the app wrote into the real datasets directory, and one run's cleanup deleted a person's own datasets while the UI was open. Specifications then repeated it more quietly: tests that saved or published one left selenium-test and selenium-user-test in the user's real specs directory, where metaseed profiles lists them beside the profiles their work depends on, indistinguishable from something they wrote.

tests/test_tests_never_touch_the_user_datasets.py holds both to it: it resolves each directory the way the application does and fails if the answer is, or sits under, the user's real one. A new store that reads or writes user data needs its redirection set in the fixture and a case added there — a directory with no gate is the one that quietly fills up with fixtures.

A test's own inputs come from the repository¶

Parametrisation runs at collection time, before any fixture, so a test that enumerates profiles through SpecLoader.list_profiles builds its case list from whatever the developer happens to have installed. test_marker_roundtrip.py did exactly that: locally it generated cases for the maintainer's own cropxr profiles and on CI it did not, so the two runs were not testing the same thing.

Enumerate from get_builtin_specs_dir() instead. What such a test asserts is a promise about what the library ships; a person's own specification is not the library's to guarantee, and cannot be, since it is not there to check.

Selenium tests start the server as a subprocess, which inherits the environment, so they are covered by the same fixture rather than needing their own.

Unit Tests¶

Unit tests cover individual components in isolation:

Models: Test Pydantic model generation from specs
Validators: Test validation rules (required fields, date ranges, entity references)
Storage: Test JSON/YAML serialization and loading
Specs: Test YAML spec parsing and entity loading

Integration Tests¶

Integration tests verify component interactions:

CLI: Test command execution with real file I/O
API: Test REST endpoints with TestClient
Facade: Test entity creation through the facade pattern

UI Tests¶

The UI is built with HTMX + FastAPI + Jinja2 templates. Two test approaches are used:

HTMX Route Tests (TestClient)¶

Fast, headless tests using FastAPI's TestClient to verify route behavior and HTML responses.

from fastapi.testclient import TestClient
from metaseed.ui.app import create_app
from metaseed.ui.state import AppState

def test_create_entity(client):
    response = client.post(
        "/entity",
        data={
            "_entity_type": "Investigation",
            "unique_id": "INV-001",
            "title": "Test Investigation",
        },
    )
    assert response.status_code == 200
    assert "Created Investigation" in response.text

Selenium E2E Tests (Visible Browser)¶

End-to-end tests with a visible Chrome browser for demonstrating and verifying UI interactions.

Prerequisites¶

Chrome browser installed
ChromeDriver (managed by selenium)

Running Selenium Tests¶

# Run all Selenium tests
uv run pytest tests/test_ui/test_selenium.py -v

# Stop on first failure
uv run pytest tests/test_ui/test_selenium.py -v -x --tb=short

Test Configuration¶

Tests use a module-scoped server fixture that starts uvicorn on port 8081:

@pytest.fixture(scope="module")
def server():
    """Start the Metaseed server for testing."""
    proc = subprocess.Popen(
        ["uv", "run", "uvicorn", "metaseed.ui.app:app", "--port", "8081"],
        ...
    )
    # Wait for server to be ready
    yield proc
    proc.terminate()

@pytest.fixture
def browser(_server):
    """Create a visible Chrome browser for testing."""
    options = Options()
    options.add_argument("--window-size=1280,900")
    driver = webdriver.Chrome(options=options)
    yield driver
    driver.quit()

Test ID Convention¶

All interactive UI elements have data-testid attributes for reliable selection:

Element	Pattern	Example
Create entity button	`btn-create-{EntityType}`	`btn-create-Investigation`
Tree node	`tree-node-{node_id}`	`tree-node-abc123`
Delete node button	`btn-delete-{node_id}`	`btn-delete-abc123`
Entity form	`form-entity`	`form-entity`
Input field	`input-{field-name}`	`input-unique-id`
Create button	`btn-create`	`btn-create`
Update button	`btn-update`	`btn-update`
Optional fields toggle	`section-optional-toggle`	`section-optional-toggle`
Nested field button	`btn-nested-{field-name}`	`btn-nested-contacts`
Table add row	`table-add-row`	`table-add-row`
Table save	`table-save`	`table-save`
Table count	`table-count`	`table-count`
Table cell	`cell-{row}-{column}`	`cell-0-name`
Table row (clickable)	`row-{idx}`	`row-0`
Breadcrumb	`breadcrumb`	`breadcrumb`
Back button	`btn-back`	`btn-back`
Notification	`notification`	`notification`

Helper Functions¶

The test file provides helper functions for common operations:

def fill_field(driver, testid: str, value: str, trigger_change: bool = False):
    """Fill a form field by data-testid."""

def click_button(driver, testid: str):
    """Click a button by data-testid and wait."""

def element_exists(driver, testid: str) -> bool:
    """Check if an element with given data-testid exists."""

def expand_optional_fields(driver):
    """Expand the optional fields section if collapsed."""

Example Test¶

@pytest.mark.ui
class TestCreateInvestigation:
    def test_create_investigation(self, browser):
        browser.get("http://127.0.0.1:8081")

        # Click create button
        click_button(browser, "btn-create-Investigation")

        # Fill required fields
        fill_field(browser, "input-unique-id", "INV-001")
        fill_field(browser, "input-title", "Test Investigation")

        # Submit
        click_button(browser, "btn-create")

        # Verify entity appears in sidebar
        sidebar = browser.find_element(By.ID, "sidebar")
        assert "Test Investigation" in sidebar.text

Date Input Handling¶

For date inputs, JavaScript is used to set values directly (avoids locale-specific formatting issues):

if element.get_attribute("type") == "date":
    driver.execute_script("arguments[0].value = arguments[1]", element, "2024-03-15")

Triggering HTMX Events¶

When filling table cells that have HTMX handlers, trigger the change event:

fill_field(browser, "cell-0-name", "Dr. Smith", trigger_change=True)

Nested Entity Editing¶

The UI supports deep nesting of entities. When viewing a table of nested entities (e.g., Studies within an Investigation), users can click on a table row to edit that nested entity in a full form view.

Behavior¶

Click row to edit: Clicking a table row opens a form for editing that nested entity
Breadcrumb navigation: A breadcrumb trail shows the navigation path (e.g., "Investigation > studies > Study[0]"). An entity opened directly for editing — from the graph, a link, or a table row — shows the same path, built from its containment chain: each ancestor and the field it sits in, every ancestor a link, so the parent is reachable without returning to the entity list
Deep nesting: Nested entities can contain their own nested entities (e.g., Study > biological_materials > BiologicalMaterial)
Save & Back: Saves changes and returns to the parent table view
Cancel: Returns to parent without saving

Test IDs for Nested Editing¶

Element	Pattern	Example
Table row (clickable)	`row-{idx}`	`row-0`
Breadcrumb	`breadcrumb`	`breadcrumb`
Back button	`btn-back`	`btn-back`

Example Flow¶

Investigation (root form)
  └─ Click "studies" button → Table view
      └─ Click row 0 → Study form
          └─ Click "biological_materials" button → Table view
              └─ Click row 0 → BiologicalMaterial form

Selenium Test Coverage¶

The following end-to-end tests are implemented:

Test Class	Test Method	Description
`TestCreateInvestigationRequiredFields`	`test_create_investigation_required_fields`	Create Investigation with only required fields
`TestCreateInvestigationAllFields`	`test_create_investigation_all_fields`	Create Investigation with all optional fields
`TestAddNestedContacts`	`test_add_nested_contacts`	Add contacts through nested table
`TestEditInvestigation`	`test_edit_investigation`	Edit existing entity and verify persistence
`TestDeleteInvestigation`	`test_delete_investigation`	Delete entity and verify removal
`TestAddNestedStudy`	`test_add_nested_study`	Add Study through nested table
`TestValidationError`	`test_validation_error_missing_required`	Verify HTML5 validation on required fields
`TestProfileSwitch`	`test_switch_profile_clears_state`	Switch profile and verify state cleared
`TestNestedEntityEditing`	`test_edit_nested_study`	Click Study row to edit, verify form shows
`TestNestedEntityEditing`	`test_deep_nesting_navigation`	Navigate Investigation > Study > BiologicalMaterial

Markers¶

Tests are marked for selective execution:

@pytest.mark.ui - UI tests (Selenium)

Run specific markers:

uv run pytest -m ui           # Only UI tests
uv run pytest -m "not ui"     # Exclude UI tests