--- name: testkase description: > Use TestKase from an AI agent: run Selenium, Playwright and Appium tests on real browsers and real Android/iOS phones, Espresso suites on Android and XCUITest suites on real iPhones/iPads (Automate), drive a real device or browser from the agent (Device Farm MCP), manage test cases and cycles (Test Management MCP), upload test results (@testkase/reporter) and compare screenshots (Visual). Use when the user mentions TestKase, DeviceFarm, hub.testkase.com, testkase:options, /espresso/v1, /xcuitest/v1, testkase-tunnel, @testkase/test-management-mcp, @testkase/device-farm-mcp, @testkase/mcp-server (old name), @testkase/reporter or @testkase/visual, or wants to run tests on cloud browsers or real devices. metadata: version: 0.5.0 docs_index: https://docs.testkase.com/llms.txt product_overview: https://www.testkase.com/llms.txt --- # TestKase Five products, one login, one access key: | Product | How an agent uses it | |---|---| | **Automate** — your Selenium/Playwright/Appium tests on real browsers and phones, Espresso suites on Android and XCUITest suites on iOS | Change where the suite connects (Steps 1–4); Espresso uploads two APKs and starts a build (Step 1b); XCUITest uploads an .ipa and a zipped Runner and starts a build (Step 1c); or let the agent drive a device itself with the Device Farm MCP (Step 4b) | | **Test Management** — test cases, cycles, plans, reports | Test Management MCP + results upload (Step 5) | | **Visual** — screenshot comparison | `npx @testkase/visual upload` (Step 6) | | **Live** — a person drives a real device by hand | Not scriptable; send the user to testkase.com | | **Accessibility** — free WCAG scans | Web app only | ## Access key (all products) The user creates one at **https://testkase.com/api-keys** (starts `xyz_`) and stores it in an environment variable or CI secret. Never ask them to paste it into the chat, never hard-code it. ## Step 1 — Connect (Automate) | Framework | Endpoint | |---|---| | Selenium, Appium | `https://x:@hub.testkase.com/wd/hub` (any username; key = password) | | Playwright | `wss://hub.testkase.com/playwright?capabilities=` (key in `testkase:options.accessKey`) | | Espresso (Android only) | `https://hub.testkase.com/espresso/v1` — its own REST API, not `/wd/hub`; see Step 1b | | XCUITest (iOS, real devices only) | `https://hub.testkase.com/xcuitest/v1` — its own REST API, not `/wd/hub`; see Step 1c | | Cypress, TestCafe | **Not supported** — say so. | Existing Node Playwright suite? Use the wrapper instead — no test changes: `npm i -D @testkase/playwright-reporter`, add `testkase.yml`, run `npx testkase-playwright test`. See docs.testkase.com → Automate → Playwright → CLI. ## Step 1b — Espresso suites (Android) No capabilities and no code change: upload the app APK and the androidTest APK, then start a build. Auth is the access key as the Basic-auth password (any username) or `Authorization: Bearer `. ```bash H=https://hub.testkase.com/espresso/v1 curl -u "x:$TESTKASE_ACCESS_KEY" -F "file=@app-debug.apk" $H/app # -> app_url farm://app- curl -u "x:$TESTKASE_ACCESS_KEY" -F "file=@app-debug-androidTest.apk" $H/test-suite # -> test_suite_url farm://suite- curl -u "x:$TESTKASE_ACCESS_KEY" -H "Content-Type: application/json" $H/build \ -d '{"app":"farm://app-","testSuite":"farm://suite-","devices":["-"]}' # -> build_id curl -u "x:$TESTKASE_ACCESS_KEY" $H/builds/ # status per device ``` - `devices`: up to 20; `{"device": "…", "virtual": true}` for an emulator. Optional filters `class`, `package`, `annotation`, `notAnnotation`, `size`. - Per-test results, video and logs: `GET $H/builds//sessions/`. Stop: `POST $H/builds//stop`. - Upload as multipart field `file`, `.apk` only (no `.aab`, no upload by URL). The suite's target package must match the app's package. - Not available: sharding, Orchestrator, `clearPackageData`, network logs, Local Tunnel, biometric/camera injection. Docs: https://docs.testkase.com/docs/automate/mobile/espresso ## Step 1c — XCUITest suites (iOS) No capabilities and no code change. Build for a real device (`xcodebuild build-for-testing -destination 'generic/platform=iOS'`), zip the `.app` into an `.ipa` (`Payload/.app`) and zip the `-Runner.app` into a `.zip`, then four calls. Same auth as Step 1b. ```bash H=https://hub.testkase.com/xcuitest/v1 curl -u "x:$TESTKASE_ACCESS_KEY" -F "file=@MyApp.ipa" $H/app # -> app_url farm://app- curl -u "x:$TESTKASE_ACCESS_KEY" -F "file=@MyAppUITests.zip" $H/test-suite # -> test_suite_url farm://suite- curl -u "x:$TESTKASE_ACCESS_KEY" -H "Content-Type: application/json" $H/build -d '{"app":"farm://app-","testSuite":"farm://suite-","devices":["-"]}' # -> build_id curl -u "x:$TESTKASE_ACCESS_KEY" $H/builds/ # status per device ``` - `devices`: up to 20 per build; real iPhones/iPads only (`"virtual": true` is refused). Optional filters `onlyTesting` / `skipTesting` (up to 50 entries each: `Target`, `Target/Class` or `Target/Class/method`). - Per-test results, video and logs: `GET $H/builds//sessions/`. Stop: `POST $H/builds//stop`. Clips arrive after the run; skipped or cut-off tests have none, very short tests usually none. No Live View while a build runs. - The app and Runner are re-signed with TestKase's development certificate: bundle ids are kept, but push, App Groups, iCloud, universal links and other special entitlements are stripped. Use a Development or Ad Hoc build. - Upload as multipart field `file` (no upload by URL). Not available: test plans, `.xctestproducts`, sharding, network logs, Local Tunnel, biometric/camera injection. Docs: https://docs.testkase.com/docs/automate/mobile/xcuitest ## Step 2 — Capabilities Everything TestKase-specific goes in `testkase:options` (unknown keys are dropped). - **Web:** `browserName` = `chrome`/`edge`/`firefox`/`safari`; `browserVersion` = `latest`/`latest-1`/major. `testkase:options.platformName` like `windows 11` or `macOS 26` (plain `windows` refused; **required on Playwright**). - **Mobile:** `platformName` `Android`/`iOS`; optional `appium:deviceName`, `appium:platformVersion`; exactly one of `appium:app` or `browserName`. Don't set `automationName`. - **Useful options:** `build`, `name`, `recording: true` (video is off by default), `idleTimeout` (max 1200 s), `tunnel`. ```python opts = webdriver.ChromeOptions() opts.set_capability("testkase:options", {"platformName": "windows 11", "name": "login", "recording": True}) key = os.environ["TESTKASE_ACCESS_KEY"] driver = webdriver.Remote(f"https://x:{key}@hub.testkase.com/wd/hub", options=opts) ``` **Apps:** `POST /api/v1/apps/upload-url` → `PUT` file → `POST /api/v1/apps/commit` (on hub.testkase.com, Bearer key) returns `farm://app-`. `.apk`, `.ipa`, `.zip` (simulator), max 500 MB. iOS builds are re-signed (bundle id changes). Demo app: `farm://app-sample` (test / test). ## Step 3 — Pass/fail `driver.execute_script("testkase-status=passed")` or `=failed` (Appium apps: add `[]` as the second argument). Always quit the driver in teardown. ## Step 4 — Localhost / private apps (Local Tunnel) Download `testkase-tunnel` from `https://pub-745d5ae3f0f2444a8eda8adf5a26934b.r2.dev/testkase-tunnel-{windows-amd64.exe,darwin-arm64,darwin-amd64,linux-amd64}`, run `testkase-tunnel --token $TESTKASE_ACCESS_KEY --name my-laptop`, and set `testkase:options.tunnel: "my-laptop"`. On a real iPhone use `http://localtest.me:`, never `localhost`. No npm or Docker version exists. ## Step 4b — Drive a real device from the agent (Device Farm MCP) `@testkase/device-farm-mcp` (26 tools) lets the agent open a real Android phone or iPhone (an uploaded app or a web page) or a desktop browser (Chrome, Edge, Firefox, Safari on Windows or macOS), read the screen as a numbered list, tap, type, swipe, press keys, open URLs and take screenshots. It also reads Automate runs (logs, video/screenshot links, why a run failed), gives Selenium/Appium/Playwright connection settings, uploads app builds (.apk, .ipa, simulator .zip), checks Local Tunnel status and stops a stuck run. One device at a time; a session ends after about 3 minutes idle or 60 minutes total; it uses the account's Automate minutes. The user runs once: ```bash claude mcp add testkase-device-farm -e TESTKASE_PAT_TOKEN= -- npx -y @testkase/device-farm-mcp ``` (Claude Desktop, Cursor, Copilot: same `npx` command in their MCP config.) Docs: https://docs.testkase.com/docs/automate/mcp-server ## Step 5 — Test Management **Let the agent manage tests** with `@testkase/test-management-mcp` (11 tools; formerly `@testkase/mcp-server`, which still installs but shows a rename notice). The user runs once: ```bash claude mcp add testkase -e TESTKASE_PAT_TOKEN= -- npx -y @testkase/test-management-mcp ``` (Claude Desktop, Cursor, Copilot: same `npx` command in their MCP config.) Docs: https://docs.testkase.com/docs/test-management/advanced/mcp-server The same access key works for both MCP servers. **Upload results** (JUnit, Playwright, Cypress, TestNG, NUnit, Cucumber): ```bash npx @testkase/reporter report --token $TESTKASE_PAT --project-id PRJ-1 \ --org-id --format junit --results-file junit.xml ``` ## Step 6 — Visual Needs a Visual subscription and project. `TESTKASE_TOKEN= npx @testkase/visual upload ./shots --project "Name"`. Exits 1 when a screen changed; approve in Visual → Builds. Keep filenames the same between runs; try `--dry-run` first. ## Good to know - Errors: 402/4402 no plan or free minutes used · 429/4429 too many parallel · 4426 tunnel not running · 4404 Playwright missing `platformName` · 4503 retry. - Sessions: max 2 h, end after 5 min idle (Espresso and XCUITest have no idle timeout; `runTimeout` in minutes can only lower the 2 h). Videos/logs kept 30 days. - Phones are cleaned, not factory-reset — use test accounts, never personal ones. - Anything not here: read https://docs.testkase.com/llms.txt. Don't invent capabilities or endpoints.