CI/CD Integration

Run CrowdRunner UX tests automatically on every deploy or pull request.

Prerequisites

1. Create an API key in Settings → API Keys (see the API Keys guide).
2. Store the key as a secret in your CI platform (e.g. CROWDRUNNER_API_KEY).

GitHub Actions

Add this workflow to .github/workflows/ux-test.yml:

name: UX Test
on:
  push:
    branches: [main]
  pull_request:

jobs:
  crowdrunner:
    runs-on: ubuntu-latest
    steps:
      - name: Trigger CrowdRunner test
        id: trigger
        run: |
          RESPONSE=$(curl -s -X POST \
            "${{ vars.CROWDRUNNER_API_URL }}/api/v1/tests" \
            -H "Authorization: Bearer ${{ secrets.CROWDRUNNER_API_KEY }}" \
            -H "Content-Type: application/json" \
            -d '{
              "url": "https://staging.example.com",
              "cohort_name": "Indie Developers",
              "task": {
                "description": "Complete the signup flow",
                "success_criteria": "User reaches the dashboard"
              },
              "panel_size": 3
            }')
          echo "test_id=$(echo $RESPONSE | jq -r .test_id)" >> $GITHUB_OUTPUT

      - name: Wait for results
        run: |
          TEST_ID=${{ steps.trigger.outputs.test_id }}
          for i in $(seq 1 60); do
            STATUS=$(curl -s \
              "${{ vars.CROWDRUNNER_API_URL }}/api/v1/tests/$TEST_ID" \
              -H "Authorization: Bearer ${{ secrets.CROWDRUNNER_API_KEY }}" \
              | jq -r .status)
            echo "Poll $i: status=$STATUS"
            if [ "$STATUS" = "completed" ]; then
              echo "Test completed successfully"
              exit 0
            elif [ "$STATUS" = "failed" ]; then
              echo "Test failed"
              exit 1
            fi
            sleep 10
          done
          echo "Timeout waiting for test"
          exit 1

GitLab CI

Add this job to .gitlab-ci.yml:

ux_test:
  stage: test
  image: curlimages/curl:latest
  variables:
    CROWDRUNNER_API_URL: "https://api.crowdrunner.dev"
  script:
    - |
      RESPONSE=$(curl -s -X POST \
        "$CROWDRUNNER_API_URL/api/v1/tests" \
        -H "Authorization: Bearer $CROWDRUNNER_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "url": "'"$CI_ENVIRONMENT_URL"'",
          "cohort_name": "Indie Developers",
          "task": {
            "description": "Complete the signup flow",
            "success_criteria": "User reaches the dashboard"
          },
          "panel_size": 3
        }')
      TEST_ID=$(echo $RESPONSE | jq -r .test_id)
      for i in $(seq 1 60); do
        STATUS=$(curl -s \
          "$CROWDRUNNER_API_URL/api/v1/tests/$TEST_ID" \
          -H "Authorization: Bearer $CROWDRUNNER_API_KEY" \
          | jq -r .status)
        echo "Poll $i: status=$STATUS"
        [ "$STATUS" = "completed" ] && exit 0
        [ "$STATUS" = "failed" ] && exit 1
        sleep 10
      done
      exit 1

Generic (curl)

Use these commands in any CI system:

# 1. Create a test
curl -X POST "$API_URL/api/v1/tests" \
  -H "Authorization: Bearer $CROWDRUNNER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://staging.example.com",
    "cohort_name": "Indie Developers",
    "task": {
      "description": "Complete the signup flow",
      "success_criteria": "User reaches the dashboard"
    },
    "panel_size": 3
  }'

# 2. Poll for completion
curl "$API_URL/api/v1/tests/{test_id}" \
  -H "Authorization: Bearer $CROWDRUNNER_API_KEY"

# 3. Get detailed results
curl "$API_URL/api/v1/tests/{test_id}" \
  -H "Authorization: Bearer $CROWDRUNNER_API_KEY" \
  | jq '.personas[] | {name, task_outcome, satisfaction_score}'

Webhook Callbacks

Instead of polling, you can provide a callback_url when creating the test. CrowdRunner will POST the results to your URL when the test completes or fails:

curl -X POST "$API_URL/api/v1/tests" \
  -H "Authorization: Bearer $CROWDRUNNER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://staging.example.com",
    "cohort_name": "Indie Developers",
    "task": {
      "description": "Complete the signup flow",
      "success_criteria": "User reaches the dashboard"
    },
    "panel_size": 3,
    "callback_url": "https://your-ci.example.com/webhooks/crowdrunner"
  }'

Tips

• Use panel_size: 3 in CI for faster runs — save larger panels for manual deep-dives.
• Set up the webhook callback to avoid polling timeouts in constrained CI environments.
• Parse the satisfaction_score and task_outcome fields to create pass/fail gates.
• Use different cohorts for different test scenarios (e.g. "Accessibility Spectrum" for a11y checks).