ci: add preview smoke test for frontend PRs

[talissoncosta]

• I have read the Contributing Guide.
• I have filled in the "Changes" section below.
• I have filled in the "How did you test this code" section below.

Changes

Contributes to #7262

Vercel Preview Smoke Test

Adds a smoke test job to the frontend PR workflow. On every PR that touches frontend/**, the job waits for Vercel to finish deploying, then verifies the preview is healthy.

How it works

Phase 1 — Wait for Vercel deployments:

• Polls the commit status API for flagsmith-frontend checks (scoped to frontend only — won't wait for docs)
• Waits until all leave pending (real builds: waits for completion; skipped builds: immediately done)
• If a build fails, reports the failure and skips testing

Phase 2 — Smoke test the preview:

• Parses flagsmith-frontend-* URLs from the Vercel bot PR comment (short poll with 30s buffer for comment update timing)
• GET /config/project-overrides — Express server is running
• GET / — static files deployed + response body contains "Flagsmith" (content validation)

Properties

• No false positives — Phase 1 ensures the current build has landed before testing
• Non-blocking — continue-on-error: true, failures warn but don't gate merge
• Bounded — timeout-minutes: 15 on the job, --max-time 30 on each curl
• Handles skipped builds — Vercel's "Ignored Build Step" completes immediately; previous preview URL is still live
• No checkout needed — uses only gh api + curl

Review feedback addressed

• Job-level timeout
• Curl timeouts (--max-time 30)
• Non-blocking (continue-on-error: true)
• Merged duplicate API calls
• Dropped /health (behind VPN, returns 404 on public URLs)
• Body content validation on /

How did you test this code?

• Validated on a throwaway test PR (test: Validate smoke test (two-phase approach) #7264) with a real Vercel deployment
• Previous iteration caught a false-positive issue — smoke test passed before Vercel finished deploying. Fixed with the two-phase approach.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
docs	Ignored	Preview	Apr 16, 2026 2:08pm
flagsmith-frontend-preview	Ignored	Preview	Apr 16, 2026 2:08pm
flagsmith-frontend-staging	Ignored	Preview	Apr 16, 2026 2:08pm

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.29%. Comparing base (9943da9) to head (27cf50f).
⚠️ Report is 10 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #7192   +/-   ##
=======================================
  Coverage   98.29%   98.29%           
=======================================
  Files        1351     1351           
  Lines       50752    50752           
=======================================
  Hits        49885    49885           
  Misses        867      867           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[talissoncosta]

Update: the previous approach produced false positives — it was testing URLs before Vercel finished deploying, hitting the stale previous build.

Rewrote to a two-phase approach:

1. Wait for Vercel deployment statuses to complete (polls commit status API for flagsmith-frontend checks)
2. Parse URLs from the bot comment and test /config/project-overrides + / with content validation

Also dropped /health (behind VPN now). Validating on #7264.

[Zaimwa9]

Last blocking point to me is the continue-on-error that defeats the purpose of blocking the deployment if the target is unhealthy.
I'd separate 2 cases:

1. We couldn't check because of timeout, no statuses or no comment (we don't know but nothing hints in not deploying) => this would be acceptable to skip
2. We managed to send a request to vercel and it failed, the app is clearly down => here we block deployment

I'm unsure how to achieve this but I think we could leverage on the exit status (0 when we want to continue; 1 => we block)

[Zaimwa9]

I'm just worried that this would let the deployment happen in case it raised a real error

[talissoncosta]

Closing — the approach is too brittle. Scraping the Vercel bot's PR comment to extract the preview URL means our CI depends on a third-party comment format we don't control, and silently breaks if Vercel changes it. If we ever want this check, it should pull the preview URL from a reliable source (Vercel API, deployment status, or a job output from Vercel's own check) rather than parsed markdown.