Skip to content

feat(cli): add checkly api generic API pass-through command#1320

Open
martzoukos wants to merge 4 commits into
mainfrom
martzoukos/ai-132-api-pass-through
Open

feat(cli): add checkly api generic API pass-through command#1320
martzoukos wants to merge 4 commits into
mainfrom
martzoukos/ai-132-api-pass-through

Conversation

@martzoukos
Copy link
Copy Markdown
Contributor

Why

The Checkly CLI has dedicated commands for a handful of API endpoints, but the Checkly API has ~150 endpoints total. Today, users and AI agents that need to reach an uncovered endpoint have to fall back to curl with manually constructed auth headers — which is error-prone and breaks agent workflows.

This is especially important for AI agents: when an agent needs data from an endpoint that doesn't have a CLI command yet, it's stuck. This command gives agents (and power users) an immediate escape hatch — authenticated curl for the entire Checkly API, similar to how gh api works for GitHub.

Linear: AI-132

What changed

New command: checkly api <endpoint>

A generic pass-through that makes authenticated HTTP requests to any Checkly API endpoint. The CLI injects auth headers and the correct base URL automatically.

checkly api /v1/checks
checkly api /v1/dashboards -X GET --jq '.[].name'
checkly api /v1/checks -X POST -F name=MyCheck -F activated:=true

Flags:

  • -X / --method — HTTP method (GET, POST, PUT, PATCH, DELETE). Defaults to GET, or POST when fields are present
  • -f — String field parameter (key=value)
  • -F — Typed field parameter (key=value for strings, key:=value for JSON like booleans/numbers/arrays)
  • -H — Custom HTTP header
  • --jq — Filter JSON output using system jq (clear error message if jq isn't installed)
  • --input — Request body from a file path or - for stdin
  • --paginate — Auto-fetch all pages (supports both Content-Range and cursor-based pagination)
  • --include / -i — Show response status and headers
  • --verbose — Print full request/response headers to stderr

Output: Pretty-printed JSON in the terminal, compact JSON when piped (matches gh api behavior).

AI context / skills update

Updated the Checkly skill file (SKILL.md) so AI agents know checkly api exists as a fallback. The section points to the Checkly API reference for endpoint discovery — no static endpoint list baked in, so it won't go stale when the backend API changes.

Security

  • URL validation — Rejects absolute URLs (https://evil.com/...) and protocol-relative URLs (//evil.com/...) to prevent leaking auth headers to external hosts
  • No shell injection — The --jq flag uses execFile (not exec), so the filter expression is passed as a direct argument to the jq binary with no shell interpretation
  • 404 docs hint — On a 404 response, stderr shows a link to the API docs to help users find the right endpoint

New files

File Purpose
packages/cli/src/commands/api.ts The command
packages/cli/src/helpers/api-fields.ts Parses -f / -F field flags
packages/cli/src/helpers/api-paginate.ts Auto-pagination (Content-Range + cursor)
packages/cli/src/commands/__tests__/api.spec.ts Command unit tests (19 tests)
packages/cli/src/helpers/__tests__/api-fields.spec.ts Field parsing tests (11 tests)
packages/cli/src/helpers/__tests__/api-paginate.spec.ts Pagination tests (5 tests)

Manual testing

Requires CHECKLY_API_KEY and CHECKLY_ACCOUNT_ID set, or a prior checkly login.

# Basic GET
./packages/cli/bin/run api /v1/checks

# Limit results
./packages/cli/bin/run api /v1/checks -X GET -f limit=2

# Filter with jq
./packages/cli/bin/run api /v1/checks --jq '.[].name'

# Verbose headers
./packages/cli/bin/run api /v1/checks --verbose

# Paginate all checks
./packages/cli/bin/run api /v1/checks --paginate

# Cursor-based pagination (status pages)
./packages/cli/bin/run api /v1/status-pages --paginate

# Include response headers
./packages/cli/bin/run api /v1/checks -i

# Compact JSON when piped
./packages/cli/bin/run api /v1/checks | jq length

# 404 with docs hint
./packages/cli/bin/run api /v1/nonexistent

# Security: rejects absolute URLs
./packages/cli/bin/run api https://evil.com/steal
./packages/cli/bin/run api //evil.com/steal

Test plan

  • Unit tests pass (141 total — 19 command + 11 field parsing + 5 pagination + 106 existing metadata)
  • ESLint passes
  • Manual smoke test against live API with real credentials
  • Verify checkly api --help output
  • Verify updated skill via ./packages/cli/bin/run skills install

🤖 Generated with Claude Code

martzoukos and others added 3 commits May 25, 2026 16:35
Adds a new `checkly api` command that lets users and AI agents call any
Checkly API endpoint directly, with authentication handled automatically.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Address all findings from multi-persona code review: fix silent jq
error swallowing, add HTTP status checks on paginated follow-up
requests, default --input to POST, guard --paginate on GET-only,
fix HTTP/1.1 status line format, extract shared Content-Range parser,
replace process.stderr.write with oclif logToStderr, add max-pages
guard and entries type validation, and add 401/403 error hints.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@martzoukos martzoukos requested review from sorccu and stefanjudis May 25, 2026 14:08

Run `npx checkly skills communicate` for the full protocol details.

## API Pass-Through (fallback for any endpoint)
Copy link
Copy Markdown
Contributor Author

@martzoukos martzoukos May 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stefanjudis I'd appreciate your 2c on this section regarding agent ergonomics.

@martzoukos martzoukos requested a review from thebiglabasky May 25, 2026 14:28
Copy link
Copy Markdown
Contributor

@thebiglabasky thebiglabasky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd love to discuss a few points before approving. Nothing really blocking, but also, not yet confident we want to go exactly as is

Comment thread packages/cli/src/commands/__tests__/api.spec.ts
Comment thread packages/cli/src/ai-context/skill.md Outdated
Comment thread packages/cli/src/commands/__tests__/api.spec.ts
Comment thread packages/cli/src/commands/api.ts Outdated
Comment thread packages/cli/src/commands/api.ts Outdated
Comment thread packages/cli/src/commands/api.ts
Comment thread packages/cli/src/commands/api.ts
Comment thread packages/cli/src/helpers/api-paginate.ts
Comment thread packages/cli/src/helpers/api-paginate.ts Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was about to propose to move this into api-paginate, but see it's used by the REST client, I guess it's fine. Looks like a tiny file now :)

@martzoukos
Copy link
Copy Markdown
Contributor Author

Great feedback @thebiglabasky !
I've addressed the following:

  • destructive = true — Flipped the flag since the command can PUT/PATCH/DELETE resources
  • Hardened URL validation — Added hostname verification after URL resolution (not just regex on the raw input). Even if someone encodes characters or uses backslashes, the resolved URL's origin must match the API base URL. Added a test for the backslash bypass case
  • Fixed cursor pagination detection — Now keys on nextId alone instead of requiring both nextId and entries. Dynamically finds whichever array field is in the response (entries, items, etc.). This fixes a real bug where --paginate silently failed for /v1/accounts/{accountId}/members which returns items instead of entries
  • Added OpenAPI spec URL — Added api.checklyhq.com/openapi.json alongside the docs link in both the command description and the AI skill file, so agents can use the machine-readable spec for endpoint discovery

- Mark command as destructive (supports
  PUT/PATCH/DELETE)
  - Harden URL validation with hostname verification after resolution
  - Fix cursor pagination to detect nextId alone, supporting both entries and items array fields
  - Add OpenAPI spec URL alongside docs reference

  Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@martzoukos martzoukos requested a review from thebiglabasky May 26, 2026 10:42
@sorccu
Copy link
Copy Markdown
Member

sorccu commented May 26, 2026

Can you show me how you'd do something more complex such as setting a retry strategy for a check or group?

@martzoukos
Copy link
Copy Markdown
Contributor Author

Can you show me how you'd do something more complex such as setting a retry strategy for a check or group?

By doing something like:

checkly api /v1/checks/YOUR_CHECK_ID -X PUT --input - <<'EOF'
{
"retryStrategy": {
    "type": "LINEAR",
    "baseBackoffSeconds": 10,
    "maxRetries": 3,
    "maxDurationSeconds": 600,
    "sameRegion": true
}
}
EOF

Are you asking about multi-line parameters in general or about something specific to retries that I'm missing?

Copy link
Copy Markdown
Contributor

@thebiglabasky thebiglabasky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fact this mimics gh makes me pretty confident in the design. I didn't test it locally, but trust that you did. Fine work!

@martzoukos
Copy link
Copy Markdown
Contributor Author

@sorccu anything regarding your question above that needs addressing or shall I go ahead and merge it?

@sorccu
Copy link
Copy Markdown
Member

sorccu commented May 26, 2026

Can we delay merging till at least tomorrow? I'd like to think about this a bit. To be clear I'm for the idea, but I'm not sure if our API is ready to be exposed like this

@martzoukos
Copy link
Copy Markdown
Contributor Author

Yeah, that works @sorccu . 👍

Copy link
Copy Markdown
Member

@MegaMaddin MegaMaddin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check internal discussion in #cli.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants