Skip to content

ci: Add package pre-publish check #1893

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vdusek merged 7 commits into from
May 18, 2026
Merged

ci: Add package pre-publish check #1893

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
7ca1e0b
ci: Add package pre-publish check
vdusek May 12, 2026
17c95fe
ci: inline package-check recipe, drop shared reusable workflow indire…
vdusek May 14, 2026
92ad341
ci: extract package_check job into _check_package.yaml
vdusek May 14, 2026
337668f
ci: point package check at apify/actions feature branch
vdusek May 18, 2026
73c912c
ci: retrigger package check against updated action ref
vdusek May 18, 2026
6b4367b
ci: point package check at apify/actions main branch
vdusek May 18, 2026
70ec899
ci: pin package check to apify/actions v1.1.0
vdusek May 18, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/workflows/_check_package.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Package check

on:
# Runs when manually triggered from the GitHub UI.
workflow_dispatch:

# Runs when invoked by another workflow.
workflow_call:

permissions:
contents: read

jobs:
package_check:
name: Package check
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v6

- name: Set up uv package manager
uses: astral-sh/setup-uv@v8.1.0
with:
python-version: "3.14"

- name: Build sdist and wheel
run: uv run poe build

- name: Verify built package
uses: apify/actions/python-package-check@v1.1.0
with:
package_name: crawlee
dist_dir: dist
python_version: "3.14"
extras: all
smoke_code: |
from crawlee.crawlers import (
HttpCrawler, BeautifulSoupCrawler, ParselCrawler,
PlaywrightCrawler, AdaptivePlaywrightCrawler,
)
from crawlee.storages import Dataset, KeyValueStore, RequestQueue
from crawlee.http_clients import HttpxHttpClient, ImpitHttpClient
from crawlee import Request
HttpCrawler()
BeautifulSoupCrawler()
ParselCrawler()
Comment on lines +29 to +46
Copy link
Copy Markdown
Collaborator

@janbuchar janbuchar May 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this is repeated three times in the diff - is that necessary? Or a lesser evil than setting up a reusable workflow/action?

Copy link
Copy Markdown
Collaborator Author

@vdusek vdusek May 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first, I tried to implement it as a dedicated reusable workflow, but I didn't see much benefit there. This is just a configuration, and each repo has its own smoke test, can have a different rollout timing for new Python versions, and so on. When I prepared it, it felt like an unnecessary middle layer with very little added value. So I removed it and kept only the action itself there.

19 changes: 19 additions & 0 deletions .github/workflows/manual_release_beta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,25 @@ jobs:
version_number: ${{ needs.release_prepare.outputs.version_number }}
ref: ${{ needs.changelog_update.outputs.changelog_commitish }}

- name: Verify built package
uses: apify/actions/python-package-check@v1.1.0
with:
package_name: crawlee
dist_dir: dist
python_version: "3.14"
extras: all
smoke_code: |
from crawlee.crawlers import (
HttpCrawler, BeautifulSoupCrawler, ParselCrawler,
PlaywrightCrawler, AdaptivePlaywrightCrawler,
)
from crawlee.storages import Dataset, KeyValueStore, RequestQueue
from crawlee.http_clients import HttpxHttpClient, ImpitHttpClient
from crawlee import Request
HttpCrawler()
BeautifulSoupCrawler()
ParselCrawler()

# Publish the package to PyPI using PyPA official GitHub action with OIDC authentication.
- name: Publish package to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Expand Down
20 changes: 20 additions & 0 deletions .github/workflows/manual_release_stable.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,26 @@ jobs:
is_prerelease: ""
version_number: ${{ needs.release_prepare.outputs.version_number }}
ref: ${{ needs.changelog_update.outputs.changelog_commitish }}

- name: Verify built package
uses: apify/actions/python-package-check@v1.1.0
with:
package_name: crawlee
dist_dir: dist
python_version: "3.14"
extras: all
smoke_code: |
from crawlee.crawlers import (
HttpCrawler, BeautifulSoupCrawler, ParselCrawler,
PlaywrightCrawler, AdaptivePlaywrightCrawler,
)
from crawlee.storages import Dataset, KeyValueStore, RequestQueue
from crawlee.http_clients import HttpxHttpClient, ImpitHttpClient
from crawlee import Request
HttpCrawler()
BeautifulSoupCrawler()
ParselCrawler()

# Publish the package to PyPI using PyPA official GitHub action with OIDC authentication.
- name: Publish package to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/on_pull_request.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ jobs:
name: Code checks
uses: ./.github/workflows/_check_code.yaml

package_check:
name: Package check
uses: ./.github/workflows/_check_package.yaml

tests:
name: Tests
uses: ./.github/workflows/_tests.yaml
Expand Down
Loading