Fix relative ordering in a complex transitive cases by victor-nexthop · Pull Request #242 · pytest-dev/pytest-order

victor-nexthop · 2026-06-01T22:15:29Z

We only use before and after constraints edit: and a first/last, and sometimes, with sparse transitive dependencies, pytest-order won't produce the order that we expect, silently ignoring one of the constraints.

This is a set of changes that replaces the current algorithm with a topological sort, and adds a warning about impossible sets of constraints, for example: @order(first) def test_foo(): pass; @order(before="test_foo") def test_bar(): pass.

Created mostly by Claude.

This is a conversation starter. Please let me know if it needs to be squashed and split into smaller commits.

…s ordering.

apply_relative_constraints() unconditionally rebuilt sorted_list as start_pinned + sorted_middle + end_pinned, discarding the unordered items that sort_numbered_items() had interspersed between numbered items in sparse ordering mode. Early-return when there are no relative or dependency marks to preserve the layout produced by sort_numbered_items(). Fixes 5 sparse ordering tests.

for more information, see https://pre-commit.ci

mrbean-bremen · 2026-06-03T09:15:15Z

Thanks! I will have a closer look at this, though this may take some time (just back from traveling).

mrbean-bremen · 2026-06-03T10:48:19Z

+
+pytest-order emits a WARNING for each such impossible constraint and preserves
+the absolute ordinal positions unchanged.
+"""


This changes the current behavior, which was intentional. Explicit dependencies between tests shall be prioritized over ordinal sorting - I still think that makes sense.
Can you elaborate why you want to change this?

I was just in the mindset of "all ordering shall be satisfied", and didn't read the docs that deep. Wouldn't have made it a requirement.

I think using topological sorting still makes sense?

I'll rework the changes according to the design decision, likely next week.

...reworked.

…n decision.

victor-nexthop · 2026-06-11T06:01:12Z

Pushed a new version. It looks decent to me.
This PR is meant to be squashed into one commit.

mrbean-bremen · 2026-06-11T06:44:31Z

+  absolute ordinal markers (`index`, `first`, `last`, …); a conflicting ordinal
+  position is relaxed instead of dropping the relative marker
+- transitive relative chains are resolved as a single globally consistent order
+


I think this merits a patch release after it is merged.

Speaking of a patch. This may be a breaking change, because:

I found that before, some constraints were dropped silently, and after the chance they produce a warning. I don't have an example in tests now, would you require one?

Obviously, ordering for some cases has changed - I had to update tests to match it. Specifically, old versions of tests.test_relative_ordering.test_dependency_loop and tests.test_relative_ordering.test_failed_tests_after_dependency_loop fail on the new code.

And of course there are customers who rely on the old, incomplete ordering 😆 😢

Well, I still think of it as a bug fix. I know you are thinking of Hyrum's Law, but you just made cases possible that before have been handled (incorrectly) as a dependency loop. If somebody relied on that behavior to prevent some implied ordering, I certainly wouldn't bother about it...

Also the warning was missing before, so it just improved the visibility of an incorrect marker.

… - _sort_by_topology() needs the dependencies before _apply_iterative() mutates it.

for more information, see https://pre-commit.ci

victor-nexthop · 2026-06-11T17:50:00Z

+## Unreleased
+
+### Fixes
+- relative order markers (`before`/`after`) now always take preference over


I wonder if this comment is necessary - this is the way it always worked, isn't it?
Or was it only about combination of relative/absolute on the same item?

Yes, I actually wanted to ask in which cases it didn't work - I thought that you maybe had found such a case.
But yes, this should have been the way it work before (i.e., the comment can be removed).

Removed that item. No, I didn't see any violations of this behavior.

…e way it always worked.

victor-nexthop · 2026-06-11T20:43:38Z

Comments addressed. Please lmk if you see anything else.
I'll be happy to see it released!

mrbean-bremen

Thanks!

victor-nexthop and others added 7 commits May 28, 2026 19:46

Topological sorting for relative ordering.

c3ad027

Topological sorting: add warning for impossible combination of rel+ab…

155250c

…s ordering.

Allow items with both absolute and relative markers to be reordered

b792b7a

Fix: Allow items with both absolute and relative markers to be reordered

e04750e

Pre-screen impossible mark combinations.

f259605

[pre-commit.ci] auto fixes from pre-commit.com hooks

487a103

for more information, see https://pre-commit.ci

victor-nexthop commented Jun 1, 2026

View reviewed changes

Comment thread tests/test_relative_ordering.py

victor-nexthop commented Jun 1, 2026

View reviewed changes

Comment thread tests/test_relative_ordering.py

victor-nexthop changed the title ~~Fix~~ Fix relative ordering in a complex transitive cases Jun 2, 2026

mrbean-bremen reviewed Jun 3, 2026

View reviewed changes

victor-nexthop added 3 commits June 10, 2026 19:01

Vibed a fix for "Combination of absolute and relative ordering" desig…

f30a168

…n decision.

This wasn't really needed.

21875a5

Restore the failure in test_failed_tests_after_dependency_loop

6f88ef6

mrbean-bremen reviewed Jun 11, 2026

View reviewed changes

victor-nexthop and others added 3 commits June 11, 2026 10:33

Addressed review comments.

d53281e

reading self.rel_marks/dep_marks from _sort_by_topology() broke tests…

4e5530c

… - _sort_by_topology() needs the dependencies before _apply_iterative() mutates it.

[pre-commit.ci] auto fixes from pre-commit.com hooks

0b0749f

for more information, see https://pre-commit.ci

victor-nexthop commented Jun 11, 2026

View reviewed changes

remove excess mention of "relative vs absolute conflict" - this is th…

125d485

…e way it always worked.

mrbean-bremen approved these changes Jun 12, 2026

View reviewed changes

mrbean-bremen merged commit c411fc8 into pytest-dev:main Jun 12, 2026
16 checks passed

Conversation

victor-nexthop commented Jun 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mrbean-bremen commented Jun 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

victor-nexthop Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

victor-nexthop commented Jun 11, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

victor-nexthop Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mrbean-bremen Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

victor-nexthop commented Jun 11, 2026

Uh oh!

mrbean-bremen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

victor-nexthop commented Jun 1, 2026 •

edited

Loading

victor-nexthop Jun 4, 2026 •

edited

Loading

victor-nexthop Jun 11, 2026 •

edited

Loading

mrbean-bremen Jun 11, 2026 •

edited

Loading