Fix OrderReporter producing empty test_order.log in parallel DRb mode#387
Closed
Fix OrderReporter producing empty test_order.log in parallel DRb mode#387
Conversation
97702b0 to
24ac7da
Compare
## Situation
OrderReporter writes test identifiers in its before_test method, but in
Rails parallel DRb mode (forked processes), before_test is never called.
The parallelization server only calls prerecord and record on the parent
process's reporters. Since tests execute in forked workers via
Minitest.run_one_method (which bypasses reporters entirely), the
test_order.log file is truncated on start and never written to.
This makes it impossible to bisect order-dependent flaky test failures
because the test order is never recorded.
## Execution
Instead of relying on the reporter API (which lacks worker identity in
DRb mode), prepend a TestOrderTracking module onto Minitest::Test that
hooks into before_setup. This fires inside the forked worker process
during run_one_method, where Process.pid correctly identifies the worker.
Each forked worker writes to its own file (test_order.worker-{pid}.log)
with sync writes enabled (workers never call report, so buffered writes
risk data loss on crash). The parent process's main test_order.log is
used in non-parallel mode where the parent is the test process.
On start, stale worker files from previous runs are cleaned up.
24ac7da to
14b8878
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Situation
OrderReporterwrites test identifiers in itsbefore_testmethod, but in Rails parallel DRb mode (forked processes withparallelize(workers: N, with: :processes)),before_testis never called. The parallelization server only callsprerecordandrecordon the parent process's reporters. Since tests execute in forked workers viaMinitest.run_one_method(which bypasses reporters entirely),test_order.logis truncated on start and never written to.This makes it impossible to bisect order-dependent flaky test failures because the test order is never recorded.
Execution
The reporter API can't solve this alone: in DRb mode,
prerecord/recordrun in the parent process with no worker identity, so we can't produce per-worker files from there.Instead, we prepend a
TestOrderTrackingmodule ontoMinitest::Test#before_setup, which fires inside the forked worker duringrun_one_method. There,Process.pidcorrectly identifies the worker.Each forked worker writes to its own file (
test_order.worker-{pid}.log) with sync writes enabled, since workers never callreportand buffered writes risk data loss on crash. The parent'stest_order.logis used in non-parallel mode where the parent runs tests directly. Stale worker files from previous runs are cleaned up onstart.To find which worker ran a failing test:
grep test_name log/test_order.worker-*.log