backendcluster: add cluster manager and cluster-scoped topology runtime by YangKeao · Pull Request #1104 · pingcap/tiproxy

YangKeao · 2026-03-19T16:45:32Z

What problem does this PR solve?

Issue Number: close #1098

What is changed and how it works:

Introduce a backend-cluster manager that owns cluster-scoped runtime instances.

This PR adds:

a manager for configured backend clusters
one runtime per backend cluster
cluster-scoped etcd / infosync / shared clients
topology aggregation across clusters
dynamic add / update / remove handling when backend-cluster config changes

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Notable changes

Has configuration change
Has HTTP API interfaces change
Has tiproxyctl change
Other user behavior changes

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

ti-chi-bot · 2026-03-19T16:45:36Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

codecov-commenter · 2026-03-19T17:45:28Z

Codecov Report

❌ Patch coverage is 68.75000% with 110 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@4d841da). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
pkg/manager/backendcluster/manager.go	64.88%	47 Missing and 12 partials ⚠️
pkg/manager/backendcluster/cluster.go	63.88%	11 Missing and 2 partials ⚠️
pkg/balance/router/group.go	59.09%	9 Missing ⚠️
pkg/server/server.go	58.82%	6 Missing and 1 partial ⚠️
pkg/balance/observer/health_check.go	44.44%	4 Missing and 1 partial ⚠️
pkg/proxy/backend/backend_conn_mgr.go	54.54%	4 Missing and 1 partial ⚠️
pkg/balance/router/router.go	55.55%	4 Missing ⚠️
pkg/balance/router/router_static.go	0.00%	4 Missing ⚠️
pkg/manager/namespace/manager.go	0.00%	3 Missing ⚠️
pkg/manager/backendcluster/backend_id.go	90.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1104   +/-   ##
=======================================
  Coverage        ?   67.09%           
=======================================
  Files           ?      144           
  Lines           ?    15081           
  Branches        ?        0           
=======================================
  Hits            ?    10118           
  Misses          ?     4270           
  Partials        ?      693

Flag	Coverage Δ
unit	`67.09% <68.75%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

YangKeao · 2026-03-20T04:53:24Z

/test all

YangKeao · 2026-03-20T05:02:29Z

TestLocalDirWatcher is not stable. Let me try to stablize it.

/retest

YangKeao · 2026-03-20T06:22:20Z

/test all

djshow832 · 2026-03-27T03:01:59Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 82ae03a9dc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

pkg/server/server.go

pkg/manager/namespace/manager.go

pkg/balance/observer/health_check.go

djshow832 · 2026-04-01T01:17:33Z

pkg/balance/router/group.go

-			StatusPort: 10080, // impossible anyway
-		},
+	g.lg.Warn("backend is not found in the router", zap.String("backend_id", backendID), zap.Stack("stack"))
+	backend = newBackendWrapper(backendID, observer.BackendHealth{


Why remove BackendInfo here? Is it safe to remove it?

A mistake. I tried to find some way to fill in these fields without parsing backendID but failed.

For now, I think it's fine to parse backendID for this strange branch.

pkg/manager/backendcluster/manager.go

Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>

pkg/balance/router/group.go

Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>

ti-chi-bot · 2026-04-02T11:16:13Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: djshow832

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [djshow832]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2026-04-02T11:16:13Z

[LGTM Timeline notifier]

Timeline:

2026-04-02 11:16:13.190136948 +0000 UTC m=+436578.395497005: ☑️ agreed by djshow832.

ti-chi-bot bot added the do-not-merge/work-in-progress label Mar 19, 2026

ti-chi-bot bot requested review from bb7133 and xhebox March 19, 2026 16:45

ti-chi-bot bot added the size/XXL label Mar 19, 2026

YangKeao force-pushed the pr/03-multi-cluster-runtime branch from 98ea284 to 3993ee3 Compare March 19, 2026 17:33

YangKeao marked this pull request as ready for review March 19, 2026 17:37

ti-chi-bot bot removed the do-not-merge/work-in-progress label Mar 19, 2026

ti-chi-bot bot requested a review from djshow832 March 19, 2026 17:37

YangKeao marked this pull request as draft March 20, 2026 04:37

ti-chi-bot bot added the do-not-merge/work-in-progress label Mar 20, 2026

YangKeao force-pushed the pr/03-multi-cluster-runtime branch from 3993ee3 to a7c5384 Compare March 20, 2026 04:52

YangKeao force-pushed the pr/03-multi-cluster-runtime branch 2 times, most recently from 25a54ca to cd317e4 Compare March 20, 2026 06:20

YangKeao force-pushed the pr/03-multi-cluster-runtime branch 7 times, most recently from f97603c to 1add4f8 Compare March 24, 2026 13:52

YangKeao marked this pull request as ready for review March 24, 2026 13:57

ti-chi-bot bot removed the do-not-merge/work-in-progress label Mar 24, 2026

YangKeao force-pushed the pr/03-multi-cluster-runtime branch from 1add4f8 to af488c9 Compare March 24, 2026 14:21

chatgpt-codex-connector bot reviewed Mar 27, 2026

View reviewed changes

pkg/server/server.go Show resolved Hide resolved

pkg/server/server.go Show resolved Hide resolved

djshow832 reviewed Mar 27, 2026

View reviewed changes

pkg/manager/namespace/manager.go Outdated Show resolved Hide resolved

pkg/balance/observer/health_check.go Outdated Show resolved Hide resolved

YangKeao force-pushed the pr/03-multi-cluster-runtime branch 3 times, most recently from 09b1223 to dfe1eac Compare March 31, 2026 16:01

backendcluster: add cluster manager and cluster-scoped topology runtime

e9c14ce

YangKeao force-pushed the pr/03-multi-cluster-runtime branch from dfe1eac to e9c14ce Compare March 31, 2026 16:27

YangKeao requested a review from djshow832 April 1, 2026 02:32

djshow832 reviewed Apr 1, 2026

View reviewed changes

YangKeao force-pushed the pr/03-multi-cluster-runtime branch from c6b881d to c6de019 Compare April 2, 2026 09:28

refactor the codes according to comments

d5daa9c

Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>

YangKeao force-pushed the pr/03-multi-cluster-runtime branch from c6de019 to d5daa9c Compare April 2, 2026 09:35

djshow832 reviewed Apr 2, 2026

View reviewed changes

pkg/balance/router/group.go Outdated Show resolved Hide resolved

parse backend id in backendcluster package

43b9b83

Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>

djshow832 approved these changes Apr 2, 2026

View reviewed changes

ti-chi-bot bot added the lgtm label Apr 2, 2026

ti-chi-bot bot added the approved label Apr 2, 2026

ti-chi-bot bot merged commit 30d5b3f into pingcap:main Apr 2, 2026
6 checks passed

Conversation

YangKeao commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Check List

Release note

Uh oh!

ti-chi-bot bot commented Mar 19, 2026

Uh oh!

codecov-commenter commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

YangKeao commented Mar 20, 2026

Uh oh!

YangKeao commented Mar 20, 2026

Uh oh!

YangKeao commented Mar 20, 2026

Uh oh!

djshow832 commented Mar 27, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

djshow832 Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

YangKeao Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ti-chi-bot bot commented Apr 2, 2026

Uh oh!

ti-chi-bot bot commented Apr 2, 2026

[LGTM Timeline notifier]

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

YangKeao commented Mar 19, 2026 •

edited

Loading

codecov-commenter commented Mar 19, 2026 •

edited

Loading

YangKeao Apr 2, 2026 •

edited

Loading