This repository was archived by the owner on May 25, 2026. It is now read-only.
feat: sqle-gaussdb-plugin GaussDB / openGauss structure diff (#2905)#1
Open
LordofAvernus wants to merge 18 commits into
Open
feat: sqle-gaussdb-plugin GaussDB / openGauss structure diff (#2905)#1LordofAvernus wants to merge 18 commits into
LordofAvernus wants to merge 18 commits into
Conversation
…l build (#2905)
为让 sqle-gaussdb-plugin 在无内网 GOPROXY 的本地开发沙箱内也能完成 `go build`：
- replace 块新增 `github.com/pingcap/parser => github.com/sjjian/parser
v0.0.0-20240704052347-b6199b7bccae`：sqle 主仓 vendor 已锁定该 fork（含
parser.Token 类型扩展），不加这条 replace 时 ../sqle/sqle/driver/mysql/splitter
会报 `undefined: parser.Token` 编译错误。版本号取自 sqle-pg-plugin/go.mod。
- replace 块固化 golang.org/x/{crypto,sys,net,text,sync} 为 sqle 主仓 vendor 实测使用
的 Go 1.19 兼容版本，避免 `go mod tidy` MVS 把 indirect 升到 v0.42 / v0.44 等
需要 Go 1.20+ 的版本（典型崩溃：crypto/ecdh / crypto/mlkem / slices 未在 GOROOT）。
- replace github.com/actiontech/dms => v0.0.0-20251027081421-309bc24335ca，与
sqle-pg-plugin/go.mod 实测对齐，避免引入更新版本带来的传递依赖飘移。
- go.sum 锁定：由 `go mod tidy` 在 GOPROXY=goproxy.cn + GOSUMDB=off 容器内生成。
依赖评估：本提交不引入新的第三方 package，仅在 replace 块固化已有 require 的版本号
与本地路径（开发沙箱专用）；CI / vendor 落盘阶段须移除 `../sqle` 与
`../sqle-pg-plugin` 本地路径 replace（README 已注明）。
验证：
- `go build ./cmd/sqle-gaussdb-plugin` 与 `go build ./cmd/sqle-opengauss-plugin`
在 golang:1.19.6 容器内均成功，分别产出 ~38MB linux/amd64 二进制
- `go vet ./cmd/... ./internal/... ./pkg/...` 通过
Refs: actiontech/sqle-ee#2905
补充内容： - 增加「开发期沙箱本地构建（无内网 GOPROXY 时）」小节，给出 docker run 完整命令 （含 --network host / GOPROXY=goproxy.cn / GOSUMDB=off），方便后续 Task-Dev-003+ 在同一沙箱内复用。 - 在「构建方式」小节顶部加 callout：CI / vendor 落盘前须移除 go.mod 内 `../sqle` 与 `../sqle-pg-plugin` 两条本地路径 replace，require 块版本号已是 生产形态。 Refs: actiontech/sqle-ee#2905
Task-Dev-003 实现 Dialector 时，internal/dialector/dialector.go 直接 import `github.com/pkg/errors` 用于 Open() 错误包装。`go mod tidy` 因此把该模块从 indirect 提升为 direct require（仅 go.mod 注释行变动，go.sum 与 vendor 解析 结果不变）。 依赖评估：本提交不引入新的第三方 package，仅是已有 indirect 依赖的直接化 标注变化。github.com/pkg/errors v0.9.1 与 sqle 主仓 vendor 实测使用版本严格 一致（License=BSD-2-Clause，长期维护，sqle / sqle-pg-plugin / dms 全系 直接或间接依赖）。 验证： - `docker run golang:1.19.6 go mod tidy` 在 sqle-gaussdb-plugin 仓库内 GOPROXY=goproxy.cn / GOSUMDB=off 下成功，go.sum 无新行 - `go vet ./internal/dialector/...` 通过 - 后续业务代码 commit 已就绪，与本提交解耦 Refs: actiontech/sqle-ee#2905
替换 Task-Dev-002a 占位实现，落地 design §4.3 关键片段： - DriverName() 固定返回 "opengauss"（database/sql 驱动名，由 dialector.go 文件头的 `_ import` 注册 gitee.com/opengauss/openGauss-connector-go-pq）。 GaussDB 与 openGauss 共用同一驱动，仅 DSN 内容不同。 - DefaultPort() 按 Variant 分支：consts.PluginNameGaussDB → 8000、 consts.PluginNameOpenGauss → 5432、未知 / 空 Variant → 0（兜底，被单测 显式覆盖）。 - BuildDSN(*driverV2.DSN) 输出 libpq key=value 串：5 个基础字段 (host / port / user / password / dbname) + connect_timeout=5； GaussDB 追加 sslmode=disable，openGauss 不显式声明 sslmode（保留驱动 默认 prefer）；DSN.AdditionalParams 追加在末尾，按 libpq last-wins 规则覆盖前面同 key 默认值。 - 所有字段值通过 escape() 包成单引号串，反斜杠加倍、单引号 \' 转义，与 github.com/lib/pq url.go 的连接串转义策略一致。Open() 通过 BuildDSN + BaseDialector.GetConn 复用同一段 DSN 构造逻辑，保证单测 (字符串断言) 与运行时 (sql.Open) 完全同源。 PluginName 字面值严格引用 pkg/consts (PluginNameGaussDB / PluginNameOpenGauss / DefaultPortGaussDB / DefaultPortOpenGauss)，禁止在 dialector.go 内 硬编码字符串。 design §4.3 伪代码 BuildDSN 入参写 *driverV2.Config 但访问 cfg.Host 等 DSN 内部字段；real sqle 链路 Dialector.Open 永远传 *driverV2.DSN。本实现 直接采用 *driverV2.DSN 入参以保持代码自洽，文件头部注释已说明该等价决策。 单元测试覆盖 design §17.1.1 全部分支，4 个 test 函数共 13 条 sub-case： - TestDialector_DefaultPort: 4 条（GaussDB → 8000 / openGauss → 5432 / 空 Variant → 0 / 未知 Variant → 0）。 - TestDialector_BuildDSN_BaselineFields: 3 条（GaussDB 5 字段 + sslmode= disable / openGauss 5 字段 + sslmode 必须不出现 / GaussDB 空 DatabaseName 回退 postgres）。 - TestDialector_BuildDSN_AdditionalParamsAndEscape: 5 条（GaussDB 用户 sslmode=verify-full 覆盖默认 / openGauss 用户 sslmode=require 落入 DSN / 单引号 password it's 转义 / 反斜杠 password 转义 / 空 key AdditionalParam 防御性忽略）。 - TestDialector_DriverName: 锁 DriverName=="opengauss" 不漂移。 全部 case 使用 Go map[string]struct 结构（不用 slice + index），纯字符串 断言，零 sql.Open / time.Sleep / 网络 / 磁盘依赖（todo.md Task-Dev-003 §C 硬约束）。dupeKeyLast 通过 strings.LastIndex 验证用户 override key 是末 位 occurrence，与 libpq last-wins 解析等价。 依赖评估：本提交不引入新依赖。pkg/errors 提升为 direct require 已在前一 commit (deps:) 单独完成。 验证（docker golang:1.19.6 + GOPROXY=goproxy.cn / GOSUMDB=off）: - go vet ./internal/dialector/... 通过（无输出） - go test -v ./internal/dialector/... 全部 PASS（4 test fn / 13 sub-case） - go mod tidy 仅产生前一 commit 的 go.mod 单行注释变化，无 go.sum diff Refs: actiontech/sqle-ee#2905
driver_test.go consumes sqlmock.New / sqlmock.MonitorPingsOption / sqlmock.ExpectPing API directly so the dependency is no longer indirect. No version change — the v1.5.0 lock matches the previous indirect entry (go.sum unchanged, vendor not vendored in this repo). 评估：License=MIT, 最近更新=2024-09, sqle-pg-plugin / sqle-tbase-plugin 等 sqle 系插件都用同一版本做 *sql.DB mock。本仓库 Task-Dev-004 起首次直接 import sqlmock，按 Go 惯例应提升为 direct require。
…_opengauss with sqlmock unit tests (#2905)
Replace the Task-Dev-002 placeholder stub.go (which only delegated to
driverPkg.NewDriverImpl) with the real Driver implementation split into
three files per plan §4.1 / §4.2 / §4.3:
- internal/driver/driver_common.go
Shared *Driver struct embedding *driverPkg.DriverImpl. Inherits the
default driverV2.Driver method set from sqle main repo, overrides
Ping to delegate through *sql.DB.PingContext (design §5.1) so the
pool-level handle is used instead of a stale *sql.Conn. newCommon
factory applies the connection-pool policy MaxOpenConns=10 /
MaxIdleConns=5 / ConnMaxLifetime=30min on the *sql.DB after
DriverImpl construction (design §5.1 line 328; compat-RISK-2 pins
the values for future alignment with sqle-pg-plugin). wrapPGError
passes *pq.Error from gitee.com/opengauss/openGauss-connector-go-pq
(or the lib/pq fork it inherits from) through unchanged so callers
still see the raw 5-character SQLSTATE (design §5.3); detection is
done via reflect type-name + package-path comparison so this file
does not import the openGauss connector directly.
- internal/driver/driver_gaussdb.go
GaussDBDriver{*Driver} aggregation + NewGaussDBDriverImpl factory.
Signature preserved bit-for-bit from stub.go so
cmd/sqle-gaussdb-plugin/main.go does not need to change. Placeholder
GetDatabaseObjectDDL / GetDatabaseDiffModifySQL return the explicit
errNotImplemented sentinel pending Task-Dev-005 (extractor) /
Task-Dev-006 (differ), so DMS sees a deterministic failure rather
than the embedded DriverImpl default empty slice.
- internal/driver/driver_opengauss.go
Same shape as driver_gaussdb.go, only the Variant tag / PluginName
differ. The Driver layer is variant-agnostic on purpose; all
branching (default port, default sslmode, libpq DSN layout) lives in
internal/dialector (design §3.2 / §4.3).
- internal/driver/driver_test.go
Map-case style unit tests with DATA-DOG/go-sqlmock so no real
database is touched (design §17.1.5):
* TestDriver_PluginNameRegistration — 3 sub-cases covering both
Variants and the embed-chain Variant propagation.
* TestDriver_CapabilityMatrix — 20 sub-cases (2 variants × 10
OptionalModules) locking the capability surface: the 2 enabled
OptionalModules return errNotImplemented; the 8 disabled modules
return either the DriverImpl zero-value default or the
"database conn not initialized" sentinel for Query (which needs
*sql.Conn, deliberately not wired here).
* TestCapabilityMatrix_I18nMetadataOnly — 1 sub-case locking the
OptionalModuleI18n enum String() so a future iota reordering
cannot silently break the cmd/* main.go capability declaration.
* TestDriver_PingTimeout — 3 sub-cases (deadline / cancel /
nil-DB) with a 2-second harness timeout so a regression that
breaks ctx propagation fails the test instead of hanging the
suite.
* TestWrapPGError — 3 sub-cases locking the pointer-identity
pass-through contract for nil / plain / sentinel errors.
* TestApplyPoolPolicy — 2 sub-cases locking the non-nil DB pool
application and the nil-DB no-op defensive branch.
Total: 32 sub-cases passing.
- internal/driver/stub.go
Deleted in the same business commit per plan §4 step E. cmd/* main.go
continues to compile because NewGaussDBDriverImpl /
NewOpenGaussDriverImpl signatures are preserved.
Verification (golang:1.19.6 container, GOPROXY=goproxy.cn):
go vet ./internal/driver/... ./cmd/... — clean
go test -v ./internal/driver/... — 32 sub-cases PASS
go build -o /tmp/sqle-gaussdb-plugin ./cmd/... — OK
go build -o /tmp/sqle-opengauss-plugin ./cmd/... — OK
Capability matrix (design §5.2; cmd/*/main.go SetEnableOptionalModule
contract): only OptionalGetDatabaseObjectDDL +
OptionalGetDatabaseDiffModifySQL are advertised this period. All other
OptionalModule methods fall through to the DriverImpl default.
compat-RISK-2 (connection pool alignment) and compat-RISK-7 (SQLSTATE
pass-through) tracked in docs/dev/compat_risks.md; this commit fulfils
the in-code part of both mitigations.
data-upgrade: A-class no-op — no init() / startup hook / database write
is added by this commit (driver_common.go / driver_gaussdb.go /
driver_opengauss.go contain only construction-time pure code).
…with sqlmock unit tests (#2905)
Task-Dev-005 第一轮 Extractor 落地：在 internal/extractor/ 下新增 4 个文件，
覆盖 design §6.1 入口分发 + §6.2.1 TABLE + §6.2.2 VIEW + §6.3 注入防御
+ §6.5 空 DDL 兜底。
- extractor.go: Extractor struct + NewExtractor + Extract(ctx, info)
入口 switch，按 driverV2.ObjectType_* 字面值分发；本期实现 TABLE / VIEW
两个 case + default（design §6.1 line 399 字面错误）；FUNCTION /
PROCEDURE 留 TODO 注释指向 Task-Dev-006。
- table.go: extractTable 走 design §6.2.1 pg_get_tabledef + relkind
IN ('r','p')，$1 / $2 参数化绑定；sql.ErrNoRows → 空 DDL 不报错；
其他错误透传保留 SQLSTATE。
- view.go: extractView 走 design §6.2.2 显式拼 CREATE OR REPLACE VIEW
+ 服务端 quote_ident + pg_get_viewdef(c.oid, true) pretty-print；
$1 / $2 参数化绑定；sql.ErrNoRows → 空 DDL 不报错。
- extractor_test.go: 5 个 map case test 函数共 31 sub-case，
sqlmock.QueryMatcherEqual 锁定 SQL 文本字面值；覆盖
TABLE/VIEW 正常 + not_found + permission_denied + 不支持
ObjectType（INDEX/TRIGGER/EVENT/SEQUENCE/TYPE/PACKAGE/FUNCTION/
PROCEDURE/empty 9 条）+ QuoteIdent 注入防御（TABLE 8 条 + VIEW 6 条，
含空格 / 大小写 / 中文 / SQL 注入串 / 单双引号 / 混合特殊字符）+
nil info 和多对象顺序聚合兜底。
类型签名漂移说明（semantic/design_pseudocode_param_type_drift_20260521）：
design §6.1 伪代码假设每对象返回一个独立的
*driverV2.DatabaseSchemaObjectResult，real 链路
(sqle/sqle/driver/v2/driver_interface.go:431-463) 实际是 per-schema
聚合的 DatabaseObjectDDLs []*DatabaseObjectDDL；按"业务字段访问语义
优先 + 文件头注释 + commit 标注"准则，本实现采用 real 链路签名，
extractor.go 顶部已显式说明等价决策。
测试结果：
- go vet ./internal/extractor/... ./internal/driver/... ./cmd/... 通过
- go test -v ./internal/extractor/... 全 31 sub-case PASS
- go build ./cmd/sqle-gaussdb-plugin + ./cmd/sqle-opengauss-plugin 双
binary 编译通过（37M each）
约束遵守：
- 仅在 internal/extractor/ 目录工作，未动 driver / dialector / cmd /
pkg / Makefile / 8 个 sibling 仓库
- $1 / $2 参数化绑定，禁字符串拼接到 SQL（compat-RISK-4 兜底）
- ObjectType 字面值统一从 driverV2 包引用，禁硬编码 "TABLE"/"VIEW"
- 单测全部基于 sqlmock，禁 sql.Open / time.Sleep / 网络/磁盘依赖
- 无 go.mod / go.sum / vendor 改动，无新增第三方依赖
需求引用：需求 2.1（TABLE 差异）+ 需求 2.2（VIEW 差异）+ 需求 4.1
（PG / MySQL 不回归——只动新仓 extractor 目录）。
…d-aware ObjectName (#2905)
按 design §6.2.3 / §6.2.4 / §6.4 / §6.5 + §17.1.3 完成 Extractor 第二轮：
FUNCTION + PROCEDURE 抽取 + 重载支持 + 配套 map case 单测。
变更范围（限定 internal/extractor/ 5 文件）:
- internal/extractor/function.go（新增）：pg_get_functiondef + pg_get_function_arguments
合并 SQL，prokind='f'；extractFunction 返回 []*driverV2.DatabaseObjectDDL（slice），
每条独立 ObjectName="funcname(arg_type_list)"；对象不存在返回单条空 DDL 占位
（ObjectName 不带参数签名）。
- internal/extractor/procedure.go（新增）：同结构 prokind='p'，
ObjectType=PROCEDURE；与 function.go byte-for-byte 对称仅 prokind / ObjectType 差异。
- internal/extractor/extractor.go（修改）：switch 追加 case ObjectType_FUNCTION /
case ObjectType_PROCEDURE 两个分支，用 append(..., rs...) 展开聚合
（helper 返回 slice 以承载重载多行）；删除 TODO(Task-Dev-006) 注释；
TABLE / VIEW / default 分支保持不变。
- internal/extractor/function_test.go（新增）：4 个 test 函数 / 25 sub-case
覆盖 design §17.1.3 全部分支：
* TestExtractor_Extract_FunctionAndProcedure (5 cases) — Extract 入口分发 +
not-found 占位 + permission denied 透传；
* TestExtractor_FunctionOverload (4 cases) — 无参 / 单参 / 同名两重载 /
同名三重载（断言 slice 长度 + ObjectName 集合 + map key 唯一性）；
* TestExtractor_ProcedureOverload (4 cases) — 同上 PROCEDURE 版；
* TestExtractor_QuoteIdent_FunctionInjectionDefense (6 cases) +
TestExtractor_QuoteIdent_ProcedureInjectionDefense (6 cases) —
compat-RISK-4 SQL 注入兜底（空格 / 大小写 / 中文 / SQL 注入串 / 单双引号）。
- internal/extractor/extractor_test.go（修改）：从
TestExtractor_Extract_UnsupportedObjectType 中删除 FUNCTION_passthrough_to_Task006 /
PROCEDURE_passthrough_to_Task006 两条 Task-Dev-005 的过渡 sub-case
（FUNCTION/PROCEDURE 不再属 unsupported 集合）；同步文件头注释。
关键设计契约:
- byte-for-byte SQL 锁定：sqlmock.QueryMatcherEqual + raw const 双侧引用，
$1 / $2 参数化绑定，Go 侧零 escape（design §6.3）。
- ObjectName 携带参数签名：让上层 compareSchema 按 (ObjectName, ObjectType) 建 map
时同名重载天然唯一；DROP FUNCTION / PROCEDURE 时自然带参数签名（R-Q4-2）。
- 对象不存在兜底：返回单条 ObjectDDL=="" 的占位（ObjectName 不带参数签名），
不报错，不返回 nil（与 TABLE/VIEW 一致，让业务层识别"该侧不存在"）。
- 错误透传：catalog 权限错误 / SQLSTATE 原样返回，不吞诊断信息。
验证（docker golang:1.19.6 沙箱）:
- go vet ./internal/extractor/... ./internal/driver/... ./cmd/... PASS
- go test -v ./internal/extractor/... PASS（Task-Dev-005 29 + Task-Dev-006 25 =
54 sub-case）
- GOOS=linux GOARCH=amd64 go build ./cmd/sqle-gaussdb-plugin OK (37.86 MB)
- GOOS=linux GOARCH=amd64 go build ./cmd/sqle-opengauss-plugin OK (37.86 MB)
需求引用: 2.3 PROCEDURE 差异 / 2.4 FUNCTION 差异 / 4.2 R-Q4-2 DROP 带参数签名。
…f branches (#2905)
实现 sqle-gaussdb-plugin internal/differ 纯模块（Task-Dev-007），覆盖 design §7
最小实现矩阵：4 类对象（TABLE/VIEW/FUNCTION/PROCEDURE）× 3 种差异分类
（仅 base / 仅 compared / 两侧都有但不同）= 12 个分支字符串拼接逻辑。
实现要点
--------
- 主入口 GenerateModifySQLs(baseSchema, baseDDLs, comparedSchema, comparedDDLs)
返回 []string ModifySQLs；纯字符串拼接，无 db / ctx / 副作用。
- 内部按 (ObjectName, ObjectType) 元组建 map（Task-Dev-006 已让 FUNCTION/
PROCEDURE 的 ObjectName 携带参数签名 → 元组天然唯一，支持同名重载）。
- TABLE 走 DROP+CREATE 兜底（PG 系无 ALTER 幂等路径，design §7.1）；
VIEW/FUNCTION/PROCEDURE 直接复用 base 侧 CREATE OR REPLACE（PG 系幂等）。
- DROP TABLE 前置 WARNING 注释 byte-for-byte 锁定（design §7.3 + impact_analysis.yml
R-Q4-1 / risk_points 第 4 条「DROP TABLE 兜底可能误删生产数据」唯一兜底）。
- DROP FUNCTION/PROCEDURE 自带参数签名（R-Q4-2），避免 PG 报 function is not
unique；ObjectName 字面值由 extractor 上游 pg_get_function_arguments 拼装。
类型签名漂移说明
---------------
design §7 伪代码把 differ 入口写成 Generate(base, compared []*DatabaseSchemaObjectResult)，
但 real 链路（sqle/sqle/driver/v2/driver_interface.go:431-468）中
DatabaseSchemaObjectResult 是 per-schema 聚合容器；按"业务字段访问语义优先"
准则（semantic/design_pseudocode_param_type_drift_20260521）将 differ 入参定为
schema-pair 内的对象 DDL 列表 []*driverV2.DatabaseObjectDDL，driver 接入
（Task-Dev-008）时一次调用 extractor 取得 base/compared 两个 result 后解构传入即可。
单测覆盖
--------
internal/differ/differ_test.go: 12 个 Test 函数 / 26 个 sub-case 全部 map case
结构，纯字符串断言（无 sqlmock / 无 sql.Open / 无网络 / 无文件 IO）。覆盖
design §17.1.4 全部 8 个测试函数命名 + impact_analysis.yml line 312「4 类对象 ×
3 种差异共 12 个分支」全量覆盖：
- 8 个 design §17.1.4 必覆盖函数：
TestDiffer_TableOnlyBase / TableOnlyCompared / TableBothDiff
TestDiffer_ViewBothDiff（含 3 个 sub-case 覆盖 only_base/only_compared/both_diff）
TestDiffer_FunctionBothDiff（同上 3 sub-case）
TestDiffer_ProcedureBothDiff（同上 3 sub-case）
TestDiffer_FunctionDropWithArgs（R-Q4-2：两条重载独立 DROP，sort.Strings 顺序无关断言）
TestDiffer_WarningCommentInjection（R-Q4-1：byte-for-byte 字面值锁定 + 拒绝尾随空白/Go注释风格/!偏离）
- 4 个增强覆盖：
TestDiffer_TableBothSame_NoOutput（无变更不输出，3 sub-case 含 TABLE/VIEW/FUNCTION）
TestDiffer_MixedObjectTypes（4 类对象混合一次性输入，顺序无关集合断言）
TestDiffer_UnsupportedObjectTypeIgnored（INDEX/TRIGGER 等静默跳过不 panic）
TestDiffer_NilAndEmptyDefensive（nil 条目 / nil DatabaseObject 边界）
中文 schema 名场景单测兜底（critical_boundaries SQL 注入防御）：differ 直接复用
extractor 上游 $1/$2 参数化产出的字面值，不在本层做额外 escape。
map 迭代顺序无关断言：所有多元素期望用 sort.Strings(got) + sort.Strings(want)
模式（参考 semantic/sqlmock_rows_iteration_empty_and_order_20260521）。
验证
----
- go vet ./internal/differ/... ./internal/extractor/... ./internal/driver/... ./cmd/... PASS
- go test -count=1 ./internal/differ/... PASS（26 sub-case 全部通过）
- go test -count=1 ./internal/extractor/... PASS（无回归）
- go build ./cmd/sqle-gaussdb-plugin OK
- go build ./cmd/sqle-opengauss-plugin OK
- 8 个 sibling 仓库（sqle / sqle-ee / sqle-pg-plugin / sqle-tbase-plugin /
dms / dms-ee / dms-ui / dms-ui-ee）git status 全部 clean
不实现的场景（design §7.5 显式列出）
--------------------------------
分区 SPLIT/MERGE / 列存变更 / 分布键变更：按整表 DROP+CREATE 兜底；
INDEX / TRIGGER：本期不在抽取范围 → DDL 不出现 → diff 不出现。
driver 接入（替换 driver_gaussdb.go / driver_opengauss.go 中的 errNotImplemented）
留给 Task-Dev-008，本任务严格限定在 internal/differ/ 目录内。
Fixes #2905
补充 internal/differ/differ_test.go 单测文件（前一个 commit 误漏，本 commit 单独 补齐；语义上属于同一 Task-Dev-007，与 differ.go 共同满足 design §17.1.4 单测 矩阵 + impact_analysis.yml line 312 12 分支全量覆盖契约）。 覆盖 ---- - 12 个 Test 函数 / 26 个 sub-case，全部 map case 结构 + 顺序无关断言 （sort.Strings 模式 + map key 唯一性兜底）。 - design §17.1.4 line 873-881 全部 8 个测试函数命名 + 4 个增强覆盖 （TableBothSame_NoOutput / MixedObjectTypes / UnsupportedObjectTypeIgnored / NilAndEmptyDefensive）。 - impact_analysis.yml R-Q4-1（DROP TABLE WARNING 注释 byte-for-byte 锁定）+ R-Q4-2（DROP FUNCTION/PROCEDURE 带参数签名重载下两条独立 DROP）专项覆盖。 - 中文 schema 名场景（critical_boundaries SQL 注入防御）兜底。 约束 ---- - 纯字符串断言，无 sqlmock / 无 sql.Open / 无网络 / 无文件 IO； - 用本地 wantWarningComment 副本断言 WARNING 字面值（包内常量被无意修改时测 试同步失效）； - 多元素期望全部用 sort.Strings(got) + sort.Strings(want) 顺序无关比较， 避免 map 迭代顺序假设（参考 semantic/sqlmock_rows_iteration_empty_and_order_20260521）。 验证 ---- - go test -count=1 ./internal/differ/... PASS（26 sub-case 全通过） - go test -count=1 ./internal/extractor/... PASS（无回归） Fixes #2905
…xtractor + differ (#2905)
shared impl on *Driver via embedding; variants no longer override (DRY)
Task-Dev-008 streams the previously placeholder GetDatabaseObjectDDL /
GetDatabaseDiffModifySQL through real wiring:
- internal/driver/driver_common.go now hosts both methods on *Driver. The
methods delegate to extractor.NewExtractor(d.DriverImpl.DB).Extract for
DDL extraction; GetDatabaseDiffModifySQL additionally calls
differ.GenerateModifySQLs after two Extract invocations (one per
schema side) to produce the ModifySQLs slice. SchemaName on the
returned DatabaseDiffModifySQLResult follows design §7 by surfacing
info.BaseSchemaName; compared schema names are already encoded into
DROP statements inside differ output. calibratedDSN is retained on
the signature (interface contract) and explicitly discarded for
future cross-instance compare scenarios.
- internal/driver/driver_gaussdb.go and internal/driver/driver_opengauss.go
drop their per-variant errNotImplemented overrides; embedding makes
both *GaussDBDriver and *OpenGaussDriver inherit the shared
implementation. errNotImplemented sentinel is removed from
driver_common.go since no caller relies on it anymore.
- internal/driver/driver_test.go updates the two enabled-this-period
sub-cases in TestDriver_CapabilityMatrix from "expect not-implemented
sentinel" to "nil objInfos -> empty slice + nil err" probes, swaps
the errNotImplemented reference in TestWrapPGError for a local
sentinel, and adds two new map-case Test functions
(TestDriver_GetDatabaseObjectDDL_RealChain and
TestDriver_GetDatabaseDiffModifySQL_RealChain), 3 sub-cases each
(6 new sub-cases total). The diff sub-case exercises end-to-end
DROP+CREATE wiring with the WARNING comment from design §7.3 / R-Q4-1.
Verification (docker golang:1.19.6 sandbox):
go vet ./... PASS
go test ./internal/driver PASS (8 funcs, 42 sub-cases including new 6)
go test ./internal/extractor PASS (no regression; 55 sub-cases)
go test ./internal/differ PASS (no regression; 26 sub-cases)
go build ./cmd/sqle-gaussdb-plugin OK
go build ./cmd/sqle-opengauss-plugin OK
Scope is limited to internal/driver/ (4 files, +461/-72 lines); 8 sibling
repos remain git-clean; no go.mod/go.sum churn.
…t-Fix-001) sqle-ee execute_comparison main path passes baseInfos[i] with only SchemaName set and DatabaseObjects=nil, expecting plugin to enumerate the schema. Initial implementation iterated info.DatabaseObjects only, returning empty DatabaseObjectDDLs and causing inconsistent_num=0 in the UI for every schema pair (docs/test/case-2-1.md ~ case-2-4.md). Add a per-Extract.entry fallback: when info.DatabaseObjects is empty, list pg_class (relkind 'r'/'v') and pg_proc (prokind 'f'/'p') in the target schema, GROUP BY proname for overload-aware dedup, then feed the resulting []*DatabaseObject into the existing switch dispatch so TABLE/VIEW/FUNCTION/PROCEDURE share their original extract helpers. SQL templates pin schema via $1 binding (design §6.3 no-Go-escape contract). Unit tests cover full-enumeration end-to-end, empty schema (0 objects), and propagated permission error.
…sk-Test-Fix-001)
GaussDB kernel 505.2.1 + openGauss official Go driver returned the
combined SELECT pg_get_functiondef(oid), pg_get_function_arguments(oid)
result as a Postgres composite/record literal, e.g.
(4,"CREATE OR REPLACE FUNCTION ..."), which Scan() into two strings
cannot strip. Symptom: comparison_statements / modify_sql_statements
API output for FUNCTION and PROCEDURE wrapped the SQL text in
(N,"..."), making it impossible to copy-paste into psql
(docs/test/case-2-3.md / case-2-4.md).
Split each helper into two single-column queries (defs and args),
both keyed by (nspname, prokind, proname) and ordered by p.oid so
the i-th row of each query refers to the same overload. min(len)
alignment defends against rare concurrent overload mutation.
Unit tests rewritten to mock the two-query pattern via two helpers
(expectFuncDefsAndArgs / expectProcDefsAndArgs); two new regression
tests assert ObjectDDL does not start with '(' nor contain ',"' so
future driver regressions surface as test failures.
…t-Fix-001) VIEW / FUNCTION / PROCEDURE CREATE branches previously emitted the base side ObjectDDL verbatim, leaving qualifiers like `CREATE OR REPLACE VIEW test_2905_base.v_user_summary` in the generated modify SQL. Executing such SQL against the compared schema would recreate the object in base namespace instead of converging the compared side to the base form (docs/test/case-2-2.md / case-2-3.md, design §7.2). Extend genOnlyBaseSide / genBothDiff signatures with baseSchema and add a rewriteSchemaQualifier helper that does a strings.ReplaceAll on the `<baseSchema>.` prefix. TABLE branch keeps current behaviour (pg_get_tabledef relies on search_path). Self-compare and empty baseSchema short-circuit to no-op. Also surfaces the existing same-DDL short-circuit (`baseDDL.ObjectDDL == comparedDDL.ObjectDDL`) which doubles as P1.3 overload-redundant CREATE filter once P1.1 stops wrapping pg_get_functiondef output in record literals; new TestGenBothDiff_SameDDLOverload_NoOutput pins that behaviour.
…d return (P1.1 amend) Initial P1.1 fix split the double-column query into two single-column queries, assuming the tuple wrapper `(N, "DDL")` was caused by the driver packing multi-column rows into a record. After redeploy the regression still produced `(4,"CREATE OR REPLACE FUNCTION ...")`. Root cause confirmed by hitting the database directly with the openGauss driver: GaussDB ships a `pg_get_functiondef(oid)` overload that returns a composite record `(headerlines int, definition text)` instead of upstream PostgreSQL's plain text. The driver serialises the record-as-string and Scan() into one string keeps the wrapper. Switch the SELECT projection to `(pg_get_functiondef(p.oid)).definition` so the server-side strips the record layer and the driver receives plain text. The split into def/args two queries is kept since it aligns with overload ordering (ORDER BY p.oid) and shields against future record-returning quirks in pg_get_function_arguments.
…DDL (Fix-002 P3) openGauss / GaussDB `pg_get_functiondef(oid)` returns PROCEDURE DDL with a trailing standalone `/` line (PL/SQL terminator, gsql / SQL*Plus style). The Go database/sql + openGauss-connector-go-pq driver — same driver used by sqle-ee workflow exec — does not recognise `/` as a statement terminator and reports `pq: syntax error at or near "/"` when the modify_sql_statements output is fed back into the driver directly. This commit adds extractor/procedure.go::stripPLSQLTerminator which trims the trailing whitespace and the standalone `/` line (with a multi-line anchored regex `\n\s*/\s*\z`). FUNCTION path (prokind='f') is untouched — its output ends with `$function$;` and does not carry the terminator. Mid-body `/` (theoretically impossible in a PL/SQL block) is preserved by anchoring the regex to the very end of the string. Test coverage: - TestExtractProcedure_TrimTrailingSlash with 4 sub-cases: slash_with_trailing_newline / slash_no_trailing_newline / slash_with_extra_blank_line_before / no_slash_unchanged. Defect source: - docs/test/case-3-4.md (initial reproduction in openGauss PROCEDURE main path) - expertise_docs/semantic/opengauss_procedure_slash_terminator_driver_incompat_20260521.md (cross-channel background: gsql OK, JDBC / psycopg2 / Go all fail)
…a-replace short-circuit (Fix-002 P2-A/P4/P5)
Three closely-related differ fixes consolidated in one commit because
they share the same string-compare short-circuit code path inside
GenerateModifySQLs.
P2-A: TABLE schema rewrite gap (case-3-1.md / case-2-1.md)
Task-Test-Fix-001 P1.2 introduced rewriteSchemaQualifier to rewrite
base→compared schema qualifier for VIEW / FUNCTION / PROCEDURE, but
TABLE was missed. pg_get_tabledef emits the TABLE DDL with a
`SET search_path = baseSchema;` header line followed by a bare-name
CREATE TABLE — so when the variant SQL runs against the compared
instance, CREATE TABLE lands inside the BASE schema and collides
with the existing same-name table (`relation "t_order" already
exists in schema "test_2905_base"`). Fix:
- new helper rewriteSetSearchPath(ddl, baseSchema, comparedSchema)
using `(?m)^SET\s+search_path\s*=\s*<base>(\s*;?)` to anchor the
rewrite to the first SET line, with explicit literal match on
baseSchema to avoid mis-firing on unrelated SET lines;
- genOnlyBaseSide TABLE branch and genBothDiff TABLE branch both
invoke rewriteSetSearchPath on baseDDL.ObjectDDL before joining.
- design §7.2 red line preserved: only base→compared, never reverse.
P4: USTORE/ustore case-only normalisation (case-2-1.md remnant)
GaussDB pg_get_tabledef outputs storage_type=USTORE on initial
CREATE TABLE but storage_type=ustore on rebuild; the case-only
difference triggers a false-positive "inconsistent" diff. Fix:
- new helper normalizeDDLForCompare(s) running
`(?i)storage_type\s*=\s*ustore` → "storage_type=ustore" on a copy
of the DDL strictly for the compare-side string compare; the
actual modify SQL output is not mutated.
P5: schema-replace short-circuit before compare (case-2-3.md remnant)
Overload functions whose body is identical but the SET / CREATE OR
REPLACE schema qualifier differs (base.fn vs compared.fn) would
still emit a redundant CREATE OR REPLACE. Fix:
- GenerateModifySQLs now runs
`rewriteSetSearchPath(rewriteSchemaQualifier(baseDDL, base, compared), base, compared)`
+ normalizeDDLForCompare on both sides before the equality check.
When normalised equal, the diff is short-circuited.
Test coverage (5 new test functions, 12 sub-cases):
- TestGenOnlyBaseSide_RewriteTableSearchPath
- TestGenBothDiff_RewriteTableSearchPath
- TestGenBothDiff_USTORENormalize_NoOutput
- TestGenBothDiff_SameDDLDifferentSchema_NoOutput
- TestRewriteSetSearchPath_GuardRails (5 sub-cases: empty base /
same schema / normal rewrite with semicolon / without semicolon /
non-matching base literal)
Defect source:
- docs/test/case-3-1.md (P2-A primary reproduction)
- docs/test/case-2-1.md (P2-A latent in GaussDB + P4 remnant)
- docs/test/case-2-3.md (P5 remnant — overload same body)
- expertise_docs/episodic/sqle_gaussdb_table_search_path_not_rewritten_20260521.md
… modify-SQL drawer (Fix-002 P2-B)
UI main path (dms-ui-ee EnvironmentComparison) ships the tree-node
display text (with parameter signature, e.g. `fn_calc(p integer)` /
`proc_alert(p TEXT)`) as ObjectName to plugin
GetDatabaseDiffModifySQL / GetDatabaseObjectDDL gRPC. Plugin's
internal extractor uses ObjectName as the `pg_proc.proname = $2`
bind parameter; pg_proc.proname is always a bare name (PG system
catalog convention), so the query matches zero rows and falls into
the "object not found → placeholder ObjectDDL=''" branch. Result:
differ sees a pair of empty placeholders, emits modify_sqls=[], the
UI drawer shows "暂无数据" for FUNCTION / PROCEDURE main path.
API bypass with bare ObjectName=`fn_calc` already worked (case-3-3.md
acceptance line 67-68), confirming differ + extractor + signature
matching are all sound; the gap is the ObjectName entry contract.
Fix strategy (option D from todo.md candidate set — equivalent to
recommended option C "fullName + bareName double-key" but with a
simpler implementation at the plugin entry):
- new helper stripObjectNameSignature(s) using strings.IndexByte('(')
+ strings.TrimRight to peel the signature off; bare-name input
passes through unchanged;
- new helper normalizeObjectsForExtractor that maps a slice of
*DatabaseObject, copying entries whose ObjectName needs stripping
(the original proto messages are NOT mutated; TABLE/VIEW pointers
are reused as-is since their ObjectName never contains '(');
- GetDatabaseDiffModifySQL and GetDatabaseObjectDDL both run the
normaliser on info.DatabaseObjects before constructing the
extractor input; SchemaName / ComparedSchemaName / response
structure are untouched.
The plugin extractor's pg_get_function_arguments-based ObjectName
rebuild (function.go / procedure.go) means BOTH base and compared
sides return the canonical `fn_calc(integer)` form after lookup —
the differ map (ObjectName, ObjectType) keys align naturally without
any double-key injection. R-Q4-2 overload semantics preserved
end-to-end.
Test coverage (3 new test functions):
- TestStripObjectNameSignature: 8 sub-cases covering parameter name
+ type / type-only / multi-args / no-args / bare name / empty /
PROCEDURE TEXT-typed / space-before-paren.
- TestNormalizeObjectsForExtractor: empty input short-circuit /
function stripped without mutating original / TABLE pointer
reuse / nil element preservation.
- TestDriver_GetDatabaseDiffModifySQL_StripsObjectNameSignature:
end-to-end sqlmock — passes `fn_calc(p integer)` to plugin,
asserts extractor receives bare `fn_calc` (via ExpectQuery
WithArgs) and the resulting modify SQL contains
`CREATE OR REPLACE FUNCTION compared.fn_calc` with `(p integer)`
signature preserved.
Defect source:
- docs/test/case-3-3.md (FUNCTION main path "暂无数据")
- docs/test/case-3-4.md (PROCEDURE main path "暂无数据" — same root)
- expertise_docs/episodic/sqle_gaussdb_modify_sql_object_name_signature_mismatch_20260521.md
- expertise_docs/semantic/sqle_plugin_overload_objectname_signature_20260521.md
(overload identification contract R-Q4-2)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概述
为 SQLE 增加 sqle-gaussdb-plugin（新插件仓库），实现 GaussDB / openGauss 结构对比功能（TABLE / VIEW / FUNCTION / PROCEDURE 四类对象 DDL 抽取 + 同类型结构差异 SQL 生成）。
Fixes actiontech/sqle-ee#2905
仓库背景
本仓库为新建插件仓库。本 PR 分支与现有
dev-2892-gaussdb-slowlog分支并行，承担"GaussDB / openGauss 结构对比"（2905）独立产品线。两条分支无共同祖先，分别建立独立 main / dev-2892 基线分支。PR base 为新建的
main分支（指向本期初始化 commita3535af），head 为 dev 分支 headfe9519a，包含本期全部 18 个增量 commit。主要变更（base 后 18 commits）
仓库骨架与依赖锁定
a3535afinit skeleton with dual binary registration（base commit）12ea2dadeps: lock indirect & add pingcap/parser sjjian fork replace for local builde030dd9docs: README 增补沙箱本地构建命令 + go.mod replace 移除提示2cfe5a3deps: promote github.com/pkg/errors to direct requireDialector 模块
150721dfeat: implement Dialector with DSN builder for GaussDB / openGaussDriver 模块（双独立二进制）
f66cd20deps: promote DATA-DOG/go-sqlmock to direct requirebebd66bfeat(driver): split stub into driver_common / driver_gaussdb / driver_opengauss with sqlmock unit testscmd/sqle-gaussdb-plugin与cmd/sqle-opengauss-pluginExtractor 模块（DDL 抽取）
665f031feat(extractor): implement Extract entry + TABLE / VIEW DDL extraction with sqlmock unit testsb97484efeat(extractor): implement FUNCTION / PROCEDURE extraction with overload-aware ObjectName99a69acfix(extractor): empty DatabaseObjects → enumerate schema (P0)ce460ccfix(extractor): split pg_get_functiondef double-column query (P1.1)d2f3d9efix(extractor): use (pg_get_functiondef).definition for GaussDB record return (P1.1 amend)d183e39fix(extractor/procedure): strip PL/SQL trailing slash from PROCEDURE DDL (Fix-002 P3)b97484e重载处理：通过pg_get_function_arguments(oid)携带参数签名解决同名重载唯一性Differ 模块（变更 SQL 生成）
ae3ac2dfeat(differ): implement GenerateModifySQLs for 4 object types x 3 diff branches362f892test(differ): add map case unit tests for GenerateModifySQLs9096b95feat(driver): wire GetDatabaseObjectDDL / GetDatabaseDiffModifySQL to extractor + differe320194fix(differ): rewrite base schema qualifier to compared (P1.2)88b501afix(differ): TABLE SET search_path rewrite + USTORE normalize + schema-replace short-circuit (Fix-002 P2-A/P4/P5)fe9519afix(driver): strip ObjectName signature at plugin entry to recover UI modify-SQL drawer (Fix-002 P2-B)DROP TABLE 强制带 WARNING 注释行（compat-RISK-5 兜底）；FUNCTION / PROCEDURE DROP 带参数签名（R-Q4-2）。
inspector 复用
测试
go vet ./...通过go test -count=1 ./...全部通过：internal/dialectorOKinternal/differOKinternal/driverOKinternal/extractorOK影响面
关联 PR