-
Notifications
You must be signed in to change notification settings - Fork 1
feat: PostgreSQL COPY IN/OUT protocol #128
Copy link
Copy link
Open
Labels
area/driverDatabase/cache driver infrastructureDatabase/cache driver infrastructuredriver/postgresPostgreSQL driverPostgreSQL driver
Milestone
Description
Summary
Implement PostgreSQL COPY protocol for high-throughput bulk data transfer. COPY IN streams data to the server, COPY OUT streams data from the server.
Design
Package: driver/postgres/protocol/copy.go
COPY IN Flow (client → server bulk insert)
Client → Server: Query { "COPY users FROM STDIN" }
Server → Client: CopyInResponse { format, columns, columnFormats }
Client → Server: CopyData { row bytes }
Client → Server: CopyData { row bytes }
...
Client → Server: CopyDone
Server → Client: CommandComplete { "COPY 1000" }
Server → Client: ReadyForQuery
Error during COPY IN:
Client → Server: CopyFail { "error message" }
Server → Client: ErrorResponse { ... }
Server → Client: ReadyForQuery
COPY OUT Flow (server → client bulk export)
Client → Server: Query { "COPY users TO STDOUT" }
Server → Client: CopyOutResponse { format, columns, columnFormats }
Server → Client: CopyData { row bytes }
Server → Client: CopyData { row bytes }
...
Server → Client: CopyDone
Server → Client: CommandComplete { "COPY 1000" }
Server → Client: ReadyForQuery
API
// CopyIn streams rows to the server
type CopyIn struct {
state copyPhase
format int8 // 0=text, 1=binary
columns int16
w *Writer
}
func (c *CopyIn) WriteRow(data []byte) error // send CopyData
func (c *CopyIn) Close() error // send CopyDone
func (c *CopyIn) Abort(reason string) error // send CopyFail
// CopyOut receives rows from the server
type CopyOut struct {
state copyPhase
format int8
columns int16
rowCh chan []byte // event loop sends rows here
}
func (c *CopyOut) ReadRow() ([]byte, error) // blocks for next CopyData
func (c *CopyOut) Close() error // waits for CopyDoneBinary COPY Format
For binary format COPY:
- Header:
PGCOPY\n\377\r\n\0+ flags (4 bytes) + header extension (4 bytes) - Each tuple: field count (int16) + for each field: length (int32) + data
- Trailer:
-1(int16)
Acceptance Criteria
- CopyInResponse / CopyOutResponse parsing
- CopyData send/receive
- CopyDone / CopyFail handling
- Text format COPY IN (tab-delimited rows)
- Binary format COPY IN/OUT with proper header/trailer
- Streaming interface (doesn't buffer all data in memory)
- Error handling: CopyFail on client error, ErrorResponse from server
- Unit tests with captured COPY exchanges
Dependencies
- Depends on 125 (simple query — COPY starts via Query message)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
area/driverDatabase/cache driver infrastructureDatabase/cache driver infrastructuredriver/postgresPostgreSQL driverPostgreSQL driver