Skip to content

feat: PostgreSQL COPY IN/OUT protocol #128

@FumingPower3925

Description

@FumingPower3925

Summary

Implement PostgreSQL COPY protocol for high-throughput bulk data transfer. COPY IN streams data to the server, COPY OUT streams data from the server.

Design

Package: driver/postgres/protocol/copy.go

COPY IN Flow (client → server bulk insert)

Client → Server:  Query { "COPY users FROM STDIN" }
Server → Client:  CopyInResponse { format, columns, columnFormats }
Client → Server:  CopyData { row bytes }
Client → Server:  CopyData { row bytes }
...
Client → Server:  CopyDone
Server → Client:  CommandComplete { "COPY 1000" }
Server → Client:  ReadyForQuery

Error during COPY IN:

Client → Server:  CopyFail { "error message" }
Server → Client:  ErrorResponse { ... }
Server → Client:  ReadyForQuery

COPY OUT Flow (server → client bulk export)

Client → Server:  Query { "COPY users TO STDOUT" }
Server → Client:  CopyOutResponse { format, columns, columnFormats }
Server → Client:  CopyData { row bytes }
Server → Client:  CopyData { row bytes }
...
Server → Client:  CopyDone
Server → Client:  CommandComplete { "COPY 1000" }
Server → Client:  ReadyForQuery

API

// CopyIn streams rows to the server
type CopyIn struct {
    state   copyPhase
    format  int8        // 0=text, 1=binary
    columns int16
    w       *Writer
}

func (c *CopyIn) WriteRow(data []byte) error   // send CopyData
func (c *CopyIn) Close() error                  // send CopyDone
func (c *CopyIn) Abort(reason string) error     // send CopyFail

// CopyOut receives rows from the server
type CopyOut struct {
    state   copyPhase
    format  int8
    columns int16
    rowCh   chan []byte  // event loop sends rows here
}

func (c *CopyOut) ReadRow() ([]byte, error)     // blocks for next CopyData
func (c *CopyOut) Close() error                  // waits for CopyDone

Binary COPY Format

For binary format COPY:

  • Header: PGCOPY\n\377\r\n\0 + flags (4 bytes) + header extension (4 bytes)
  • Each tuple: field count (int16) + for each field: length (int32) + data
  • Trailer: -1 (int16)

Acceptance Criteria

  • CopyInResponse / CopyOutResponse parsing
  • CopyData send/receive
  • CopyDone / CopyFail handling
  • Text format COPY IN (tab-delimited rows)
  • Binary format COPY IN/OUT with proper header/trailer
  • Streaming interface (doesn't buffer all data in memory)
  • Error handling: CopyFail on client error, ErrorResponse from server
  • Unit tests with captured COPY exchanges

Dependencies

  • Depends on 125 (simple query — COPY starts via Query message)

Metadata

Metadata

Labels

area/driverDatabase/cache driver infrastructuredriver/postgresPostgreSQL driver

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions