Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 55 additions & 1 deletion .github/workflows/build-js-bindings.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Build, pack and upload seekdb JS bindings for multiple platforms to S3
#
# Platforms: linux-x64, linux-arm64, darwin-arm64 (macOS x64 not supported)
# Platforms: linux-x64, linux-arm64, darwin-arm64, win32-x64 (macOS x64 not supported)
# S3: s3://oceanbase-seekdb-builds/js-bindings/all_commits/<sha>/seekdb-js-bindings-<platform>.zip
#
name: Build JS bindings
Expand Down Expand Up @@ -138,13 +138,67 @@ jobs:
name: seekdb-js-bindings-${{ matrix.platform }}
path: seekdb-js-bindings-${{ matrix.platform }}.zip

# ---------- Build JS bindings on Windows ----------
build-windows:
name: Build JS bindings (${{ matrix.platform }})
runs-on: ${{ matrix.runner }}
strategy:
fail-fast: false
matrix:
include:
- platform: win32-x64
runner: windows-2022

steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}

- name: Install pnpm
uses: pnpm/action-setup@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 20
cache: "pnpm"

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install dependencies
run: pnpm install --frozen-lockfile

- name: Build bindings (Windows)
working-directory: packages/bindings
run: pnpm run build

- name: Pack artifact (Windows)
working-directory: packages/bindings/pkgs/js-bindings
shell: pwsh
run: |
$root = (Resolve-Path ../../../..).Path
$zip = Join-Path $root "seekdb-js-bindings-${{ matrix.platform }}.zip"
if (Test-Path $zip) { Remove-Item $zip -Force }
Compress-Archive -Path * -DestinationPath $zip -Force

- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: seekdb-js-bindings-${{ matrix.platform }}
path: seekdb-js-bindings-${{ matrix.platform }}.zip

# ---------- Collect artifacts and upload to S3 ----------
release-artifacts:
name: Collect artifacts and upload to S3
runs-on: ubuntu-22.04
needs:
- build-linux
- build-macos
- build-windows
steps:
- name: Download all artifacts
uses: actions/download-artifact@v4
Expand Down
13 changes: 7 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -97,14 +97,13 @@ jobs:
docker logs seekdb-server

- name: Run server tests
working-directory: packages/seekdb
env:
SEEKDB_HOST: 127.0.0.1
SEEKDB_PORT: 2881
SEEKDB_USER: root
SEEKDB_PASSWORD: ""
SEEKDB_DATABASE: test
run: pnpm exec vitest run --exclude 'tests/embedded/**'
run: pnpm run test:server

# Embedded-mode tests on multiple platforms (requires native bindings build per OS; Docker per job)
test-embedded:
Expand All @@ -121,6 +120,8 @@ jobs:
runner: ubuntu-22.04-arm
- platform: darwin-arm64
runner: macos-14
- platform: windows-x64
runner: windows-latest
steps:
- name: Checkout
uses: actions/checkout@v4
Expand Down Expand Up @@ -149,7 +150,7 @@ jobs:
- name: Build packages
run: pnpm run build

# macOS has no Docker; skip container and exclude mode-consistency (needs seekdb server)
# Docker seekdb server is Linux-only; macOS/Windows skip container and exclude mode-consistency (needs server)
- name: Start seekdb container
if: runner.os == 'Linux'
shell: bash
Expand All @@ -159,7 +160,7 @@ jobs:
docker logs seekdb-server

- name: Run embedded tests
working-directory: packages/seekdb
shell: bash
env:
SEEKDB_HOST: 127.0.0.1
SEEKDB_PORT: 2881
Expand All @@ -168,9 +169,9 @@ jobs:
SEEKDB_DATABASE: test
run: |
if [ "$RUNNER_OS" = "Linux" ]; then
pnpm exec vitest run tests/embedded/ 2>&1 | tee /tmp/vitest.log
pnpm run test:embedded 2>&1 | tee /tmp/vitest.log
else
pnpm exec vitest run tests/embedded/ --exclude '**/mode-consistency.test.ts' 2>&1 | tee /tmp/vitest.log
pnpm run test:embedded -- --exclude '**/mode-consistency.test.ts' 2>&1 | tee /tmp/vitest.log
fi
exit_code=$?
if [ $exit_code -ne 0 ]; then exit $exit_code; fi
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,4 @@ coverage/
examples/seekdb-prisma/generated/**
spec/
.vscode
.codegraph
14 changes: 8 additions & 6 deletions DEVELOP.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,22 +96,24 @@ pnpm build
The project uses Vitest. From the project root:

```bash
# Run all tests (seekdb + @seekdb/prisma-adapter; runs once and exits)
# Full suite: seekdb server tests + embedded tests + @seekdb/prisma-adapter (each runs once)
pnpm test

# Run only seekdb package tests
pnpm --filter seekdb run test
# seekdb only — server-mode tests (paths outside tests/embedded/; need seekdb/OceanBase on SEEKDB_*)
pnpm run test:server

# seekdb only — embedded-mode tests (under tests/embedded/; native addon required)
pnpm run test:embedded

# Watch mode for seekdb (during development)
pnpm --filter seekdb exec vitest

# Run only @seekdb/prisma-adapter tests
pnpm --filter @seekdb/prisma-adapter run test

# Run only embedded-mode tests (no server required)
pnpm --filter seekdb exec vitest run tests/embedded/
```

From `packages/seekdb`, the same split is available as `pnpm run test:server` and `pnpm run test:embedded`.

**Tests and running mode**:

- **Embedded-mode tests** live under `packages/seekdb/tests/embedded/` and use a temporary database path per test file. They do not require a seekdb/OceanBase server.
Expand Down
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ For complete usage, see the official documentation.
[Packages](#packages)<br/>
[Installation](#installation)<br/>
[Running Modes](#running-modes)<br/>
[Embedded mode platform support](#embedded-mode-platform-support)<br/>
[Quick Start](#quick-start)<br/>
[Usage Guide](#usage-guide)<br/>
[Examples](#examples)<br/>
Expand Down Expand Up @@ -69,6 +70,19 @@ The SDK supports two modes; the constructor arguments to `SeekdbClient` determin
- **SeekdbClient**: Pass `path` for embedded mode, or `host` (and port, user, password, etc.) for server mode.
- **SeekdbAdminClient()**: For admin operations only; pass `path` for embedded or `host` for server. In embedded mode you do not specify a database name.

### Embedded mode platform support

Embedded mode uses the native package **`@seekdb/js-bindings`**. Prebuilt binaries are supported on:

| Platform key | OS / CPU |
| -------------- | --------------------------- |
| `linux-x64` | Linux x86_64 |
| `linux-arm64` | Linux ARM64 |
| `darwin-arm64` | macOS Apple Silicon (ARM64) |
| `win32-x64` | Windows x86_64 |

Optional environment variables for the loader: `SEEKDB_BINDINGS_BASE_URL` (artifact directory URL), `SEEKDB_BINDINGS_CACHE_DIR` (download cache). See [`packages/bindings/README.md`](./packages/bindings/README.md).

## Quick Start

**Embedded mode** (local file, no server):
Expand Down
5 changes: 4 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@
"build": "pnpm --filter seekdb run build && pnpm --filter '@seekdb/*' run build",
"build:seekdb": "pnpm --filter seekdb run build",
"build:embeddings": "pnpm --filter '@seekdb/*' run build",
"test": "pnpm --filter seekdb run test && pnpm --filter @seekdb/prisma-adapter run test",
"bindings:clean": "pnpm --filter @seekdb/js-bindings-build run clean",
"test": "pnpm run test:server && pnpm run test:embedded && pnpm --filter @seekdb/prisma-adapter run test",
"test:server": "pnpm --filter seekdb run test:server",
"test:embedded": "pnpm --filter seekdb run test:embedded",
"lint": "pnpm -r run lint",
"type-check": "pnpm -r run type-check",
"prettier": "prettier --write .",
Expand Down
11 changes: 6 additions & 5 deletions packages/bindings/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The native addon is structured in three layers:

2. **JavaScript Wrapper** (`pkgs/js-bindings/seekdb.js`)
- Loads native `.node` from same dir (npm package / local build) or on-demand download (Node fetch + adm-zip)
- Supports Linux (x64/arm64) and macOS (arm64 only). **Native bindings are not on npm**; built by CI and hosted on S3.
- Supports Linux (x64/arm64), macOS (arm64), and Windows (x64). **Native bindings are not on npm**; built by CI and hosted on S3.

3. **TypeScript API Layer** (`../seekdb/src/client-embedded.ts`)
- High-level TypeScript API
Expand All @@ -22,7 +22,7 @@ The native addon is structured in three layers:

## Distribution (S3, not npm)

Native bindings are **not** published to npm. They are built by [`.github/workflows/build-js-bindings.yml`](../../.github/workflows/build-js-bindings.yml) and uploaded to S3. Each set of artifacts lives in a directory that contains `seekdb-js-bindings-<platform>.zip` for each platform (e.g. linux-x64, linux-arm64, darwin-arm64).
Native bindings are **not** published to npm. They are built by [`.github/workflows/build-js-bindings.yml`](../../.github/workflows/build-js-bindings.yml) and uploaded to S3. Each set of artifacts lives in a directory that contains `seekdb-js-bindings-<platform>.zip` for each platform (e.g. linux-x64, linux-arm64, darwin-arm64, win32-x64).

**Usage**: When embedded mode is first used, the loader uses same-dir `seekdb.node` (npm package or local build) or downloads bindings on demand. Optional env:

Expand All @@ -45,7 +45,7 @@ This will:

1. Fetch the libseekdb library for your platform (Python scripts invoked by `binding.gyp`)
2. If the archive contains a `libs/` directory, copy it to `pkgs/js-bindings/libs/` (e.g. macOS runtime deps)
3. Compile the C++ bindings with node-gyp and copy `seekdb.node` and `libseekdb.so`/`libseekdb.dylib` into `pkgs/js-bindings/`
3. Compile the C++ bindings with node-gyp and copy `seekdb.node` and `libseekdb.so` / `libseekdb.dylib` / `seekdb.dll` (as appropriate for the platform) into `pkgs/js-bindings/`

## Platform Support

Expand All @@ -54,12 +54,13 @@ The bindings support the following platforms:
- Linux x64
- Linux arm64
- macOS arm64 (Apple Silicon)
- Windows x64

Note: macOS x64 and Windows are not currently supported.
Note: macOS x64 (Intel) is not currently supported in CI; local builds may still be possible.

## C API Integration

The bindings use the seekdb C API (see `seekdb.h` in `libseekdb/` after fetch) and link against `libseekdb.so` / `libseekdb.dylib`. The native library is downloaded and extracted by platform-specific Python scripts in `scripts/` (invoked from `binding.gyp`); see `scripts/README.md` for details.
The bindings use the seekdb C API (see `seekdb.h` in `libseekdb/` after fetch) and link against `libseekdb.so`, `libseekdb.dylib`, or `seekdb.dll` / import lib on Windows. The native library is downloaded and extracted by platform-specific Python scripts in `scripts/` (invoked from `binding.gyp`); see `scripts/README.md` for details.

### Current Implementation

Expand Down
43 changes: 37 additions & 6 deletions packages/bindings/binding.gyp
Original file line number Diff line number Diff line change
Expand Up @@ -7,30 +7,39 @@
['OS=="linux" and target_arch=="x64"', {
'actions': [{
'action_name': 'run_fetch_libseekdb_script',
'message': 'Fetching and extracting libseekdb',
'inputs': [],
'message': 'Ensuring libseekdb is present',
'inputs': ['<(module_root_dir)/libseekdb/libseekdb.so'],
'action': ['sh', '-c', 'cd "<(module_root_dir)" && python3 scripts/fetch_libseekdb_linux_x64.py'],
'outputs': ['<(module_root_dir)/libseekdb/libseekdb.so'],
}],
}],
['OS=="linux" and target_arch=="arm64"', {
'actions': [{
'action_name': 'run_fetch_libseekdb_script',
'message': 'Fetching and extracting libseekdb',
'inputs': [],
'message': 'Ensuring libseekdb is present',
'inputs': ['<(module_root_dir)/libseekdb/libseekdb.so'],
'action': ['sh', '-c', 'cd "<(module_root_dir)" && python3 scripts/fetch_libseekdb_linux_arm64.py'],
'outputs': ['<(module_root_dir)/libseekdb/libseekdb.so'],
}],
}],
['OS=="mac" and target_arch=="arm64"', {
'actions': [{
'action_name': 'run_fetch_libseekdb_script',
'message': 'Fetching and extracting libseekdb',
'inputs': [],
'message': 'Ensuring libseekdb is present',
'inputs': ['<(module_root_dir)/libseekdb/libseekdb.dylib'],
'action': ['sh', '-c', 'cd "<(module_root_dir)" && python3 scripts/fetch_libseekdb_darwin_arm64.py'],
'outputs': ['<(module_root_dir)/libseekdb/libseekdb.dylib'],
}],
}],
['OS=="win" and target_arch=="x64"', {
'actions': [{
'action_name': 'run_fetch_libseekdb_script',
'message': 'Ensuring libseekdb is present',
'inputs': ['<(module_root_dir)/libseekdb/seekdb.dll'],
'action': ['python3', '<(module_root_dir)/scripts/fetch_libseekdb_windows_x64.py'],
'outputs': ['<(module_root_dir)/libseekdb/seekdb.dll'],
}],
}],
],
},
{
Expand Down Expand Up @@ -91,6 +100,20 @@
},
],
}],
['OS=="win" and target_arch=="x64"', {
'win_delay_load_hook': 'true',
'link_settings': {
'libraries': [
'<(module_root_dir)/libseekdb/seekdb.lib',
],
},
'copies': [
{
'files': ['<(module_root_dir)/libseekdb/seekdb.dll'],
'destination': '<(module_root_dir)/pkgs/js-bindings',
},
],
}],
],
},
{
Expand Down Expand Up @@ -122,6 +145,14 @@
},
],
}],
['OS=="win" and target_arch=="x64"', {
'copies': [
{
'files': ['<(module_root_dir)/build/Release/seekdb.node'],
'destination': '<(module_root_dir)/pkgs/js-bindings',
},
],
}],
],
},
],
Expand Down
2 changes: 1 addition & 1 deletion packages/bindings/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"clean": "npm run clean:gyp && npm run clean:libseekdb && npm run clean:package",
"clean:gyp": "node-gyp clean",
"clean:libseekdb": "rimraf libseekdb",
"clean:package": "rimraf pkgs/js-bindings*/**/*.node pkgs/js-bindings*/**/*.so pkgs/js-bindings*/**/*.dylib pkgs/js-bindings*/**/*.dll",
"clean:package": "rimraf pkgs/js-bindings*/libs pkgs/js-bindings*/*.node pkgs/js-bindings*/*.so pkgs/js-bindings*/*.dylib pkgs/js-bindings*/*.dll pkgs/js-bindings*/**/*.node pkgs/js-bindings*/**/*.so pkgs/js-bindings*/**/*.dylib pkgs/js-bindings*/**/*.dll",
"test": "echo \"Tests not yet implemented\""
},
"dependencies": {
Expand Down
9 changes: 7 additions & 2 deletions packages/bindings/pkgs/js-bindings/download.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,14 @@ const fs = require("fs");
const os = require("os");
const AdmZip = require("adm-zip");

const SUPPORTED_PLATFORMS = ["darwin-arm64", "linux-x64", "linux-arm64"];
const SUPPORTED_PLATFORMS = [
"darwin-arm64",
"linux-x64",
"linux-arm64",
"win32-x64",
];
const DEFAULT_BASE_URL =
"https://oceanbase-seekdb-builds.s3.ap-southeast-1.amazonaws.com/js-bindings/all_commits/7548fd4ac9bb9d8a06621dfb1ade3924a95145d6";
"https://oceanbase-seekdb-builds.s3.ap-southeast-1.amazonaws.com/js-bindings/all_commits/491beee0a7e67ba004502a83ea2da484f5fd9cdc";

function getPlatformArch() {
const key = `${process.platform}-${process.arch === "arm64" ? "arm64" : "x64"}`;
Expand Down
3 changes: 2 additions & 1 deletion packages/bindings/scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,9 @@ These scripts download libseekdb files for specific platforms. They are automati
- `fetch_libseekdb_linux_x64.py` - Linux x64
- `fetch_libseekdb_linux_arm64.py` - Linux arm64
- `fetch_libseekdb_darwin_arm64.py` - macOS arm64 (Apple Silicon)
- `fetch_libseekdb_windows_x64.py` - Windows x64

Note: Windows and macOS x64 (Intel Silicon) is not currently supported.
Note: macOS x64 (Intel) fetch/build is not wired in CI; Windows x64 is built in `build-js-bindings.yml`.

**Manual usage (if needed):**

Expand Down
Loading
Loading