From e3ad8bc90bdfe4bc34c041d0e67a69c26098fd7d Mon Sep 17 00:00:00 2001 From: Rita Chen Date: Tue, 17 Mar 2026 17:11:09 -0400 Subject: [PATCH] ghcp(crawl): ex1 repo orientation + low-risk module --- ai-track-docs/SYSTEM-OVERVIEW.md | 95 ++++++++++++++++++++++++++++---- 1 file changed, 83 insertions(+), 12 deletions(-) diff --git a/ai-track-docs/SYSTEM-OVERVIEW.md b/ai-track-docs/SYSTEM-OVERVIEW.md index 107376b0c..558b2b9ad 100644 --- a/ai-track-docs/SYSTEM-OVERVIEW.md +++ b/ai-track-docs/SYSTEM-OVERVIEW.md @@ -2,7 +2,7 @@ ## Purpose -The **MarkLogic Java Client API** (`com.marklogic:marklogic-client-api`) is a Java library that exposes MarkLogic Server's REST API as a type-safe, fluent Java interface. It supports reading, writing, deleting, and querying JSON, XML, binary, and text documents, as well as ACID multi-statement transactions, semantic (SPARQL/RDF), Full-text search, alerting, Data Services, and Row Manager (Optic API). +The **MarkLogic Java Client API** (`com.marklogic:marklogic-client-api`) is a Java library that exposes MarkLogic Server's REST API as a type-safe, fluent Java interface. It supports reading, writing, deleting, and querying JSON, XML, binary, and text documents, as well as ACID multi-statement transactions, semantic (SPARQL/RDF), full-text search, alerting, Data Services, and Row Manager (Optic API). ## Repository Root @@ -10,13 +10,81 @@ The **MarkLogic Java Client API** (`com.marklogic:marklogic-client-api`) is a Ja ## Modules -| Module | Description | -| -------------------------------------- | ---------------------------------------------------------------------------- | -| `marklogic-client-api` | Core library — all production source code | -| `marklogic-client-api-functionaltests` | Functional / integration tests requiring a live MarkLogic instance | -| `ml-development-tools` | Kotlin-based developer tooling (code generation helpers) | -| `test-app` | MarkLogic application deployed to the test server (modules, schemas, config) | -| `examples` | Standalone usage examples | +| Module | Folder | Description | +| -------------------------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------------ | +| `marklogic-client-api` | `marklogic-client-api/` | Core library — all production source code (~616 Java files) | +| `marklogic-client-api-functionaltests` | `marklogic-client-api-functionaltests/` | Functional / integration tests — all require a live MarkLogic instance (147 files) | +| `ml-development-tools` | `ml-development-tools/` | Kotlin-based Gradle plugin + CLI tools for Data Services code generation (8 Kotlin source files) | +| `test-app` | `test-app/` | MarkLogic application deployed to the test server (modules, schemas, ML config) | +| `examples` | `examples/` | Standalone usage examples (63 Java files, no automated tests) | + +## Languages + +| Language | Location | Notes | +| ------------------ | ----------------------------------------------------------------------------------------- | ----------------------------------------------- | +| **Java** | `marklogic-client-api/src/`, `examples/src/`, `marklogic-client-api-functionaltests/src/` | Primary language (~997 source + test files) | +| **Kotlin** | `ml-development-tools/src/main/kotlin/` | Code generator and Gradle plugin only | +| **Server-Side JS** | `marklogic-client-api/src/test/resources/`, `ml-development-tools/src/test/` | Test transforms and module stubs (.sjs/.mjs) | +| **XQuery** | `marklogic-client-api/src/test/resources/` | Eval tests, rule transforms | +| **XML / JSON** | Throughout `src/test/resources/` and `test-app/src/` | Test fixtures, ML config, endpoint declarations | + +## Public API Entry Points + +All live under `marklogic-client-api/src/main/java/com/marklogic/client/`. + +| Class | Path | Role | +| ----------------------------------------- | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `DatabaseClientBuilder` | `DatabaseClientBuilder.java` | **Preferred modern entry point** (since 6.1.0). Fluent builder: `.withHost()`, `.withPort()`, `.withBasicAuth()`, `.withConnectionString()`, `.build()`. | +| `DatabaseClientFactory` | `DatabaseClientFactory.java` | Older static factory; also hosts nested auth-context types (`BasicAuthContext`, `DigestAuthContext`, `CloudAuthContext`, etc.) and the Spring-friendly `Bean` class. | +| `DatabaseClient` | `DatabaseClient.java` | Central interface returned by both factory and builder. Gateway to all managers via `.newJSONDocumentManager()`, `.newQueryManager()`, `.newRowManager()`, `.newDataMovementManager()`, `.openTransaction()`, etc. | +| `QueryManager` / `StructuredQueryBuilder` | `query/` (33 classes) | Full-text and structured search DSL, obtained via `client.newQueryManager()`. | +| `DataMovementManager` | `datamovement/DataMovementManager.java` | Bulk/streaming operations. Creates `QueryBatcher` (bulk read/export) and `WriteBatcher` (bulk write). ~45 interfaces/classes in `datamovement/`. | + +## Test Approach + +- **Framework**: JUnit 5 throughout `marklogic-client-api` and `marklogic-client-api-functionaltests`; JUnit 4 in `ml-development-tools` (intentionally kept — noted in `ml-development-tools/build.gradle`). +- **Mocking**: Mockito present but minimal (2 files: `SPARQLQueryManagerImplTest.java`, `SSLTest.java`). OkHttp `MockWebServer` used for HTTP-level tests. The dominant pattern is a real MarkLogic connection via `Common.newClient()`. +- **Server dependency split** (core module, 214 test files): roughly half require a live MarkLogic instance; half run standalone. +- **Functional tests** (`marklogic-client-api-functionaltests/`): 100% server-dependent. MarkLogic is started via `docker-compose.yaml`. + + > **Assumption**: The ~1:1 standalone/server split was estimated by scanning for `Common.connect`/`Common.newClient` imports. Verify with: + > + > ```powershell + > $t="marklogic-client-api\src\test\java" + > (Get-ChildItem -Path $t -Recurse -Filter "*.java" | + > Select-String "Common\.connect|Common\.newClient" | + > Group-Object Path | Measure-Object).Count + > ``` + +## Recommended Low-Risk Starting Point + +**`MarkLogicVersion`** — a pure string-parsing utility with full standalone test coverage. + +| File | Path | +| ------ | ---------------------------------------------------------------------------------------- | +| Source | `marklogic-client-api/src/test/java/com/marklogic/client/test/MarkLogicVersion.java` | +| Test | `marklogic-client-api/src/test/java/com/marklogic/client/test/MarkLogicVersionTest.java` | + +Why it's the safest first touch: + +- **No server, no Docker** — runs in seconds with no infrastructure. +- **Blast radius is zero** — parsing utility; nothing in the auth or network path depends on it. +- **Fast feedback loop** — 8 parameterised JUnit 5 tests cover the full version-string format. +- **Tiny surface area** — both files together are under 100 lines; ideal for getting comfortable with the project's code style and test conventions. + +Run it in isolation: + +```bash +./gradlew :marklogic-client-api:test --tests "com.marklogic.client.test.MarkLogicVersionTest" +``` + +Other low-risk standalone targets (no server needed): + +| Class | Source path | Test path | +| ------------------------------ | ------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------- | +| `PlanRowColTypesImpl` | `marklogic-client-api/src/main/java/com/marklogic/client/impl/PlanRowColTypesImpl.java` | `marklogic-client-api/src/test/java/com/marklogic/client/impl/PlanRowColTypesImplTest.java` | +| `TransformDefImpl` | `marklogic-client-api/src/main/java/com/marklogic/client/impl/TransformDefImpl.java` | `marklogic-client-api/src/test/java/com/marklogic/client/impl/TransformDefImplTest.java` | +| `DatabaseClientPropertySource` | `marklogic-client-api/src/main/java/com/marklogic/client/impl/DatabaseClientPropertySource.java` | `marklogic-client-api/src/test/java/com/marklogic/client/impl/DatabaseClientPropertySourceTest.java` | ## Runtime Requirements @@ -26,16 +94,19 @@ The **MarkLogic Java Client API** (`com.marklogic:marklogic-client-api`) is a Ja ## Technology Stack - Build: **Gradle** (wrapper at `./gradlew`) -- Test framework: **JUnit 5** (unit) + MarkLogic functional test harness +- Test framework: **JUnit 5** (core) / **JUnit 4** (`ml-development-tools`) - CI: **Jenkins** (`Jenkinsfile`) — Docker-based MarkLogic, parallel Java 17/21 builds - Primary language: **Java**; developer tooling in **Kotlin** ## Key External Dependencies -- OkHttp (HTTP client transport) -- Jackson (JSON serialization) +- OkHttp 3 (HTTP client transport) +- Jackson Databind + Jackson CSV (JSON serialization) - SLF4J / Logback (logging) +- JAXB 4 / Glassfish runtime (XML binding) +- `okhttp-digest` (Digest auth) +- Optional: JDOM2, dom4j, Gson ## Relationship to MarkLogic Server -All network communication travels over the **MarkLogic REST Management and Client APIs** (typically port 8000/8002). The library never connects directly to MarkLogic's internal ports; authentication is via HTTP Digest or certificate. +All network communication travels over the **MarkLogic REST Client and Management APIs** (typically port 8000/8002). The library never connects to MarkLogic's internal ports directly; authentication is via HTTP Basic, Digest, certificate, Kerberos, SAML, or OAuth/Cloud token.