Skip to content

test(bqjdbc): add e2e otel test and code refinements#13226

Draft
keshavdandeva wants to merge 8 commits into
jdbc/feature-branch-otelfrom
jdbc/e2e-test-otel
Draft

test(bqjdbc): add e2e otel test and code refinements#13226
keshavdandeva wants to merge 8 commits into
jdbc/feature-branch-otelfrom
jdbc/e2e-test-otel

Conversation

@keshavdandeva
Copy link
Copy Markdown
Contributor

b/499079838

Changes

1. BigQueryJdbcOpenTelemetry.java

  • Feature: Added safe, generous default attribute value length limits of 60KB (61,440 characters) to the autoconfigured OpenTelemetry instance properties.
  • Why: Prevents GCP Cloud Trace from silently rejecting and dropping span batches when we log massive BigQueryException stack traces or Arrow schema payloads exceeding the hard 64KB Cloud Trace backend limits.
  • Design: If the user explicitly configures their own limits the driver automatically skips the defaults and respects their overrides.

2. OpenTelemetryJulHandler.java

  • Fix: Configured the handler level to Level.ALL in the constructor.
  • Why: Bypasses a standard Java Logging (JUL) constraint where handlers default to Level.INFO and silently drop FINE/DEBUG queries. Delegates log filtering exclusively to the Connection loggers.

3. BigQueryConnection.java

  • Visibility: Exposed the visibility of the connection session identifier by changing getConnectionId() from package-private to public.
  • Why: Allows automated E2E tests to retrieve the UUID and harvest specific logs/traces accurately.

4. ITOpenTelemetryTest.java

  • Feature: Implemented a new standalone E2E integration test suite verifying the live GCP OTel egress.
  • Test 1 (testExecute_withOpenTelemetryGcpExporter): Natively resolves target project via ServiceOptions.getDefaultProjectId(). Runs an optimized in-memory array query and iterates results to trigger small-page JSON pagination. Queries Cloud Trace E2E to strictly assert that async pagination child spans are parented perfectly under the root JDBC span.
  • Test 2 (testExecute_withErrorCorrelation): Triggers database failures, captures SQLException, harvests Trace IDs from standard logs, and verifies failed span ingestion in Cloud Trace.

5. BigQueryConnectionTest.java

  • Feature: Added a new unit test (testConnect_withCustomOpenTelemetry_usesCustomInstance) verifying the custom OTel injection pipeline.
  • What it does: Leverages OpenTelemetryExtension to mock an OTel provider locally. Injecting the custom SDK via properties, it validates that BigQueryConnection resolves the instance and routes spans exclusively to the custom provider

6. pom.xml

  • Dependencies: Added google-cloud-trace test-scoped dependency to query Cloud Trace v1 API programmatically during E2E validation.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces default limits (60KB) for OpenTelemetry attribute values to prevent ingestion failures in GCP Cloud Trace. It also exposes the connection ID, configures the JUL handler to capture all log levels, and adds comprehensive unit and integration tests for OpenTelemetry integration. Feedback focuses on improving the new integration test by switching from JUnit 4 to JUnit 5 for consistency and adhering to standard naming conventions for static constants.

Comment on lines +19 to +41
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertNotNull;
import static org.junit.Assert.assertTrue;

import com.google.api.gax.paging.Page;
import com.google.cloud.ServiceOptions;
import com.google.cloud.bigquery.jdbc.BigQueryConnection;
import com.google.cloud.logging.LogEntry;
import com.google.cloud.logging.Logging;
import com.google.cloud.logging.LoggingOptions;
import com.google.cloud.trace.v1.TraceServiceClient;
import com.google.devtools.cloudtrace.v1.Trace;
import com.google.devtools.cloudtrace.v1.TraceSpan;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import org.junit.Test;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The project appears to be using JUnit 5 (as seen in BigQueryConnectionTest.java), but this new integration test uses JUnit 4 (org.junit.Test and org.junit.Assert). For consistency and to leverage modern testing features, please switch to JUnit 5.

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;

import com.google.api.gax.paging.Page;
import com.google.cloud.ServiceOptions;
import com.google.cloud.bigquery.jdbc.BigQueryConnection;
import com.google.cloud.logging.LogEntry;
import com.google.cloud.logging.Logging;
import com.google.cloud.logging.LoggingOptions;
import com.google.cloud.trace.v1.TraceServiceClient;
import com.google.devtools.cloudtrace.v1.Trace;
import com.google.devtools.cloudtrace.v1.TraceSpan;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.Assumptions;

Comment on lines +45 to +49
private static final String projectId = ServiceOptions.getDefaultProjectId();
private static final String connectionUrl =
String.format(
"jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=%s;OAuthType=3;Timeout=3600;",
projectId);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Static final fields should follow the UPPER_SNAKE_CASE naming convention (e.g., PROJECT_ID). Additionally, consider using Assumptions.assumeTrue(PROJECT_ID != null) at the beginning of the tests to gracefully skip them if no default project is configured. Note: Per repository rules, avoid declaring constants as 'static final' if there is a plan to configure them at the instance level in the future.

Suggested change
private static final String projectId = ServiceOptions.getDefaultProjectId();
private static final String connectionUrl =
String.format(
"jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=%s;OAuthType=3;Timeout=3600;",
projectId);
private static final String PROJECT_ID = ServiceOptions.getDefaultProjectId();
private static final String CONNECTION_URL =
String.format(
"jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=%s;OAuthType=3;Timeout=3600;",
PROJECT_ID);
References
  1. Avoid declaring constants as 'static final' if there is a plan to expose or configure them at the instance level in the future.

Base automatically changed from jdbc/complete-otel-intrumentation to jdbc/feature-branch-otel May 19, 2026 12:43
@keshavdandeva keshavdandeva changed the base branch from jdbc/feature-branch-otel to main May 19, 2026 12:45
@keshavdandeva keshavdandeva changed the base branch from main to jdbc/feature-branch-otel May 19, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants