Skip to content

feat(storage): Adding CumulativeHasher wrapper class for Full object …#13239

Open
Dhriti07 wants to merge 1 commit into
mainfrom
add-cumulative-hasher
Open

feat(storage): Adding CumulativeHasher wrapper class for Full object …#13239
Dhriti07 wants to merge 1 commit into
mainfrom
add-cumulative-hasher

Conversation

@Dhriti07
Copy link
Copy Markdown
Contributor

@Dhriti07 Dhriti07 commented May 20, 2026

Adding CumulativeHasher wrapper class for Full object checksum

Refer to: go/full_checksum_java

@Dhriti07 Dhriti07 requested review from a team as code owners May 20, 2026 15:08
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the CumulativeHasher class, which wraps a Hasher to track and validate cumulative CRC32C checksums for full object reads. It also updates the visibility of existing checksum exception constructors in Hasher.java. The review feedback highlights a logic gap where direct calls to hash methods were not being accumulated, potentially leading to incorrect cumulative totals. Additionally, suggestions were made to mark the new exception class as final and to use Locale.US in string formatting for consistency with existing patterns in the codebase.

Comment on lines +45 to +88
@Override
public Crc32cLengthKnown hash(ByteBuffer b) {
return delegate.hash(b);
}

@Override
public Crc32cLengthKnown hash(ByteString byteString) {
return delegate.hash(byteString);
}

@Override
public void validate(Crc32cValue<?> expected, Supplier<ByteBuffer> b) throws ChecksumMismatchException {
ByteBuffer byteBuffer = b.get();
Crc32cLengthKnown actual = delegate.hash(byteBuffer);
if (actual != null) {
if (expected != null && !actual.eqValue(expected)) {
throw new ChecksumMismatchException(expected, actual);
}
accumulate(actual);
}
}

@Override
public void validate(Crc32cValue<?> expected, ByteString byteString) throws ChecksumMismatchException {
Crc32cLengthKnown actual = delegate.hash(byteString);
if (actual != null) {
if (expected != null && !actual.eqValue(expected)) {
throw new ChecksumMismatchException(expected, actual);
}
accumulate(actual);
}
}

@Override
public void validateUnchecked(Crc32cValue<?> expected, ByteString byteString)
throws UncheckedChecksumMismatchException {
Crc32cLengthKnown actual = delegate.hash(byteString);
if (actual != null) {
if (expected != null && !actual.eqValue(expected)) {
throw new UncheckedChecksumMismatchException(expected, actual);
}
accumulate(actual);
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The CumulativeHasher should accumulate checksums for all data processed, including calls to the hash methods. Currently, hash(ByteBuffer) and hash(ByteString) only delegate without updating cumulativeHash. Updating these methods to accumulate and then having the validate methods leverage them ensures consistency and reduces duplication.

  @Override
  public Crc32cLengthKnown hash(ByteBuffer b) {
    Crc32cLengthKnown actual = delegate.hash(b);
    if (actual != null) {
      accumulate(actual);
    }
    return actual;
  }

  @Override
  public Crc32cLengthKnown hash(ByteString byteString) {
    Crc32cLengthKnown actual = delegate.hash(byteString);
    if (actual != null) {
      accumulate(actual);
    }
    return actual;
  }

  @Override
  public void validate(Crc32cValue<?> expected, Supplier<ByteBuffer> b) throws ChecksumMismatchException {
    Crc32cLengthKnown actual = hash(b.get());
    if (actual != null && expected != null && !actual.eqValue(expected)) {
      throw new ChecksumMismatchException(expected, actual);
    }
  }

  @Override
  public void validate(Crc32cValue<?> expected, ByteString byteString) throws ChecksumMismatchException {
    Crc32cLengthKnown actual = hash(byteString);
    if (actual != null && expected != null && !actual.eqValue(expected)) {
      throw new ChecksumMismatchException(expected, actual);
    }
  }

  @Override
  public void validateUnchecked(Crc32cValue<?> expected, ByteString byteString)
      throws UncheckedChecksumMismatchException {
    Crc32cLengthKnown actual = hash(byteString);
    if (actual != null && expected != null && !actual.eqValue(expected)) {
      throw new UncheckedChecksumMismatchException(expected, actual);
    }
  }

}
}

class UncheckedCumulativeChecksumMismatchException extends com.google.api.gax.rpc.DataLossException {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Mark the exception class as final for consistency with other exception classes in the com.google.cloud.storage package.

Suggested change
class UncheckedCumulativeChecksumMismatchException extends com.google.api.gax.rpc.DataLossException {
final class UncheckedCumulativeChecksumMismatchException extends DataLossException {

Comment on lines +135 to +137
String.format(
"Mismatch cumulative checksum value. Expected %s actual %s",
expected.debugString(), actual.debugString()),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Use Locale.US with String.format to ensure consistent message formatting across different environments, matching the pattern used in ChecksumMismatchException.

Suggested change
String.format(
"Mismatch cumulative checksum value. Expected %s actual %s",
expected.debugString(), actual.debugString()),
String.format(
Locale.US,
"Mismatch cumulative checksum value. Expected %s actual %s",
expected.debugString(), actual.debugString()),

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant