Skip to content

KAFKA-14473: Move AbstractIndex to storage module#13007

Merged
ijuma merged 2 commits intoapache:trunkfrom
ijuma:kafka-14473-move-abstract-index-to-storage
Dec 20, 2022
Merged

KAFKA-14473: Move AbstractIndex to storage module#13007
ijuma merged 2 commits intoapache:trunkfrom
ijuma:kafka-14473-move-abstract-index-to-storage

Conversation

@ijuma
Copy link
Member

@ijuma ijuma commented Dec 17, 2022

For broader context on this change, please check:

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@ijuma ijuma force-pushed the kafka-14473-move-abstract-index-to-storage branch 3 times, most recently from 049719f to d686dda Compare December 17, 2022 18:40
@ijuma ijuma force-pushed the kafka-14473-move-abstract-index-to-storage branch from d686dda to 0aab4d5 Compare December 17, 2022 20:00
Copy link
Member

@satishd satishd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ijuma for the PR.

/**
* The abstract index class which holds entry format agnostic methods.
*/
public abstract class AbstractIndex {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Earlier AbstractIndex is Closeable. But this java class is missing implementing Closeable.

abstract class AbstractIndex(@volatile private var _file: File, val baseOffset: Long, val maxIndexSize: Int = -1,
                             val writable: Boolean) extends Closeable

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I'll fix this soon.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@ijuma ijuma force-pushed the kafka-14473-move-abstract-index-to-storage branch 2 times, most recently from c68327a to ea87980 Compare December 19, 2022 15:03
Copy link
Member

@satishd satishd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ijuma for addressing the review comments. LGTM.

@ijuma ijuma force-pushed the kafka-14473-move-abstract-index-to-storage branch from ea87980 to 3033552 Compare December 19, 2022 19:42
@ijuma ijuma requested a review from junrao December 19, 2022 19:42
Copy link
Contributor

@junrao junrao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ijuma : Thanks for the PR. Just a couple of minor comments.

else
return new BinarySearchResult(mid, mid);
}
int blah;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of defining blah, could we just reuse hi?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, I meant to rename this, but missed it. Your suggestion sounds good.

}
}

// GuardedBy `lock`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we change the comment to the following?

The caller is expected to hold lock when calling this method.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to use the @GuardedBy annotation, but there are some licensing issues with the jcip jar (LGPL). I'll go with your suggested comment for now.

storage/src/main/java/org/apache/kafka/server/log/internals/TimeIndex.java
@ijuma
Copy link
Member Author

ijuma commented Dec 19, 2022

@junrao Thanks for the review, I addressed the comments.

Copy link
Contributor

@junrao junrao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ijuma : Thanks for the updated PR. LGTM

@ijuma
Copy link
Member Author

ijuma commented Dec 20, 2022

Test failures are unrelated.

Build / JDK 11 and Scala 2.13 / org.apache.kafka.streams.integration.SmokeTestDriverIntegrationTest.[1] true
Build / JDK 17 and Scala 2.13 / org.apache.kafka.connect.mirror.integration.MirrorConnectorsWithCustomForwardingAdminIntegrationTest.testCreatePartitionsUseProvidedForwardingAdmin()
Build / JDK 8 and Scala 2.12 / kafka.api.PlaintextAdminIntegrationTest.testInvalidAlterConfigs(String).quorum=zk
Build / JDK 8 and Scala 2.12 / kafka.api.PlaintextAdminIntegrationTest.testConsumeAfterDeleteRecords(String).quorum=zk

@ijuma ijuma merged commit d521f81 into apache:trunk Dec 20, 2022
@ijuma ijuma deleted the kafka-14473-move-abstract-index-to-storage branch December 20, 2022 03:33
guozhangwang pushed a commit to guozhangwang/kafka that referenced this pull request Jan 25, 2023
For broader context on this change, please check:

* KAFKA-14470: Move log layer to storage module

Reviewers: Jun Rao <junrao@gmail.com>, Satish Duggana <satishd@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants