feat(tracing): Adding tracing for Buffered reader flows

[thrivikram-karur-g]

Description

Adding tracing for Buffered reader flows .
(Dependent on PR: #4244 to be merged).

Link to the issue in case of a bug fix.

b/491801017

Testing details

1. Manual - NA
2. Unit tests - NA
3. Integration tests - NA

Any backward incompatible change? If so, please explain.

N/A

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the observability of read operations by integrating detailed tracing for buffered reader flows. The changes provide a granular view into how data is prefetched, cached, and read, enabling better performance analysis and bottleneck identification. By standardizing span names and introducing context propagation, the system now offers a clearer, more interconnected trace of read requests across different components.

Highlights

• Tracing for Buffered Reader Flows: Introduced comprehensive tracing capabilities for buffered reader operations, providing detailed visibility into prefetching, block management, and read execution paths.
• Context Propagation and Span Linking: Implemented context.Context propagation across various reader components and added span linking to connect related tracing spans, improving the visualization of complex asynchronous operations like file downloads and block scheduling.
• Standardized Span Naming and Attributes: Refactored and standardized span names using a hierarchical fs. prefix for filesystem operations and introduced new, descriptive span names for cache and prefetching activities. Added specific attributes for cache read operations (e.g., cache.hit, read.size).
• Reader Interface Enhancement: Extended the Reader interface with a ReaderName() method, allowing each reader implementation to identify itself in tracing spans, which aids in debugging and performance analysis.

Changelog

• internal/bufferedread/buffered_reader.go
  • Added tracing spans for prefetch block pool generation, waiting for prefetch blocks, and reading from prefetch blocks.
  • Introduced a ReaderName() method to identify the buffered reader.
  • Modified several internal functions (freshStart, prefetch, scheduleNextBlock, scheduleBlockWithIndex) to accept and propagate context.Context.
• internal/bufferedread/buffered_reader_test.go
  • Updated test calls to internal buffered reader functions to pass context.Background().
• internal/bufferedread/download_task.go
  • Added a traceHandle field to the downloadTask struct.
  • Implemented tracing spans for DownloadPrefetchBlock and recorded errors within the download task.
• internal/cache/file/downloader/job.go
  • Added tracing context propagation and span linking for the FileDownloadJob in the Download method.
• internal/fs/tracing_test.go
  • Disabled buffered read in the test configuration.
  • Updated span names in tracing test cases to align with the new fs. prefixed naming convention.
• internal/fs/wrappers/tracing.go
  • Updated the import alias for the tracing package.
• internal/gcsx/client_readers/gcs_reader.go
  • Added a traceHandle to GCSReaderConfig.
  • Modified NewGCSReader to pass the traceHandle to NewRangeReader and NewMultiRangeReader.
  • Introduced a ReaderName() method to identify the GCS reader.
• internal/gcsx/client_readers/multi_range_reader.go
  • Added a traceHandle field to the MultiRangeReader struct.
  • Modified NewMultiRangeReader to accept and store the traceHandle.
  • Updated readFromMultiRangeReader to pass the traceHandle to the underlying mrdWrapper.Read call.
• internal/gcsx/client_readers/multi_range_reader_test.go
  • Updated NewMultiRangeReader calls to include tracing.NewNoopTracer().
• internal/gcsx/client_readers/range_reader.go
  • Added a traceHandle field to the RangeReader struct.
  • Modified NewRangeReader to accept and store the traceHandle.
  • Updated startRead to accept context.Context and propagate tracing context.
  • Modified readFromRangeReader to pass context.Context to startRead.
• internal/gcsx/client_readers/range_reader_test.go
  • Updated NewRangeReader calls to include tracing.NewNoopTracer().
• internal/gcsx/file_cache_reader.go
  • Introduced a ReaderName() method to identify the file cache reader.
  • Added tracing spans for FileCacheRead within tryReadingFromFileCache, including setting cache read attributes and recording errors.
• internal/gcsx/mock_reader.go
  • Added a ReaderName() method for the mock reader.
• internal/gcsx/multi_range_downloader_wrapper.go
  • Modified ensureMultiRangeDownloader to accept context.Context and tracing.TraceHandle, and to propagate tracing context.
  • Updated NewMultiRangeDownloader calls to pass the context.Context.
  • Modified the Read method to accept tracing.TraceHandle and pass it to ensureMultiRangeDownloader.
• internal/gcsx/multi_range_downloader_wrapper_test.go
  • Updated Read and ensureMultiRangeDownloader calls to pass tracing.NewNoopTracer() and context.Background().
• internal/gcsx/random_reader.go
  • Added tracing spans for FileCacheRead in tryReadingFromFileCache, including setting cache read attributes and recording errors.
  • Modified startRead to accept context.Context and propagate tracing context.
  • Updated readFromRangeReader to pass context.Context to startRead.
  • Modified readFromMultiRangeReader to pass traceHandle to mrdWrapper.Read.
• internal/gcsx/random_reader_test.go
  • Updated startRead calls to pass context.Background().
• internal/gcsx/read_manager/mock_read_manager.go
  • Added a ReaderName() method for the mock read manager.
• internal/gcsx/read_manager/read_manager.go
  • Added TraceHandle to the NewReadManager configuration.
  • Introduced a ReaderName() method to identify the read manager.
  • Added tracing spans for each reader within the ReadAt loop.
• internal/gcsx/read_manager/visual_read_manager.go
  • Added a ReaderName() method for the visual read manager.
• internal/gcsx/reader.go
  • Added a ReaderName() method to the Reader interface.
• internal/gcsx/shared_chunk_cache_reader.go
  • Added a ReaderName() method for the shared chunk cache reader.
• tracing/noop_tracer.go
  • Implemented StartSpanLink and SetCacheReadAttributes methods as no-op functions.
• tracing/otel_tracer.go
  • Added parentLinkAttributes for GCP Cloud Trace visualization.
  • Implemented StartSpanLink to create spans with links to parent spans.
  • Implemented SetCacheReadAttributes to add cache hit and bytes read attributes to spans.
• tracing/span_attributes.go
  • Added a new file defining constants IS_CACHE_HIT and BYTES_READ for tracing span attributes.
• tracing/span_names.go
  • Refactored and categorized existing span names with fs. prefixes.
  • Added new span names for cache and prefetching operations, including descriptive comments for each.
• tracing/trace_handle.go
  • Added StartSpanLink and SetCacheReadAttributes to the TraceHandle interface.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

Hi @kislaykishore, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you!

[github-actions]

Hi @kislaykishore, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you!

[github-actions]

Hi @kislaykishore, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you!

[github-actions]

Hi @kislaykishore, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you!

[github-actions]

Hi @kislaykishore, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you!

[github-actions]

Hi @kislaykishore, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you!

[kislaykishore]

At the moment, this is not a priority.