Skip to content

[SPARK-53314] Add DataStreamReader and DataStreamWriter convenience methods#21

Open
rafafrdz wants to merge 1 commit intoapache:masterfrom
rafafrdz:spark-53314-datastream-reader-writer-methods
Open

[SPARK-53314] Add DataStreamReader and DataStreamWriter convenience methods#21
rafafrdz wants to merge 1 commit intoapache:masterfrom
rafafrdz:spark-53314-datastream-reader-writer-methods

Conversation

@rafafrdz
Copy link
Copy Markdown

Summary

  • Add csv(), json(), orc(), parquet(), text() convenience methods to DataStreamReader, each accepting a path and ConfigOpts config
  • Add table() method to DataStreamReader for reading a stream from a named table
  • Add csv(), json(), orc(), parquet(), text() convenience methods to DataStreamWriter, each delegating to format().start()

Test plan

  • cargo build passes
  • cargo clippy passes
  • cargo fmt -- --check passes
  • Integration test with Spark Connect server (streaming sources)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant