Skip to content

feat: Add cancellation support for remote operations (plot, upload, gfql_remote) #922

@lmeyerov

Description

@lmeyerov

Summary

Add the ability to cancel in-flight remote operations when using plot(), upload(), and gfql_remote().

Motivation

When integrating pygraphistry with systems that support background task cancellation (like GraphistryGPT), there's currently no way to cancel running remote operations. These operations can block for extended periods:

  • upload() - Large dataset uploads (100MB+) can take minutes
  • plot() - Combines upload + server-side processing
  • gfql_remote() - Server-side GFQL execution (hypergraph, UMAP, etc.) can be very slow for large graphs

Currently all these use synchronous requests.post() with no timeout or cancellation:

# chain_remote.py:111
response = requests.post(url, headers=headers, json=request_body, ...)

Proposed API

Add an optional on_cancel_ready callback parameter that receives a cancel function:

def gfql_remote(
    self,
    chain: ...,
    on_cancel_ready: Callable[[Callable[[], None]], None] | None = None,
    ...
) -> Plottable:
    """
    Args:
        on_cancel_ready: Optional callback that receives a cancel function.
                        Called before the HTTP request starts.
    """
    session = requests.Session()
    
    def cancel():
        session.close()  # Interrupts in-flight request
    
    if on_cancel_ready:
        on_cancel_ready(cancel)
    
    response = session.post(url, ...)

Similar pattern for plot() and upload().

Usage Example

import graphistry

def register_cancel(cancel_fn):
    # Store cancel_fn for later use (e.g., in a task manager)
    task_manager.register_canceler(cancel_fn)

# Cancel-aware remote GFQL
result = g.gfql_remote(
    call('hypergraph', {'entity_types': ['user', 'product']}),
    on_cancel_ready=register_cancel
)

# Cancel-aware upload
g.upload(on_cancel_ready=register_cancel)

# Cancel-aware plot
g.plot(render=False, on_cancel_ready=register_cancel)

Implementation Notes

  1. Client-side cancellation: Use requests.Session() and call session.close() to interrupt in-flight requests

  2. Server-side cancellation: For gfql_remote, the Graphistry server should also support job cancellation via a separate API endpoint (see graphistry/graphistry#3006)

  3. The callback pattern: Allows callers to decide how to store/use the cancel function without pygraphistry needing to know about task managers, context vars, etc.

Operations Affected

Operation Blocks On Cancellation Mechanism
upload() HTTP POST (data upload) Close session
plot() HTTP POST (upload + server processing) Close session
gfql_remote() HTTP POST (GFQL execution) Close session + server job cancel

Related

This is part of a broader effort to add cancellation support across remote operations in GraphistryGPT.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions