works

[BenCowen]

Type of Change

This is a bit of a workaround for "custom retries" (i.e. retrying via modal.Retries on some exceptions, and circumventing retries on other, more fatal errors). It's based on this demo I made for Achira last year, and I'm bringing it up again because another customer asked for it here.

The demo follows a scripted sequence of errors that requires each function execution to "know" which iteration it's on. This is accomplished with a modal.Dict but could potentially simplify this by perhaps choosing a random error? (but then, it's random)

• New example for the GitHub repo

Monitoring Checklist

• Example is configured for testing in the synthetic monitoring system, or lambda-test: false is provided in the example frontmatter and I have gotten approval from a maintainer
  • [X Example is tested by executing with modal run, or an alternative cmd is provided in the example frontmatter (e.g. cmd: ["modal", "serve"])
  • Example is tested by running the cmd with no arguments, or the args are provided in the example frontmatter (e.g. args: ["--prompt", "Formula for room temperature superconductor:"]
  • Example does not require third-party dependencies besides fastapi to be installed locally (e.g. does not import requests or torch in the global scope or other code executed locally)

Documentation Site Checklist

Content

• Example is documented with comments throughout, in a Literate Programming style
• All media assets for the example that are rendered in the documentation site page are retrieved from modal-cdn.com

Build Stability

• Example pins all dependencies in container images
  • Example pins container images to a stable tag like v1, not a dynamic tag like latest
  • [] Example specifies a python_version for the base image, if it is used
  • --> currently just using default image, should i add .debian_slim(add_python=...)?
  • Example pins all dependencies to at least SemVer minor version, ~=x.y.z or ==x.y, or we expect this example to work across major versions of the dependency and are committed to maintenance across those versions
    • Example dependencies with version < 1 are pinned to patch version, ==0.y.z

Outside Contributors

You're great! Thanks for your contribution.

---

[BenCowen]

Might also want SDK's blessing on this (hence @thomasjpfan tag)?

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 2 additional findings.

[thomasjpfan]

Do we have a list of exceptions that we'll retry? Do users want to configure their function so they can control when their retries?

I'm thinking of:

@app.function(retries=modal.Retries(max_retries=5, initial_delay=1.0)) 
def my_func():
    try:
        my_custo_funcdtion()
    except MyCustomException as exc:
        raise ModalRetriableError(...) from exc  # signal to model that we want to retry.
       

[BenCowen]

Do users want to configure their function so they can control when their retries

I have only gotten this request twice in the last year, so possibly not often enough to dedicate engineering...

Do we have a list of exceptions that we'll retry

Right now, retries are configured based on task status, and there is no exception filtering.

[thomasjpfan]

In this example, is the proposal to have users do this?

try:
    my_custom_function()
except MyCustomException as exc:
    raise ConnectionError("manually trigger retry")

[BenCowen]

Since modal.Retries will catch all exceptions in general, it depends which approach makes sense for the app.

This would be the pattern for e.g. an inference app that should retry except for specific circumstances (e.g. invalid input)

try:
    custom_f()
except CRASHABLE_EXCEPTIONS as exc:
    return exc # bypass modal.Retries
else:
    raise # triggers retry

This would be the pattern for e.g. a training function that should always crash except for timeouts

try:
    custom_f()
except modal.exception.ExecTimeoutError  as exc:
    raise exc # use modal.Retries
else:
    return # something's wrong , stop retrying

[thomasjpfan]

User wise, I guess it's hard to know what the crashable exceptions are.

[thomasjpfan]

User wise, I guess it's hard to know what the crashable exceptions are.

[thomasjpfan]

DevX wise, is it good to have Modal see a success here? It kind of hides the failure in the UI, etc.