Skip to content

Evaluation: Sanitize SQL error messages #691

@vprashrex

Description

@vprashrex

Is your feature request related to a problem?
The /api/v1/evaluations?limit=-1&offset=0 endpoint returns a raw SQL error to the client. This is problematic as clients should receive a proper sanitized error message instead of technical details.

Describe the solution you'd like

  • Implement error handling to return a sanitized error message for SQL issues.
  • Use FastAPI query parameters that enforce limits between 1 and 100, with a default of 50:
    limit: int = Query(
        default=50, ge=1, le=100, description="Maximum number of datasets to return"
    )
Original issue

Describe the bug
A clear and concise description of what the bug is.

/api/v1/evaluations?limit=-1&offset=0
this endpoint returns raw sql error, the client should not be getting errors like this, instead client should get proper sanitized error

{
    "success": false,
    "data": null,
    "error": "(psycopg.errors.InvalidRowCountInLimitClause) LIMIT must not be negative\n[SQL: SELECT evaluation_run.id, evaluation_run.run_name, evaluation_run.dataset_name, evaluation_run.type, evaluation_run.language_id, evaluation_run.providers, evaluation_run.config_id, evaluation_run.config_version, evaluation_run.dataset_id, evaluation_run.batch_job_id, evaluation_run.embedding_batch_job_id, evaluation_run.status, evaluation_run.object_store_url, evaluation_run.score_trace_url, evaluation_run.total_items, evaluation_run.score, evaluation_run.error_message, evaluation_run.organization_id, evaluation_run.project_id, evaluation_run.inserted_at, evaluation_run.updated_at \nFROM evaluation_run \nWHERE evaluation_run.organization_id = %(organization_id_1)s::INTEGER AND evaluation_run.project_id = %(project_id_1)s::INTEGER AND evaluation_run.type = %(type_1)s::VARCHAR ORDER BY evaluation_run.inserted_at DESC \n LIMIT %(param_1)s::INTEGER OFFSET %(param_2)s::INTEGER]\n[parameters: {'organization_id_1': 1, 'project_id_1': 1, 'type_1': 'text', 'param_1': -1, 'param_2': 0}]\n(Background on this error at: https://sqlalche.me/e/20/9h9h)",
    "errors": null,
    "metadata": null
}

use fastapi query for this which will handle all this

    limit: int = Query(
        default=50, ge=1, le=100, description="Maximum number of datasets to return"
    ),

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions