[WIP] Add test for presence of backtrace in API forwarded errors

[PietrH]

Added a simple test to see if backtrace information is included in API forwarded errors.

Tests #414, #477

• Set repo secret to use test api
• Turn off vcr HTTP response caching
• Turn vcr back on for CI
• change back to production etnservice deployment
• test on production etnservice deployment
• Have CI handle missing value for ETN API URL or remove it from repo secret

This is not a blocker for the ETN API release

This config change has been pushed to etnservice production, so we should be able to test this on the production API.

[PietrH]

The test deployment is still crashing on memory issues: https://github.com/inbo/etn/actions/runs/21942585345/job/63371858144?pr=452

[PietrH]

etnservice test is no longer failing on memory issues, but it might start failing again if the load on the API server increases.

[PietrH]

I can not locally replicate the test failure in the codecov CI runner. Which is indeed the test that checks for the backtrace:

test_that("opencpu should not pass backtrace", {
  skip_if_offline("opencpu.lifewatch.be")

  # Errors from the API should be forwarded, but backtrace should not be.
  error_message <- expect_error(
    forward_to_api("get_animals",
                   payload = list(animal_id = "not_an_animal_id"),
                  )
  )
  expect_no_match(error_message$message, "Backtrace")
})

This is probably because the codecov runner doesn't switch to the test API because it doesn't have access to the repo secret that is needed to get the test API url, thus it falls back to the production API deployment.

[PietrH]

I can not locally replicate the test failure in the codecov CI runner. Which is indeed the test that checks for the backtrace:

test_that("opencpu should not pass backtrace", {
  skip_if_offline("opencpu.lifewatch.be")

  # Errors from the API should be forwarded, but backtrace should not be.
  error_message <- expect_error(
    forward_to_api("get_animals",
                   payload = list(animal_id = "not_an_animal_id"),
                  )
  )
  expect_no_match(error_message$message, "Backtrace")
})

This is probably because the codecov runner doesn't switch to the test API because it doesn't have access to the repo secret that is needed to get the test API url, thus it falls back to the production API deployment.

This is indeed the case! And since the change we are testing on is only present on the test deployment, codecov fails. I think this is a desirable behavior so I don't feel the need to change it at the moment.

[PietrH]

repo config reset to use vcr and production api as preparation for ETN API release.

[PietrH]

@sannegovaert , this is no longer ready for review. I need to investigate test failures first. I'll re-request review when I'm ready again.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.76%. Comparing base (b673a44) to head (7992042).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #452   +/-   ##
=======================================
  Coverage   86.76%   86.76%           
=======================================
  Files          26       26           
  Lines         748      748           
=======================================
  Hits          649      649           
  Misses         99       99           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[PietrH]

The ETN_TEST_API repo secret was removed, this might cause the curl parsing error?

[PietrH]

The ETN_TEST_API repo secret was removed, this might cause the curl parsing error?

Nope, that's not it!

[PietrH]

It seems all API functions are failing to form correct urls, nothing in this PR should be different to main in that aspect. Why is it only failing here?

get_etnservice_version() lists available functions of etnservice ──
Error in `curl::curl_parse_url(req$url, decode = FALSE)`: Failed to parse URL: Malformed input to a URL function
Backtrace:
     ▆
  1. └─etn (local) `<fn>`()
  2.   └─etn:::forward_to_api(...) at etn/R/utils-api.R:232:3
  3.     ├─httr2::req_retry(...) at etn/R/forward_to_api.R:55:3
  4.     │ └─httr2:::check_request(req) at httr2/R/req-retries.R:101:3
  5.     │   └─httr2:::is_request(req) at httr2/R/req.R:76:5
  6.     ├─httr2::req_body_json(...) at etn/R/forward_to_api.R:55:3
  7.     │ └─httr2:::check_request(req) at httr2/R/req-body.R:103:3
  8.     │   └─httr2:::is_request(req) at httr2/R/req.R:76:5
  9.     └─httr2::req_url_path_append(...) at etn/R/forward_to_api.R:55:3
 10.       └─curl::curl_parse_url(req$url, decode = FALSE) at httr2/R/req-url.R:102:3

I need a sanity check, is main broken?

[PietrH]

It's got something to do with the test api url injection, when I add the url secret back in, I get a 403 forbidden. Did I paste the wrong url?

[PietrH]

The situation is this, it works when a valid url is set to ETN_TEST_API, but fails otherwise. Since main doesn't seem to suffer form this issue (and it has the same CI), I believe the issue lies in the situation where the workflow has access to a system environmental variable, but it's unset in the repo.

I think removing variable entirely from the workflow would work, but I think it would be even better if the package would cope correctly with this status. The first step is replicating this locally.

What happens if you set ETN_TEST_API to a random string that is not a valid url?