PoC: Return the same evaluation errors from TPE as from concrete evaluation#2162
PoC: Return the same evaluation errors from TPE as from concrete evaluation#2162luxas wants to merge 1 commit intocedar-policy:mainfrom
Conversation
…uation Signed-off-by: Lucas Käldström <lucas.kaldstrom@upbound.io>
| err: err.clone(), | ||
| ty: ty.clone(), | ||
| }, | ||
| (_, Residual::Error { err, ty }) => Residual::Error { |
There was a problem hiding this comment.
from live review: the left hand side could return an error, even if it is a residual right now. To make the error exactly as it would be in the concrete case, we need a way to say "we got an error, but we not sure which"
There was a problem hiding this comment.
I was thinking about this a bit and I don't think it should be too bad. One nice way to do it might be splitting ResidualError into Residual::ConcreteError and Residual::PartialError to represent these two cases. PartialError would carry a ResidualKind instead of an EvaluationError.
Then this functions becomes
// arg1 evals either to a concrete or residual error, that error is the result
(r1, _) if r1.is_error() => make_error(r1, ty),
// r1 isn't an error variant, but it could be residual which eventually
// evaluates to an error, so we can't ignore it.
(_, r2) if r2.is_error() => residual_error(arg1, arg2, ty)
There was a problem hiding this comment.
split into
Residual::ConcreteErrorandResidual::PartialError
yep, that's my plan 👍 If we need later, we could provide a function to collect known errors from the partial error tree (if the user would like to know what definitely did go wrong).
| }) | ||
| .next() | ||
| { | ||
| // return the first error encountered, like normal eval |
There was a problem hiding this comment.
TODO: Again, we need to return "we are sure we are erroring, but do not know which error" here instead.
| match r { | ||
| Residual::Error { err, ty } => return Residual::Error { err, ty }, | ||
| _ => { | ||
| m.insert(a, r); |
There was a problem hiding this comment.
try to align with the Set logic above, and return "error, but not sure yet"
Description of changes
Draft PR, I need to double-check some of the logic here still and add some more unit tests, but it should be good enough to discuss at least. What do you think about this rewrite? I think it could help make the code between TPE and the concrete evaluator more similar and hopefully comparable, especially when the older
partial-evalis deprecated and removed from the concrete evaluator.Discussed this with @john-h-kastner-aws. The idea is to return a concrete error whenever the concrete evaluator would, and just exactly the same error as the concrete evaluator would. However, cases like
<residual> && <error>do not fold to<error>, as<residual>could evaluate to another error.Also noted that the stack overflow check was not added to TPE. Also I'm trying to make the errors nice by preserving the Loc information, but it seems like I need to do some other thing still to produce nice reports in miette to show where the error came from.
Fixes: #1736
(at least part of, we could add the
Diagnosticsin a follow-up PR)Checklist for requesting a review
The change in this PR is (choose one, and delete the other options):
I confirm that this PR (choose one, and delete the other options):
Not sure, do you want a release note for this one? I can add one.
I confirm that
cedar-spec(choose one, and delete the other options):Would you be interested in updating cedar-spec with a theorem that the errors should be the same between concrete and TPE?
I confirm that
docs.cedarpolicy.com(choose one, and delete the other options):