Add FoldLeft, FoldLeftX; and ForAllX, ForAnyX, FilteredX, NumberX, PerformX to complement ListX, SetX, SumX, ProductX

[fingolfin]

This PR adds a new operation FoldLeft and a more general helper FoldLeftX,
then uses FoldLeftX to simplify the implementation of the existing
ListX, SumX, and ProductX functions and to add several further
X-style functions:
ForAllX, ForAnyX, FilteredX, NumberX, and PerformX.

FoldLeft is implemented as an operation with early methods for built-in
lists and regular methods for general collections. In particular, lists with
holes are handled consistently by iterating past holes, just as in ordinary
for-loops.

FoldLeftX takes a list of generators and filters describing nested loops and
conditions, together with an accumulator update function and an initial value.
It also supports an optional stopping value for efficient short-circuiting,
which is used by ForAllX and ForAnyX.

One idea for a future follow-up would be to broaden the scope and allow
ListX and related functions to accept iterators directly. But this PR
has already been here since 2019 (and the underlying code even longer),
so I'd rather not tackle this now.

AI disclosure: I used Codex for reviewing the branch, implementing the final
polish, extending tests, and completing the documentation.

[ChrisJefferson]

I think this looks very reasonable.

For the shortcutting, I'm tempted to suggest an optional single argument which is a object which is compared using IsIdenticalObj. My reasoning is:

1. Nice and efficient.
2. The values I expect most commonly to be tested are 0, true, false and fail, where this will work.
3. If we make an otherwise unused bag, we can use that value when nothing was given by the user (I thought about using null, but that might mean we short-circut early when given a function which doesn't return, which might be surprising).

[fingolfin]

@ChrisJefferson I like that idea! We could simply pass "" instead of a special "unused bag", as that would be a completely fresh object guaranteed to be non-identical to any other (of course that's a slight inefficiency, too, but I'd imagine not measurable in practice).

[wilfwilson]

I like this project.

[codecov]

Codecov Report

Merging #3357 (cc79085) into master (12c3bac) will increase coverage by 11.21%.
The diff coverage is 76.27%.

❗ Current head cc79085 differs from pull request most recent head aed891a. Consider uploading reports for the commit aed891a to get more accurate results

@@             Coverage Diff             @@
##           master    #3357       +/-   ##
===========================================
+ Coverage   82.18%   93.39%   +11.21%     
===========================================
  Files         678      716       +38     
  Lines      287772   812207   +524435     
===========================================
+ Hits       236500   758590   +522090     
- Misses      51272    53617     +2345     

Impacted Files	Coverage Δ
src/stats.c	94.48% <ø> (-0.41%)	⬇️
src/listfunc.c	92.21% <58.90%> (-5.17%)	⬇️
lib/coll.gi	94.67% <87.36%> (-1.54%)	⬇️
lib/coll.gd	95.46% <100.00%> (+0.13%)	⬆️
lib/memory.gi	43.12% <0.00%> (-41.75%)	⬇️
src/io.c	58.09% <0.00%> (-26.11%)	⬇️
lib/methsel.g	51.16% <0.00%> (-24.70%)	⬇️
lib/colorprompt.g	46.15% <0.00%> (-22.60%)	⬇️
src/finfield.h	83.05% <0.00%> (-13.03%)	⬇️
src/permutat_intern.hh	87.50% <0.00%> (-12.50%)	⬇️
... and 243 more

[coveralls]

Coverage decreased (-0.02%) to 84.715% when pulling 3bc8f92 on fingolfin:mh/fold into 791286a on gap-system:master.

[fingolfin]

Almost exactly 7 years after opening this PR, finally it is ready for review. Thanks to Codex for the final commit that finished the documentation, discovered a few edge cases in input validation, and added more tests. I hope this can get into GAP 4.16 :-)

[ThomasBreuer]

Nice.
Besides my few comments, perhaps a cross-reference from ListX to FoldLeftX could be added.
First I thought that manual examples would be nice but also SetX and SumX have no examples, they just refer to ListX.

[fingolfin]

@ThomasBreuer I hope I've addressed all your points.

[ThomasBreuer]

Yes (and more).
I think this pull request can be marked as ready for review.
If nobody else wants to comment, it can be merged.

[limakzi]

Nice job! ❤️