FR: Support Serialize for template .map results

[mystor]

Is your feature request related to a problem? Please describe.
Currently when building complex templates, the .map operation on sequences results in a loss of type information in the enclosing context. The result of the map operation is required to implement Template ¹, and the returned value (a ListTemplate² rather than List³) only implements Template and the .join method.

The main place where comes up is in the json(...) function. This requires Serialize ⁴, meaning that you cannot write json(self.parents().map(|c| c.commit_id())), and instead need to manually build the json (e.g. with'[' ++ self.parents().map(|c| json(c.commit_id())).join(',') ++ ']').

This is likely to become more annoying with #6869, where an expression like json({ parents => self.parents().map(|c| c.commit_id()) }) requires the RHS to be Serialize,

The loss of type information also unfortunately prevents any operations like filter being chained after map. Some methods on List are implementable on ListTemplate (len() and Boolean conversions would be trivial, though questionably valuable).

Describe the solution you'd like
Ideally, lists would behave more uniformly, with the map operation returning a normal List object implementing the features one would expect based on the underlying type. Specifically I would expect the following:

1. All lists continue to convert to Boolean, and support the len, filter, map, any, and all methods.
2. If an element type T implements Serialize, List<T> should implement Serialize
3. If an element type T implements Template, List<T> should implement Template by concatenating the elements together with ' ' ⁵, as well as the join method.

Describe alternatives you've considered

1. As mentioned in the summary, the json function can be simulated by manually constructing the JSON output using templates with string concatenation. This is a bit tedious and will mesh poorly with the proposed implementation from templater: serialize map literal by json({..}) #6869.

2. A type-specific map_serialize method could require the template author to explicitly state how they plan to use the result. This reduces internal implementation complexity, but shifts the burden of type management onto the template author.

3. Rather than returning a "true" List<T>, the returned type could be expanded to being one of a set of types (e.g. ListTemplate (the current one), ListTemplateSerialize (for values which are both Template and Serialize), and ListSerialize (for values which are only Serialize)). Which type is returned depends on the inner type. These types would continue to not support the filter, any, map, and all methods. I expect this will be easier to implement than arbitrary list types (as it avoids adding generic types into the template language - fitting in easier with the current architecture), but is less extensible for potential future method additions.

Additional context
Add any other context or screenshots about the feature request here.

Footnotes

1. https://github.com/jj-vcs/jj/blob/7bcef373b12120e6d9efb6d126ab21bc9b6dc32f/cli/src/template_builder.rs#L1737-L1739 ↩

2. https://docs.jj-vcs.dev/latest/templates/#listtemplate-type ↩

3. https://docs.jj-vcs.dev/latest/templates/#list-type ↩

4. https://github.com/jj-vcs/jj/blob/7bcef373b12120e6d9efb6d126ab21bc9b6dc32f/cli/src/template_builder.rs#L1997 ↩

5. An exception would need to be made for List<Trailer> which already defines special methods, and concatenates elements with "\n" ↩

[yuja]

The underlying problem was explained in #5366.

Some workarounds that wouldn't require rewrites of the template engine:

1. allow serialization of Template/ListTemplate to String/[String]
   • works only for [String] type
   • may be source of errors because ListTemplate is also a Template
2. add .map_serialize(|item| -> Serialize) -> Serialize (suggested above)
3. add "map" syntax (like list comprehension) that would be evaluated to caller's expected type