Incorrect each function signature validation

[sergeevik]

jsonata-java version 0.9.9

Example:
json: empty {}
jsonata expression: [] ~> $each(?, function($v) { $v })

expected result: empty
actual result: com.dashjoin.jsonata.JException: Argument "1" of Object "each" does not match Object signature

details from debug:
functionName: each
badSig: af
partialPattern: ^[om]?
badArgs:
0 - Utils@JList
1 - ParserSymbol

to reproduce the bug use simple test:
@Test public void testEach() { var expression = Jsonata.jsonata("[] ~> $each(?, function($v) { $v })"); expression.evaluate("{}"); }
On https://try.jsonata.org/ example work correct, return ** no match **

[sergeevik]

some debug, maybe i found place with difference
method partialApplyNativeFunction

when jsonata-js create function with empty body and add ast.body = native function, this signature is null and not verify later

in jsonata-java native function write inline. after parse ast it's have signate wich verify later and throw Exception

now bodyAST.body required Symbol class, but _native function is JFunction

[sergeevik]

@aeberhart fix it. local all test success. please check pr #80

[aeberhart]

Dear @sergeevik, thanks again for your bug report and PR. I think this is the cause of the issue:

In JSONata, there's a difference between anonymous and builtin functions. Builtin functions have a signature, whereas anonymous functions do not.

Ran this code in node to test:

async function f() {
	var jsonata = require('./jsonata');
	var expr = jsonata(EXPR);
	var res = await expr.evaluate();
	console.log(res);
}

f();

Output with EXPR: '$substring'
{
  _jsonata_function: true,
  implementation: [Function: substring],
  signature: { definition: '<s-nn?:s>', validate: [Function: validate] }
}

Output with EXPR: '$substring(?, 1)'
{
  _jsonata_lambda: true,
  input: undefined,
  environment: {
    bind: [Function: bind],
    lookup: [Function: lookup],
    timestamp: null,
    async: false,
    isParallelCall: false,
    global: { ancestry: [Array] }
  },
  arguments: [ { value: 'str', type: 'variable', position: 13 } ],
  body: [Function: substring]
}

In JSONata, a partial function creates an anonymous function, which explains that no error (Argument 1 of function "each" does not match function signature) is thrown.

In our code, we reuse the builtin function object when a partial function is created. Hence, the difference in that our code throws the error and JSONata JS does not.

I think this is the key discrepancy:

https://github.com/jsonata-js/jsonata/blob/ff36f0bd0f1aa4307662ffcc9f68abbba2f20915/src/jsonata.js#L1613

jsonata-java/src/main/java/com/dashjoin/jsonata/Jsonata.java

Line 1839 in 78ce14c

if (Functions.isLambda(proc)) {

I think we should fix the issue there. Thoughts?

[sergeevik]

@aeberhart There's another difference between JS and Java.
JS uses duck typing, and JS arrays have keys() and values() methods.

In Java, an array is an array, and it only has a list of values. In this regard, JS returns indices when calling Object.keys(). This means that an array in JS is a key-value object like any other object. This is not the case in Java. In Java, an array is an array, and a map (as a key-value mapping of a JS object) is a map.

The difference isn't just in the signature, but also in duck typing and the ability to treat a regular array as an associative array.

Therefore, in my opinion, my approach is more suitable, as it creates the same context: a Java array becomes associative, where the key is the index.

For example, check part of each method
https://github.com/jsonata-js/jsonata/blob/ff36f0bd0f1aa4307662ffcc9f68abbba2f20915/src/functions.js#L1814

const arr = ['test', 'data']
for (var key in arr) {
 console.log(key)
}

output is:
'0'
'1'

[aeberhart]

@sergeevik, our goal is to stay as close to the original as possible. In lines 1843-1844:

//  } else if (typeof proc === "function") {
//      result = partialApplyNativeFunction(proc, evaluatedArgs);

there's an obvious TODO which we missed. Pretty sure it's related to the issue you reported. Your fix keeps the signature and converts the list to a map (like JS duck typing), thereby satisfying the signature. But I don't think that's the right way to address it.

[sergeevik]

@aeberhart I'm not sure this is the right approach. In JavaScript, arrays and objects have a method for iterating over keys. Java doesn't have this capability, and this is a fundamental difference between the languages ​​themselves and their typing. I tried simply removing the signature check, but then the following problem arose: the 'each' method accepts a 'HashMap', and we're passing an array. This causes typing issues, and then the 'each' method would need to be changed to accept an object and automatically detect what was passed. But since this behavior is typical for all JavaScript, it seems correct to change the lambda function logic so that it passes the array as an object (in the case of Java, a 'HashMap'). Otherwise, the behavior would have to be adjusted in all methods (which is also an option, but more labor-intensive).

[sergeevik]

@aeberhart by discover of #83 maybe i found better place
Functions.java row ~ 2163

public static Object call

have comment

        // If the function receives the args as JList:
        // i.e. a varargs fn like zip can use this

maybe there write another if. If method recieves the args as Map wrap it

upd: bad idea, method "call" invoke after signature check

[sergeevik]

and another example

source:

[
  [],
  {
    "test": "data"
  }
]

jsonata: $ ~> $merge(?)

try.jsonata returns

{
  "test": "data"
}