MockMvcRequestConverter uses US_ASCII for URL decoding while encoding uses UTF-8

[config25]

Summary

MockMvcRequestConverter.decode() uses StandardCharsets.US_ASCII for URL decoding, while urlEncode() in the same class uses StandardCharsets.UTF_8. This causes non-ASCII characters (e.g., CJK, accented characters) in
query parameters to be corrupted.

This was likely introduced unintentionally in commit f5a629af ("Handle form and query parameters separately"), as QueryParameters.decode() created in the same commit correctly uses StandardCharsets.UTF_8.

Changes

• MockMvcRequestConverter.java: Changed US_ASCII → UTF_8 in decode() method
• MockMvcRequestConverterTests.java: Added two tests verifying non-ASCII parameter handling for both GET (query string) and POST (form URL encoded body)

Fixes gh-1033

[wilkinsona]

These tests don't exercise the changed code. The decode method is only used when the request has both a form URL encoded body and a query string. See postRequestWithParametersAndQueryStringCreatesFormUrlEncodedContentWithoutDuplication below. Instead of adding these two tests, that test could be modified with some UTF-8 values or a new test added to cover that case.

[config25]

Thank you for the review. You're right — the two added tests don't exercise the decode() method since it's only invoked when the request has both a query string and form parameters. I'll remove those two tests and add a new
one that covers the non-ASCII case with both a query string and parameters, similar to postRequestWithParametersAndQueryStringCreatesFormUrlEncodedContentWithoutDuplication. I'll update the PR shortly.

[wilkinsona]

Thanks for the updates. There are some formatting problems now (run ./gradlew format to fix those) and I think the new test also fails. Can you please update the PR to address these issues?

[config25]

Thanks for catching those issues! I've pushed an update that addresses both:

Formatting:
Ran ./gradlew format to fix the formatting problems.

Failing test:
The test was failing because the URL template "/foo?name=%ED%99%8D%EA%B8%B8%EB%8F%99" was being double-encoded.
MockMvcRequestBuilders.post(...) internally calls UriComponentsBuilder.encode(), which re-encodes % into %25.
As a result, request.getUri() became name=%25ED%2599%258D... instead of the expected name=%ED%99%8D....

I fixed it by passing raw (unencoded) characters in the URL template and letting Spring handle the encoding:

@Test
void postRequestWithNonAsciiParametersAndQueryStringCreatesFormUrlEncodedContentWithoutDuplication() {
    OperationRequest request = createOperationRequest(
        MockMvcRequestBuilders.post("/foo?name=\uD64D\uAE38\uB3D9")
            .param("name", "\uD64D\uAE38\uB3D9")
            .param("other", "\uD64D\uAE38\uB3D9")
    );

    assertThat(request.getUri())
        .isEqualTo(URI.create("http://localhost/foo?name=%ED%99%8D%EA%B8%B8%EB%8F%99"));

    assertThat(request.getMethod())
        .isEqualTo(HttpMethod.POST);

    assertThat(request.getContentAsString())
        .isEqualTo("other=%ED%99%8D%EA%B8%B8%EB%8F%99");

    assertThat(request.getHeaders().getContentType())
        .isEqualTo(MediaType.APPLICATION_FORM_URLENCODED);
}

This test now covers the case where both a query string and form parameters are present, which is the scenario where decoding and de-duplication are applied. It verifies that non-ASCII query parameter values are handled correctly and that duplicate parameters are removed from the form body.

Verified locally that:
./gradlew :spring-restdocs-mockmvc:checkFormat :spring-restdocs-mockmvc:test passes.

[wilkinsona]

Thanks very much, @config25.