API Call always get 400 Bad Request response after upgrading Spring Boot to 2.7.x
In the past few weeks, my team met a critical issue. Our service ABC just upgraded the spring boot version from 2.2.x to 2.7.x. After testing and deployment to the production environment, we found that all tenants are working fine except one tenant Tom.
When we tried to call the APIs from ABC in the browser, the service ABC always returned the 400 Bad Request response.
We are so confused by this situation:
- If the service ABC has a problem, why do other tenants have no such issue?
- If the service ABC has no problem, why does after deployment only Tenant Tom have such an issue?
1. Analysis
1 |
logging_level_org_apache_http: TRACE |
1 |
http-outgoing-9 << "HTTP/1.1 400 Bad Request[\r][\n]" |
Then we downgraded the log level for undertow which is running in the container as an embedded server.
1 |
logging_level_io_undertow_request_io: TRACE |
1 |
io.undertow.request.io: UT005014: Failed to parse request |
2. Root Cause
Finally, we have found the solution and the root cause. In the spring boot version 2.7.x, the Undertow is used in the container instead of the Jetty.
The default Max-HTTP-Header-Size as described in the Spring document is 8KB, for both Jetty and Undertow.
But other resources like baeldung and stackoverlow said the default value for Undertow is 1MB —– We have proved that this is incorrect!
BUT, for some reason, the older spring boot version DOES NOT limit the request header size to 8KB.
And unfortunately, in the tenant Tom, the JWT token with Keycloak roles in the header of the API requests is large than 8KB. We have a huge role list in the JWT token which is required in our services.
That’s the reason why ONLY tenant Tom and only service ABC which are using the new Spring Boot have a such weird issue.
3. The final solution:
Please set the following config in your application.yml file. You can set your preferred value.
1 |
server.max-http-header-size=100KB |
4. Open points
why jetty in the old spring boot version does not limit requests with 8kb header? Even the spring document said it is 8kb. Maybe the jetty-to-jetty communication will ignore the header limitation?
Refer: