Dieser Artikel ist auch auf Deutsch verfügbar
As a result of the tendency in new projects to do without heavyweight infrastructure for the integration of different services, in all my recent projects I have had at least one integration via HTTP, usually in connection with JSON. This raises the question which HTTP library we should be using for these integrations.
Not least due to the longevity of Java, there are now a wide range of options open to us. In this article we will therefore take a look at a selection of libraries. We will highlight differences and at the end hopefully be able to make a qualified choice for our specific use case.
The use case under consideration here is the reading out of members of a public list on Twitter. To do this we need to execute two HTTP requests. In the first request we receive an OAuth access token from the programming interface. For this purpose we communicate the static, secret API key and the API key secret known to us with the API by means of Basic-Auth. As a response we then receive an access token. We then need to send this with every further request as an HTTP header.
The second request serves the purpose of querying the members of the selected list. In order to keep the example simple, we only parse the JSON of the response and then display it on the default output. For the same reason we also do without pagination.
In order to be able to parse and use JSON, for example to read the access token, we use org.json implementation. But of course we could also use any other implementation. I will mention below whether the libraries support native JSON or do so by means of another implementation.
With the exception of the final example, the
main method from Listing 1 is used
for execution. This coordinates the two requests and at the end displays the result.
Already since Java 1.1 and therefore for almost 25 years Java itself provides
an implementation for HTTP requests in the form of
HttpsURLConnection for HTTPS connections was added to Java 1.4 in February
In order to generate such a connection, we have to start with a
URL. With this
we receive the reference to a connection by means of
contrast to what the name might imply, a proper connection has not yet actually
been made. This only occurs when we explicitly call up
connect on the
connection or call up a method, such as
getResponseCode, that implicitly
requires the request to be executed.
Before the actual request starts, we can use various methods to give additional
information such as the HTTP method or header to be used. If we also want to send
a request body, we have to give advance notice of this by means of
Then we can establish a connection via
connect. If we gave advance notice of
sending a request body, we can now write this using
we inspect the response code and read out the body of the HTTP response using
getInputStream and parse it in a
JSONObject. This can be seen as code in
Listing 2, including minimal logging of request and response.
From today’s perspective the code comes across as rather unwieldy. Instead of
headers the term request properties is used, and the implicit establishing
of the actual connection can lead to surprises, depending on the method used.
In addition, the nomenclature of
always confuses me as I always think from the perspective of the request, so
for me the body that I send is the input and the one I receive the output.
From the perspective of a stream this is however of course the other way round
and thus correct.
Nonetheless, this method allows us to implement our use case without any major problems, as can be seen in Listing 3. This method only covers the part required for our use case. For more complex scenarios it will certainly be necessary to write additional code. The biggest disadvantage however is that there is no support for HTTP/2. Although this may not be a deal-breaker at the moment, it is likely to develop into one.
Alongside the support of HTTP/2, the growth of non-blocking input and output and asynchronous programming was the driver for a new HTTP client in JDK. This was finally made available with Java 11 and is therefore the currently preferred option for HTTP requests in Java without the use of a third-party library.
As can be seen in Listing 4, this API is based above all on the well-known
Builder design pattern. In my opinion this creates compact, but still readable
and easily understandable code. In addition to the use of
send we could also
have switched to the asynchronous version with
I consider this version better than the one based on
However, there are still a few features missing for me. For one thing, there
are unfortunately no constants for the common HTTP headers and media types.
I therefore have to either write the concrete value each time, and frequently
look it up beforehand, or define constants myself in every project. In addition,
it would have been great to have had a preprepared
BodyPublisher for HTML
forms. This can however be added on if required, in addition to publishers
for other types. The final thing I’m missing is the possibility to register
globally with a
HttpClient filter or interceptors. Although this can be
done via an external library, it would have been
good to have included it directly in the JDK.
The fact that we have to Base64 encode and compile the Authorization
header for the first HTTP request is due to a quirk of the Twitter API.
Theoretically it is possible to resolve the authorization globally and
transparently using the
Authenticator class. However, the client sticks
very precisely to the specification and only uses this class once the server
has communicated by means of status code 401 that authorization is required.
In this case however Twitter replies with status code 403, thus preventing
the use of the authenticator.
It almost goes without saying that Apache Commons also offers an HTTP client. This hasn’t been around for quite as long as the original implementation in JDK, but is still over 20 years old. The current client has emerged from the Apache Commons project and is a self-contained solution within Apache known as HttpComponents.
This client also uses the Builder pattern in its programming interface,
above all in order to generate the desired request. A particular feature,
which can be seen in Listing 5, is that some of the generated objects, such
as the client itself or the response, can be closed after use by means of the
close function. Although using try-with-resources this
is easy to write.
For the validation of the status code, the applied media types, and the headers,
Apache HttpClient has many constants in the classes
HttpHeaders that make it easier for us to find the appropriate values. In
addition, as can be seen in Listing 5, there is the option of registering request
and response interceptors during the generation of a client.
Here too however we hit the problem of the Twitter API and Basic-Auth. In contrast to the new Java client however, here we are given the opportunity to configure preemptive authentication. This ensures that the client sends the Authorization header even if the server has not requested it.
Needless to say, Apache HttpClient now also supports HTTP/2 and also offers us greater configurability. For example, we can swap the way in which network sockets are created; the client supports the remembering of cookies and sends them with the next request, similar to the behavior of a browser; and the incorporation of a cache for responses is possible. There is also an asynchronous programming interface.
All three of the HTTP clients presented above have in common that they are based on a very traditional programming model. We generate a client, specify what type of request we want to execute, implement it, and then process the response.
Retrofit on the other hand decided to take a different path. Here we define the HTTP requests to be executed via an interface, and with additional annotations if required. How this looks for the two requests in our example can be seen in Listing 6.
The centerpiece here is our own
getAccessToken serve only to
organize all examples for these articles in the same way, and otherwise only
verify if the request was successful.
As we define the program using an interface and Retrofit dynamically generates
an implementation for us at runtime, in contrast to the previous examples here
we have to extend our
main method with the code in Listing 7. Here it can also
be seen that we register a
Converter. These converters are used to convert the
body of the HTTP response in the return types or method parameters into the body
of the HTTP requests or strings for HTTP headers, query parameters, or path
parameters. In our case the implementation presented in Listing 8 is relevant
in order to be able to handle
JSONObject as the return value. Through its own
submodules, Retrofit offers several off-the-shelf converters, which support above
all libraries with data binding, such as Jackson.
For the actual communication via HTTP, Retrofit uses OkHttp. This gives us the option of defining interceptors or undertaking further configurations, such as the use of HTTP proxies or response caches.
In this article we learned about four options for communicating via HTTP in Java projects – HttpURLConnection, java.net.http.HttpClient, Apache HttpClient, and Retrofit. In addition to these four there are many other candidates that we could use. For example, OkHttp, which is used by Retrofit, can also be used alone. Feign offers us a programming model similar to Retrofit, while larger frameworks, such as Spring, include an HTTP client.
The biggest differences between the options are found in the APIs: programmatic or declarative via interface and annotations. But whether the client supports an asynchronous programming model and whether we can define interceptors, for example for request and response logging, can also be relevant. We should also consider whether the client supports HTTP/2.
Personally I prefer the programmatic over the interface-based programming model as I have more direct control, and I generally try to use an existing client, without introducing a new dependency into a project.
The complete example code, including code for the clients not presented in detail here, can be found at https://github.com/mvitz/javaspektrum-http-clients.