A Guide to Apache HttpClient

How to Migrate Your Codebase from Google’s HttpClient to Apache’s HttpClient

Creative Cloud Enterprise subscribers use the HTTP protocol to communicate with third-party agents external and internal to Adobe Servers. Migrating the codebase from Google’s HttpClient to Apache’s enhances the overall flexibility, speed and performance.

This blog post is designed for backend developers who have some coding experience and want to migrate their codebase from Google’s HttpClient to Apache’s HttpClient to enhance the overall flexibility, speed and performance while communicating with third-party agents external and internal to Adobe Servers.

By the end of this guide, you will understand the client better, and the implementation steps will help you implement client and requests on your own. This is a complete guide to Apache HTTP requests.

When we think about fetching data on the backend side of any application, be it on desktop or mobile, web requests are the most frequently used solution. Of the many types of web requests like TCP, IP, UDP etc protocols, the most frequently used method is HTTP. Creative Cloud Enterprise customers use the HTTP protocol to communicate with third-party agents external and internal to Adobe Servers. For example, it used to fetch policy service data from FFC (Fulfillment Center). The web requests are so popular that the HTTP clients had to become more flexible. HTTP clients should be stateless in order to consider each request as a new request, establish reliable TCP connections, and allow web servers and browsers to exchange data over the web with greater ease and increased speed.

The Core of Web Requests

Hypertext Transfer Protocol (HTTP) is a popular protocol used on the internet.

An HTTP client sends an HTTP request to a server in the form of a request message which includes the following format:

• A request line which starts with a method token, followed by the Request-URI and the protocol version, and ending with CRLF (Carriage Return and Line Feed)
• Zero or more headers fields followed by CRLF
• An empty line with nothing preceding CRLF indicating the end of header fields
• Optional message body

For an example of what request messages look like, check out this article.

Web services, network-enabled applications, and the growth of network computing continue to expand the role of the HTTP protocol and increase the number of applications that require HTTP support.

However, in the context of the Adobe Admin Console’s HTTP Client, usage is limited to user-driven web browsers.

Types of WebRequests

HTTP requests can be broken down into various types depending on the method to be performed on the resource identified by the given request URI (Uniform Resource Identifier).

• GET: The GET method is used to retrieve information from a server using a given URI. Requests using GET should only retrieve data and should have no other effect on the data.
• HEAD: Same as GET, but it transfers the status line and the header section only.
• POST: A POST request is used to send data to the server, for example, customer information, file uploads, etc., using HTML forms.
• PUT: Replaces all the current representations of the target resource with the uploaded content.
• DELETE: Removes all the current representations of the target resource given by URI.
• CONNECT: Establishes a tunnel to the server identified by a given URI.
• OPTIONS: Describe the communication options for the target resource.
• TRACE: Performs a message loop-back test along with the path to the target resource.

Apache HTTP Client vs Google HTTP Client

Looking at the project complexity and costs overhead of the Google HttpClient we were using at Adobe, we decided to migrate the enterprise-side code to Apache HttpClient. The Apache HttpClient provides complete flexibility of functionality many applications need by providing an efficient, up-to-date, and feature-rich package implementing the client side of most recent HTTP standards and recommendations. One of the biggest reasons for this migration was that Google’s HTTP Client internally uses Apache HTTP Client, thereby introducing overhead as an extra layer; thus, just using Apache’s Client would be a lot faster!

Implementation Details

Well! We went ahead with the project and my job was to migrate the library. When I was migrating the HTTP Client library, I found the documentation pretty confusing and few resources to do it the easy peasy way. That’s when I decided to write this article!

Here are simple steps to implement Apache’s HTTP Client:

1. Add a Maven Dependency: To use this library, add org.apache.httpcomponents dependency to the pom.xml file:

The latest version can be found at Apache HTTPClient.

2. Making a Simple Get Request: The user is expected to provide a request object to execute, and the HTTPClient will transmit the request to the target server and return the corresponding response or throw an exception in case of failure:

Similarly,, you can make put request (using HttpPut), post request (using HttpPost) and delete request using (using HttpDelete)

3. Adding Custom Headers to Web Request: The addHeader command can easily add the header key-value pairs as shown in the example below:

4. Adding the Request body to a Web Request: The setEntity command can add the request payload/body to the request as shown in the example below:

5. Adding query parameters: The query params can be added to the list, and this list can then be passed as a parameter to Request URI as shown in the example below:

6. Adding Request Config: The request config builder is used to set the connection and sockets’ timeout; in enterprise flow context, one second was sufficient, but it varies from case to case.

Migration Scheme

Web requests are required when writing code at the industry level, so it becomes necessary to keep the code reusable, robust, and easy to understand. During the migration from Google Httpclient to Apache Httpclient, we followed a scheme to ensure code reusability and easy writing, reading and understanding of the code at a granular level. Even if changes/additions of new features occur in future, the code written should be robust enough to adapt to the changes scenarios. Here are a few techniques that I used while implementing Apache’s HTTP Client:

For the sake of reusability, you can initialize an instance of HTTP Client once and use it while making web requests like get, put and post. However, in case of a failure, you may wish to retry the same request. Adding retry implementation like this will make the server invariably execute the same transaction more than once:

Above, requestDTOis the Data Transfer Object having 3,5,..times the server has to retry. The waitPeroid is the time (in seconds) after which the server makes a retry web request. The waitPeriod increases exponentially (by a factor of two) as the retry count increases.

Please note that the network retry-able Http Status Codes (408, 429, 500, 503, 504) are a set of requests where we must retry in case of failure.

2. Due to the wide variety of parameters to be used while making web requests, it’s essential to make sure that the code is easy to write, read, and understand. The builder pattern’s setter method returns the Builder object such that it can be chained. For example:

In this example, the build method could be modified to check correctly injected parameters and throw an IllegalStateException if an invalid parameter value has been supplied. Because the object is created over one call, it won’t lead to an inconsistent state partway through its construction and ensures thread safety. This pattern is flexible, it’s easy to add more parameters to it in the future, and it is useful if you are going to have more than four or five parameters for a constructor.

Conclusion

In conclusion, we were able to migrate the code from Google’s HttpClient to Apache’s HttpClient and enhance overall flexibility, speed and performance while communicating with third-party agents external and internal to Adobe Servers. Through this article, anyone who wishes to make the same migration can easily do it by following the steps and code snippets mentioned above. To learn more about Apache’s HttpClient, you can check out this link.