Connection Sharing Sideeffects

Today Christian and I fought hard with a misterious connection problem. As a part of a longer process a web service call was failing from time to time. Actually it failed only under load. Due to that we did just debugging outputs, rather than breakpoints, as we were afraid of breaking some runtime behaviour by stopping the threads. But that was not so trivial as we would liked it to have: The webservice client is based upon the Apache Commons HTTPclient. The webserive itself was realized in Struts2 using its Restful2ActionMappers. Both got several services injected via Spring.

Initially we were after the unusual large size of the request and response, but after some analysis it seemed ok, however we still got flooded by variations of:

java.io.IOException: CRLF expected at end of chunk: 72/84
java.io.IOException: Bad chunk size: somexml

As we realized that this has to happen on the HTTP transport level, we had a look around and found following spring config for our bean:

<bean name="restClient" class="de.codecentric.framework.RestClient">
  <property name="httpClient">
    <bean class="org.apache.commons.httpclient.HttpClient" />
  </property>
</bean>

Doesn’t look bad, but by default the HttpClient has a ConnectionManager which will return always the same Connection instance. Even to different threads. Once you get to know this you will find immediately

org.apache.commons.httpclient.MultiThreadedHttpConnectionManager

which unfortunately did not help much. In addition we now go an

java.io.IOException: connection closed

What might suprise you is that MultiThreadedHttpConnectionManager is multithreading capable, but does only allow 2 connections per host. After we did increase that out application was running fine again. Even under load.

The Spring config does look similar to this now:

<bean name="restClient" class="de.codecentric.framework.RestClient">
  <property name="httpClient">
    <bean class="org.apache.commons.httpclient.HttpClient">
      <property name="httpConnectionManager">
        <bean class="org.apache.commons.httpclient.MultiThreadedHttpConnectionManager" destroy-method="shutdown">
          <property name="params">
            <bean class="org.apache.commons.httpclient.params.HttpConnectionManagerParams">
              <property name="defaultMaxConnectionsPerHost" value="20" />
            </bean>
          </property>
        </bean>
      </property>
    </bean>
  </property>
</bean>

Leasson learnt: When multiple threads are using the same UrlConnection, it can go fine, but more likely, some very obscure things (exceptions) can (will) happen :-)

  • Facebook
  • Delicious
  • Digg
  • StumbleUpon
  • Reddit
  • Blogger
  • LinkedIn
Fabian Lange

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>