Last week I had the pleasure attended Heinz Kabutz Java Specialists Master course to sharpen my Java skills. Java Champion Heinz, is a great trainer who manages to combine anecdotes, hard facts and deep Java knowledge with engaging exercises to a well done course. The scope was the whole spectrum of Java, but focusing on the details you normally do not use, or know how to use. Some of the material he already published as part of his newsletters, which are read all around the world.

Let me share my impressions on the course with you in this day by day review…

Day 1

The course started with discussing Threads, and how we should use them. Quite a complex topic for early morning. We played with a ThreadGroup that became a custom Thread pool. ThreadGroup is not the best designed class. Heinz called it a child’s drawing from the early years of Java. I found using the java.util.concurrent Locks really easy. Finally gone are the days of synchronized(). Before the lunch break, Heinz showed us his laws on concurrency, which he also presented in an abbreviated form at our meet the experts – performance event. We came across this code:

private boolean running = true;
public void dojob() {
  while(running) {
    // do something useful
 }
}
public void shutdown() {
  running = false;
}

When running in a Server VM (with java -server), this might never stop, due to optimizing running would be inlined by HotSpot with being true all the time. To avoid this you would have to make it volatile. Because I asked about debugging, we tried stopping there and the debugger showed us: running = false. Still the code continued to executed, because the debugger sees the correct field value, but the running code doesn’t. It gets more interesting with this code:

public void doJob() {
  boolean myRunning = running;
  while(running){
    // do something useful
    myRunning = running;
  }
}

When looking with the debugger we saw this:

running = false; myrunning = true;

however the loop still looped. But when forcing the line to execute via F7, code terminated. This can be a nightmare to debug, so it is good to know what you should take care of when writing multithreaded programs.

Also something to remember is to check

if (Thread.interrupted()) {
  throw new InterruptedException()
}

as first code in all methods declaring an InterruptedException.

We learned about CompletionService looks like an interesting interface for mass processing of asynchronous work. So, who needs closures?

Day 2

We started with Java new (yet another new?) IO, which brings quite a lot of new features, but somehow is not as widely used as it should be. One reason might be that it can easily get much more complicated to use. Perhaps one should try easy examples before writing an Async nonblocking server (which you can after attending the course ).

FileChannel fc = new RandomAccessFile("test.txt", "r").getChannel();
MappedByteBuffer buffer = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());

Second half was on understanding Java memory management. This of course is less often really applicable, but understanding it is quite critical to solving problems. We had a look at stuff like caching and pooling, and why this creates leaks or loitering objects. This reminds me of my older post about tag pooling in servlet containers. I never understood really why tags are pooled, and even worse, do not have proper lifecycle methods to detect this when using an IDE. We used jVisualVm and HPjMeter to watch the GC at work.

Day 3

On day 3, I learned some interesting inner mechanics of Collection classes I did not use before (like PriorityQueue), as well as some nasty tricks on classloading. Heinz explained java.lang.reflect.Proxies really well, and once understood, using them was not that difficult. Actually the best instruction is in the JavaDoc, but you must know how to read it:

Foo f = (Foo) Proxy.newProxyInstance(
		Foo.class.getClassLoader(), new Class[] { Foo.class },
		new InvocationHandler() {
		  public Object invoke(Object foo, Method method, Object[] arguments) throws Throwable {
		    return method.invoke(foo, arguments);
		  }
		});

In the afternoon we discussed about Exceptions, and I made up my mind on checked vs unchecked exceptions. Personally I will use unchecked exceptions for developer/programming errors. Catching them is not required, app may crash – Developers should fix this. However everything which is related to the Environment the app runs ins should work with checked exceptions. Ideally they provide sensible information, not just a message. Also quite important: Just rethrow Exceptions! Found yourself not able to decide what to do with an InterruptedException? Well just rethrow it And handle it in the Thread code (calling interrupted() and exiting a loop). I never did that often, because I don’t like polluting my method signature, but it should be considered. Do not be afraid of retrowing Exceptions.

Day 4

The last day of the course started with a tough performance optimization exercise. The tough part was that we were not allowed to improve the code until we had all the numbers written down and eliminated any overhead of testing code. I especially liked this, because it thought me how eager I am sometimes fixing issues that I forget proving them first. As kind of side note, we discussed the various modes the JVM can run in and found out how slow java -Xint is.  After having sped up the code down to 10% of its initial runtime, we moved on to Date and Time, which was a bit short chapter. I can recommend using jodatime and icu4j, and try to stay away from java.util.Date. Before the end of the course we covered logging including some nifty tricks. The most important lesson about logging is that you need to use code guards (which was not new to me, but I like the term, I never heard before):

if (log.isDebugEnabled()){
  log.debug(complexObject.toString() + expensive.toString());
}

Wrap-Up

I can wholeheartedly recommend this course. 4 days packed with lots of information and exercises that are well done so that they are a challenge for every participant. You should have worked with Java already some time. This is definitely not a beginners course. You will rush past topics which you can only grasp when you have experienced the problem before. I can also recommend taking this training in German, because Heinz has a really funny accent

Every player had to play one round of each of the games: Agricola, a pretty complex strategy game, Stoneage, a round based resource gathering game, Dominion, an interesting card game, in which players compile a custom card deck according to the available cards and their strategy, and Heckmeck am Bratwurmeck, a pretty simple dice-rolling game, which trains you calculating probabilities.

• First round of the Tournament, sporting Dominion went really well for our Team. 2 first places and a second one were great. Only one game was lost due to the high amount of curses played by witches.
• Agricola strained our nerves for over two hours. It was pretty intense, and we were unable to oppose our co-players, who were well prepared. We tried both, the farmer and rancher strategy, but only achieved a second, 2 third and a fourth place.
• After a short snack-break, we had some relaxing dice rolling activities with Heckmeck am Bratwurmeck. But it seems gambling is not our profession. 3 times a fourth place was really sub-par. The other third place could not compensate those losses.
• Last game of the day was Stoneage. 3 of us tried the “starvation” strategy, which was working fairly well, but only brought us 2 third and a fourth place. The one victory was done with some more mainstream farming and building strategy. Overheard discussions tought us that most players already have overcome “starvation” and came up with something better.

Despite the not very successful results, we had a fun day! And lots of motivation for the coming year. Perhaps we should then run a board game training bootcamp?

I have pretty high expectations, as I already read his newsletter since quite some time and also my impressions of Heinz, when he gave his talk at our meet the experts, were very positive. He really knows all the nasty details about Java, and I hope I can get some of those out of his and into my brain. The course promises “Extreme Java”. I am looking forward to very esotheric subtleties without any practical value. And of course to many caveates that you indeed would find in everyday work.

I am also looking forward to meeting other Java geeks to exchange knowledge and have a good week. Will you be there?

(note that the course is in German)

January

2009 was kicked off by moving to our beautiful premises at the old courthouse in Solingen Ohligs. We left behind the old offices in the Gründer und Technologiezentrum where codecentric dwelt well for 4 years.

February

Our partner dynatrace awarded us with the “Partner of the Year” award for our strong relationship and making most dynatrace sales. Partnership with dynatrace has always been of great importance for both parties. Also during February our team “Dr. codecentric und seine kranken Pfleger” missed the finals of the German Board Game Championship by just one position. We try better this year

April

Dr. Heinz Kabutz offers his excellent Java Specialist Master course in Germany exlusively on the codecentric webpage.

June

Our first “Meet The Experts” took place. The topic “Performance” is strongly tied to the roots of codecentric and presented by first class experts. The success of the event was overwhelming proving our ideas more than correct. Its really great to have experts in talking range, rather than on just listening to them as you can do on big conferences. Meet the Experts was continued with “Agility” and “Architecture” events and will be continued 2010.

July

“codecentric coding night” was the name of our experiment. Many codecentricer camped at the office for the weekend and produced a time tracking system using Spring Web MVC as technology and a lightweight Scrum as process. On Sunday afternoon, when the experiment ended, we knew that we learned a lot on technology and process, but unforunately did not finish our product as planned. But we will for sure start another coding night soon… perhaps using Grails?

November

codecentric won the annual Start Award for being the most successful startup. Fantastic!  There was a camera team in our offices a few weeks before the final round, and they produced a nice promotional clip that I would like to share with you.

Have a happy and successful year 2010!

The second topic is much more interesting: Ok, we have Scrum, but how does this affect our organization? Do we need different management levels? Do we need different payment models? What motivates people to work in a SCRUM environment?
My colleague Andreas talked about the science behind motivation and why SCRUM is really good at motivating people. I can really recommend his talk, so I included it below from slideshare. Also Christiane Philipps gave some interesting insights. I think these ideas need to be explored further, because SCRUM reveals people issues which traditional models try to suppress.

However the event as such was a bit disappointing. It felt like the main audience were the speakers presenting each other and to the exhibitors. Not that many “regular guests”. Time between sessions was spot on, but the sessions itself felt a bit short with only 45 minutes. I think 60 would have been better.
I personally liked more our meet the experts instead, even though (or perhaps because?) there is just one track, excellent speakers and a much more familiar atmosphere in our premises rather than the anonymous hotel lobby, which is not that suited for having interesting discussions.

As promised here Andreas slides. All credits for them please attribute to him

• OSGi is a great module concept that is missing in pure Java. It is somewhat sad that the average enterprise developer will not be able to use it for a while. The main problem seems to be the migration and interoperability of existing non OSGi code, and the changed runtime environment, which is new for the standard operations department.
• You shall make many small modules for your own software. There is a benefit in having them although it cost slightly more work to manage them (for example in eclipse).
• It is mandatory to have somebody watching over external dependencies.
• Many third party dependencies cause various troubles: Classloading, legal, filesize, deployment time, runtime memory
• Maven or Ivy do not solve the problem but they help a lot
• Evaluate the usage of third party code on a regular basis. Consider removing dependencies that are only used for very few code

Thanks to all attendees. Over twenty people joining the discussion shows that there is a need for good ideas.

Caused by: javax.xml.stream.XMLStreamException: java.io.UnsupportedEncodingException: "utf-8"
	at com.ctc.wstx.stax.WstxOutputFactory.createSW(WstxOutputFactory.java:257)
	at com.ctc.wstx.stax.WstxOutputFactory.createXMLStreamWriter(WstxOutputFactory.java:124)
	at org.apache.cxf.interceptor.StaxOutInterceptor.handleMessage(StaxOutInterceptor.java:67)
	... 25 more
Caused by: java.io.UnsupportedEncodingException: "utf-8"
	at sun.nio.cs.StreamEncoder.forOutputStreamWriter(StreamEncoder.java:42)
	at java.io.OutputStreamWriter.<init>(OutputStreamWriter.java:83)
	at com.ctc.wstx.stax.WstxOutputFactory.createSW(WstxOutputFactory.java:253)
	... 27 more

That looks strange, because utf-8 should be a supported encoding. But a closer look reveals that the encoding passed contains the quotes. And that does not work. According to the specification quotes are allowed around the charset, but that is not very common.
The quotes were introduced in JAX-WS 2.1.2.

luckily the problem is easy to patch when you know this:

if (encoding != null && encoding.startsWith("\"") && encoding.endsWith("\"")) {
  encoding = encoding.substring(1, encoding.length() - 1);
}

insert this code at two places:

org.apache.cxf.interceptor.StaxInInterceptor#handleMessage()
after
String encoding = (String)message.get(Message.ENCODING);
 
and
org.apache.cxf.interceptor.StaxInInterceptor#getEncoding()
before
return encoding;

The patched class needs to be on the classpath before the original one. When doing so, the webservices worked again. A small unit test verifies this behavior and ensures that after a potential Mule/CXF upgrade the quoted encodings work.

Closures are tricky to explain. I am not going to try it. But it is basically a piece of code that gets its own life. Closures can reference state that is no longer existing outside their scope.

Closures are neat. no doubt. I like programming with them in JavaScript. And I was pretty much attracted by Clojure when Howard Lewis Ship demoed it yesterday in his session. When talking about languages like Clojure it is crystal clear that this is a new language; it comes with its very own concepts you need to understand before you can develop. Knowing how to develop in OO Java does not help for developing in Clojure, even though it has interop with Java. I do wonder where Java is heading with adding closures. The average Java developer does not understand it, or even when he does, has no need for it. You need a different mental model to successfully implement this.

This new model is map&reduce. Well, ok, map and reduce are not the mental model behind Clojure, but understanding it allows developing in Java and using closures efficiently. Map&reduce was almost in every session, and it seems to be the answer to the scaling question. It simplifies a lot, because it only works when you simplify. To quote Gregor Hohpe: “if we would have rebuilt a relational database, just to make it faster, we would have failed”. This is so true. The inherent complexity of an relational model / database does not get any faster. If it would be possible, Oracle certainly would have done this already.

So we have to simplify, but how? Treat data as key value pairs, like in the good old Cobol times. Dump relations, what makes them important? They just cause troubles.

When you have made it to this with you mental transition you are ready to go with any language. With or without closures. you can do map and reduce and paralleliz as you like. The link between those is that you accept to handle one item at a time, you don’t have global state or side effects, and you invoke your closures as workers on data which return results.

Gregor showed a few patterns they apply at Google. I especially liked Sawzall, which seems not to be open sourced by Google yet. It is a nice descriptive and concise calculation instruction per data item. Its actually pretty much a single closure:

errorcount table sum of int
--------
if contains input 'error'
  emit errorcount <- 1

It is basically a call to think outside the box. We cannot solve problems without, because we cannot improve existing solutions anymore. They are highly optimized. We need something new to solve our challenges.

Already during the fist keynote, #devoxx grew to a trending topic on twitter. Partially due to the still working wifi, which is withstanding the enormous activity quite well.

Antwerp and the Devoxx crew offer a great sideshow as well. We have our traditional steak dinner, Belgium beer and the sponsored  2012 movie here at the Metropolis cinema, where also all the sessions and the exhibition take place.

I am pretty sure we will have one or another post on interesting sessions or topics as well.