Search This Blog

Friday, May 15, 2009

Don't whine about Java being slow

I came across the article Java is too slow! via The Server Side today. I think most Java programmers in the late 90's and early 00's have seen one Dick or the other in their career. I made a post there on why you can't whine about Java being slow and thought of expanding on that here.
I think Java got a bad rep mainly becuase of yuppie programmers (and yes, many from India with more interest in catching on to the next wave than learning fundamentals of programming) spewing out junk code. The big firms were no exception either. I vividly remember IBM's Visual Age for Java, the pre-cursor to the now popular Eclipse - a kludgy, super-slow IDE that made users cringe and forced them away from Swing and over to writing web apps for desktop systems.
I have implemented a number of highly transactional systems as well as GUI applications (yes, Swing) in Java that have run extremely fast.
The top reasons I have found for Java being slow are
  1. Lack of application of fundamentals of programming (such as taking care of the number of objects you create). Spring to some extent reduces this risk, if used correctly. The impact is the highest in Swing applications where layouts are nested too much just because someone didn't spend enough time to do the design properly. In some cases, even moving some unwanted "new"s out of loops will help things significantly.
  2. Not using the right software and instead reinventing the wheel. I have used MQ Series for a distributed system that happily processed around 100,000 records or more per hour without any complaints. The average transaction was around 7ms. And it was all written in Java.
  3. Not using threads optimally. One program that used the Java Advanced Imaging packaged processed and uploaded 50,000 images (each image around 200K) per hour easily when multiple threads were used (you have to of course check how many threads are optimal).
  4. Not using a profiling tool to check your code. In the glory days of Borland, I used their OptimizeIt software - an excellent tool to detect object leaks, something that's ignored due to Java's garbage collection.
  5. Finally, not tuning the JVM. After optimizing your code, you need to tune the GC so that it works for your application. Just leaving the defaults as they are (or just modifying the ms/mx values doesn't cut it). It's like expecting an auto-gear car to win in a drag race.
I am sure there are a number of other reasons I can come up with, but these 5 tend to be the most common. It looks like Java itself succumbed to the slowness by creating stupidities such as the original J2EE, where there was way too much architecture/framework and way little code. Thankfully, the open source community - with Spring, Hibernate, and others - seem to have brought some sanity back and hope it continues before another wave of stupidity (possibly int he form of overuse of annotations) sets in.

Thursday, May 14, 2009

Mapping your mind - or at least concepts

I was first introduced to the idea of mind maps when I saw the Java Concept Map. While it itself is not a mind map in the strictest sense, the idea is the same and made me explore this new way of capturing information.
Since then, I have heard rants and raves about mind maps - how crazy minded lunatics try to apply mind maps to everything on one hand and how procedure-oriented people write pages of documentation without a simpler visual representation. I think mind maps can be used and abused in a way similar to UML diagrams. While use case diagrams, class diagrams, and state charts can certainly be useful to represent the flow of a program nicely, over-use of the diagrams (such as strictest implementation of Model-Driven-Architecture or using Rational Architect) almost always ends in a disaster. I should know, I've lived through a few of them!
Back to the concept map, I love the use of typography there - big fonts for important concepts and decreasing font sizes for less important topics - kind of like tag clouds. Simple but effective.
Interestingly, I haven't seen any mind mapping software, including the most popular MindManager from MindJet, or the other free open source versions, such as XMind or FreeMind, provide this type of functionality out of the box. 
While no doubt these folks have done way more research in this area than I, my feeling is that the following two features that I have seen missing so far would be very useful.
  • Each topic should optionally have a definition that will be displayed in a smaller font right next to the topic. Having it as a tooltip is not good enough as it will not be visible in the diagram.
  • The connecting line should be 'describable'. In other words, I should be able to explain why two topics are related. This helps to read the relations in a meaningful sentence.
The closest that comes to achieving this is a nice piece of free software called Cmap Tools. The only downside I have seen in this one is that it by default is intended for a more collaborative environment than individual use.
Maybe it's time for me to reinvent the wheel! After all, that's how most open source projects start :)

Process of candidate selection

I recently attended an interview for a Technical Lead position. The interviewer (a very bright intellectual, especially if he's reading this blog!) asked me a series of questions around software development in general and specific questions around theory behind exceptions and threads. While I could answer most of the standard questions along with what I've been doing, I failed to impress him on the theoretical questions. Not that I did not know the answer or was not familiar with the concepts, but that it has been a while since I've read about them or used them. As a technical lead, combined with offshore development, the focus nowadays I believe has shifted more to design, process, and governance as opposed to hands-on development (which is where the offshore piece comes in). This made me wonder how we select people for a particular task.
It's interesting how your career or future depends on how you perform at a single instance regardless of how better or worse you've performed in the past. Our entire system seems to have been built around this concept - be it education, sports, politics, or job. Exams, match formats, and interviews all favor how alert you are on a given moment in time with respect ot the opposition compared to how you've been doing over time.
While I can understand the current rationale behind this (it's not feasible to keep track of everyone at all times to see how they do, especially those who you don't know, such as potential candidates for job), I think it's quite out-dated and incorrect.
Most US universities at least at the collegiate level use a combination of your final exam marks along with how you've done in all your assignments and interim exams, with varied weights. I fail to see why the same concept cannot be applied to other areas. Let me take the specific scenarios I mentioned above and provide some alternate examples.
Sports
This is probably the more controversial of the lot since it's not just the process, but also the thrill provided by the finish. Nevertheless, instead of providing a point win-lose scenario, why not have a weighted system, where, say, the number of wins up to finals accounts for, say, 40% of the final game, while the remaining 60% goes to the final game itself? Such a mechanism will provide a fair ground to a team that, say, has been winning for the whole tournament but ends up losing in the last match. In such a case, their past record will provide an additional boost for them to keep up the momentum to win the final piece.
Game Shows
Game shows are no different from sports in that they are a type of a sport, with similar format. A recent example which added to my thoughts is the latest season of the Biggest Loser, a game show in USA where the person who loses the most weight by the end of the season gets a grand prize. 
In the show, an ex-model Tara, won all the rounds handily including all additional challenges, but lost the final weigh-in by 5 pounds. I thought it was a bit unfair and sends a relatively wrong message, since her opponent seemed to have lost a little too much weight! With a weighted system, her past victories would have contributed to her final score and would have helped her, especially since she lost by such a small margin.
Politics
Another controversial topic I am sure, and definitely harder to implement, but think about this. Why not have a system where, in an election, the winner is selected not just by votes on how they did in the election process but also on how they performed during the previous tenure? While I am no political expert, I can think of some fairly reasonable metrics to measure the performance during a tenure such as the attendance of the politicians in the parliament procedures, the percentage of growth in development in various sectors, relative stability of the country - both economically and militarily, potentially calculated by level of threat, people confidence, GDP growth, and inflation rate. I am sure more metrics can be added.
Such a system would ensure that only a candidate, either ruling or opposition, who has really worked for his/her people is selected for another term, without being swayed by last minute sympathy waves or other external criteria. It also tends to be more meritocratic and will tend to keep the politicians on bay WHILE they are in power and not after.
Job
Finally, to something that made me write this blog. I think the evolution of Internet has provided some excellent opportunities to change the way we select candidates. Instead of deciding the worth of a candidate in a 30 minute interview, why not combine the responses with other factors such as what they have done outside their core skillset? This could include how active they are in social networking sites, what sites they are active on, what they have contributed, and so on. This is similar to valuing PhD candidates based on the articles they have written on conferences they've attended, for example. 
Such a weighted system would not only help a team lead pick the candidate with the right technical background (based on the interview) but also with the right passion, interest, and intellect, given that you need more than just technical smarts to execute a project.
Is this a gripe because the interviewer didn't consider my candidate worthy? No. I honestly believe that human nature is more complicated than a binary solution and that a weighted system that evaluates a person over time (what we would call experience) is a more accurate measure.

Better Documentation

One of the interesting things in IT I have seen so far is how people tend to learn less of what they use more. I call this Inverse law of IT learning. For example, we take training courses, read books, and participate in discussions about say, Weblogic or WebSphere or SAP or any other technology we tend to use sporadically in different projects. However, we barely tend to read a book or take a course about Microsoft Word or PowerPoint. We have come to see them equivalent to a DVD Player or a TV where we're just supposed to 'get' it without RTFM.

I don't think the analogy we developed applies much. Desktop publishing software such as the Microsoft Office Suite are much more complicated than Notepad for example. They are excellent tools to deliver quality documentation, if only we spend a little bit of time understanding them a bit more. I have seen experienced professionals spend minutes (which adds up significantly over time as we use Word almost every day) in cutting/pasting/selecting by fumbling with the mouse without knowing some basic keyboard shortcuts that can let them do the same in seconds.

We are more happy to crib about the 'user unfriendliness' of Microsoft Office than trying to spend a few minutes to understand the design and concepts involved, such as the Style library and Paragraphs.

I made a presentation (in the traditional death-by-bullets style) a while ago for my colleagues on how small tricks in Microsoft Office suite can significantly improve documentation that is produced daily by teams - be they defect reports, project status, or design documents.

While writing the presentation, I ended up reading Presentation Zen and Non-designers design book, which completely changed my perception on how to create presentations. You can see the final version in SlideShare. Please comment!

Blogging again!

Not that anyone cares, but it took me ages (as can be seen from my last post) to get back on track to start blogging again. I have lived through one of the biggest hurdles to blogging that one can go through - I am not doing anything interesting, so what exactly can I blog, and that too daily?!

It's a fairly easy state of mind to get into, especially if you are working constantly. I got a chance to take a breather and realized that I have quite a bit to blog actually. I read a lot, I surf a lot (and no, I don't spam-a-lot!). That's good fodder for the blog.

As a software consultant, I always end up getting another doubt. Consultants by profession, are supposed to get into the heart of the problem, understand it quickly, and provide a solution. In most cases, we have to pretend that we know all the answers to build customer's confidence and then, in the background, cram as much information as possible to be one-step ahead of the questioners. This poses a problem with blogging. Let's say I get into a new domain or technology. It's hard to blog in the process of learning for the fear of letting the customer, who may potentially stumble upon the blog, that I am learning. And once I am done learning, I move on to other things, and end up not blogging about what I learned. A kind of a catch-22 situation I guess.

Well, in any case, I have taken my keyboard up again and hope to put it to the page. Let's see how long it lasts this time.