Mobile companies and their analogous countries

Sometimes when a problem is too big or too abstract to grasp, we tend to equate it to something we can grasp better. We call it an analogy. Dictionary.com gives this definition:

A similarity between like features of two things, on which a comparison may be based: the analogy between the heart and a pump.

There is also a different application to an analogy. You can use it to compare a new concept to a mature concept to get a sense of how the new idea might mature in future. While this of course is not perfect, it is a reasonable approximation.

Recently, I came across an article in Engadget called Is Android fragmented or is this the new rate of innovation?

The article led me to think beyond just Android and more about the mobile market in general, and even the companies that are responsible for the primary mobile platforms. I saw some interesting similarities between these platforms and how they are modeled and some of the leading countries in the world and how they operate. I felt it was a reasonable assumption as, after all, we are seeing the mobile platform spawn the world effortlessly. So, here's my analogy of the mobile platforms (and the companies) to some of the countries of the world.

iPhone / Apple : China
The first one of course, is the most popular mobile platform at the moment, the iPhone. 

The success of iPhone as come about due to the following factors.

• Hundreds of applications at a relatively cheap price point
• One single platform and phone model (with minor variations)
• Tight grip over the user and developer base
• Tight control / censorship over how information is disseminated over its platform

While you might argue that its design innovation is what made it the leader in the space, I would argue that the factors above are what has made it sustainable beyond the initial shock and awe. As you can see, the tenets are not too dissimilar from how China operates as well.

While China is the envy of the world and is fast gaining economic power, there is also resentment about the practices it has used to get there. The point of this blog is not to argue whether such practices are ethical or not - it is up to the larger population to figure out. Rather, to me, what is striking is the similarity in itself.

Interestingly, like China, while people grudgingly complain about the non-openness of the platform and authoritarian practices, they nevertheless go ahead and buy iPhones. 

Android / Google : USA
Google is fast becoming a major competitor in the mobile space. Already a world leader in the search space, the Android OS has positioned Google well to compete in the mobile area, which to Google, is another feed for its search domination.

Android (and hence, Google) has the following features.

• It is based on a 'relatively' open platform 
• Everyone is welcome to develop in the modified Java system, but the platform retains control over its future
• It has an 'open' market although with a slant towards it's own powers (search)
• It attempts to corral the rest of the industry towards standardization
• The company promises to do 'no evil' (but may not be acting on it or perceived to be acting on it)

A lot of the fundamental principles to me seem to resonate with how the US Government operates. While in general, it tries to do more good than harm, there is general skepticism that it maybe doing it the other way around. People are cautiously optimistic about the new platform.

RIM / Blackberry : Canada
Research In Motion's Blackberry OS has become the de facto standard in business mobile usage. In that aspect, maybe it is not that similar to Canada, however there do seem to be some similarities.

• The platform is loosely based on a more standardized platform (J2ME)
• Even though the platform is quite open, it is still not as popular as it was expected to be
• It is happy to be in its own space, without trying to influence others

I am always surprised at how the platform has really not tried to expand to other markets, even after other players such as iPhone have come in and at the same time also maintain its hold on what it does best. May be they follow the words of  Jim Collins in his book Good to Great to the dot, which is not to say that it's a bad thing.

Microsoft / Windows Mobile : European Union
The article I had mentioned earlier differentiates Windows and Android in the way they see the device and software partners. Since the beginning, Microsoft has always thrived on a license-and-partner relationship, which has worked to its advantage over the years.

Windows Platform has the following characteristics.

• It works on a partnership model. Everyone pays a license to develop on the platform
• The platform heeds to the needs of the partners and waits till everyone is on board
• The overall fee structure is more expensive compared to other similar platforms
• Change is hard to make and it takes a while to happen as well

It's also ironic that Windows platform is similar to EU, given its recent problems with EU! However, the partner model (and problems thereof) does seem to resonate well.

Oracle / J2ME : India
Finally, we have the Operating System that most platforms other than those mentioned above, are based on - the Java Micro Edition. I would potentially put Symbian in the same category as well.

Let us look at the defining characteristics and the similarities.

• It's the most open platform compared to all others
• It tries to please everyone and sometimes ends up pleasing no one
• By trying to find a common minimum that will satisfy all devices, it tends to fall short of user expectations, leading users to create their own variant platforms
• Being a 'democratic' system, it has little control over the devices where it is used, leading to significant fragmentation, which in turn, compounds the problem of trying to find a common minimum.
• Even though it is a credible and viable platform, it is outdated and is fast losing its market relevance

To me, the J2ME platform's similarity to India is probably as striking as Apple's similarity to China. Interestingly enough, while both are hailed as leaders in their own way (largest democracy vs economic super power), they have their own differences.

While it may take a while for the countries to play out their roles, may be it will be interesting to see how the market reaction is to these mobile platforms, which in turn, may give a good indication of how the world will shape up - and I am not sure if the results we may see in future is something we may like to see ;)

Top 5 Missed Opportunities for Java

Last year around the time it was becoming evident that Sun was up for acquisition, I had blogged about how Sun could have made money off Java. Given the recent developments of Oracle's acquisition of Sun as well as a number of Java luminaries leaving their long-held posts (including James Gosling, the creator), I wanted to expand on the post a bit more and list the top 5 missed opportunities for Java and leave you at the end with a bit of hope.

Opportunity #5: Enhanced JCP Mechanism
A lot of bigwigs in the industry have talked about the Java Community Process (JCP) - where it falls short and what can be done to improve it. Not having been an active member of the JCP, I cannot comment much more than what has been said already. However, as an active user of Java, the happenings of JCP does impact me.

Personally, I feel that the idea of JCP is great. You have a coordinated steering committee with participation from multiple corporation all having a vested interest in making Java a success. It is reasonably democratic. However, that seems to have given rise to JCP's problems as well. There are two fundamental issues in a democracy, as can be understood from India and USA, the two leading democratic countries in the world.

1. Too many voices can cause cacophony: When the participants are numerous, nothing useful gets done. The Indian democracy is a prime example of this. Those who are supposed to make decisions tend to squabble over petty issues or short-term issues rather than focus on the greater good. 
2. With power comes pork: As can be evinced from the US democracy, great power leads to greater demands from vested interests ('pork' projects). While the participation of major corporations has been great in boosting Java's position in the Enterprise, it has also led to them seeking customizations that can give them an edge against the competitors, who are also part of the process.

Thus, instead of making Java better, the JCP seems to have given rise to more and more politics. I can think of one layman option to fix this.

Instead of Oracle (or Sun) leading the JCP efforts, it must be floated as a separate non-profit organization. Each corporation must pay to be a part of the group. Final decision on a specification must be in a time-bound manner and must be resolved by a common vote. There must also be checks and balances to ensure that things don't get stalled. More importantly, the process must be agile enough to adapt to changes in the industry quickly. I know that most of these were the original goals of the industry, but it helps to revisit the initial goals and see what failed along the way.

Such a non-profit group can also lead to better Intellectual Property management, which seems to be the primary contention among a number of folks.

It is sad to see how the Java ecosystem is going the same way as the multi-party Indian democracy. When someone (Google) does not like the way things are, they form a splinter party (Android). This leads to an erosion of the core base. Now, when time comes to do something useful, those in power are more interested in getting a big enough coalition to retain power than to do anything useful. I hope this is not how the JCP ends up.

Opportunity #4: Desktop Java
When I started working on Java, one of my first projects was on Swing. Having had a bad taste with Visual C++ and Windows apps (it just seemed way too complex), I was excited to see the ease with which I could develop desktop applications with Swing (using Borland JBuilder, which was the best at that time).

At one time, Java came really close to creating its own Operating System and there were talks about a Java OS. However, nothing much came out of it.

In addition, there was another cool innovation within Java that was never utilized very well, and that is the Java Web Start technology. Java Web Start was the grandfather of the Apple App Store. All the pieces were there, but no one really did anything with it. When Java Store came recently, it was too little, too late.

Java could have very easily leveraged on the core components of Swing and Java Web Start and could have developed an ecosystem like App Store for the desktop. The widgets that we see on Windows 7 today could've been delivered with Java Web Start.

Google OS to me is a fancy version of what could have been Java OS. Unfortunately, this did not, and still has not, materialized.

Whenever I see Ubuntu's Software Center, I see the power of Java Web Start and what it could have been, and it saddens me.

Opportunity #3: Applets
I have been involved in Java pretty much since I started getting seriously into the world of computers. My very first encounter with Java was around 1996 when my friend told me that there is something really cool called Applets that could run applications within a browser. It took me a while for me to grasp the concept - how can you run an app inside a static web browser? It would be another two years before I fully understood the technology behind the whole system.

Applets were a great missed opportunity. When applets came into the picture, it was like Jetsons. People imagined all sorts of dynamic web that would be possible in the future due to the technology. Eventually that has been materialized with AJAX and Silverlight and Flash, but it sad to see how the technology that started it all never went anywhere. Java FX again, was too little, too late.

I still think the potential is there, but it would require a significant overhaul before it gets anywhere.

Opportunity #2: Meta-libraries
As I mentioned in my previous blog, I think a huge area where Sun failed to realize the potential of Java was in commercializing it. I don't think the folks at Sun never fully grasped the concept of 'style over substance' - something that Apple has shown all of us since then with iPhone and iPad.

While it was important to focus on the mechanics of various functions, that is never enough. You need a great looking skin on top to sell. While JCP and Java focused on standardization and developing a great API, they never focused on making it user-friendly. It would take Spring and Hibernate frameworks to show the need for the skin. Till then, one had to write a bunch of lines of code just to send an e-mail or create a simple application.

Microsoft .NET still regained its popularity even though it had a lot less functionality than Java primarily because of this reason - there were so many wizards that could be leveraged by a developer to develop an application quickly.

All Sun (and others) needed to do was to create an abstract layer on top of the various APIs that would allow developers to be productive from the start - and what's more, they could've charged companies for it! I still fail to understand why that never happened.

Opportunity #1: Mobile Java
Finally, the biggest missed opportunity of all - the mobile space. After all, Java started as a language for non-desktop systems, such as set-top boxes and toaster ovens. With such a strong foundation and need for atypical systems, Java meandered and never delivered on the promise it started out with - to create a platform for resource-sensitive systems.

Having written a book on this subject, I can understand the rationale. Java wanted to be the lowest common denominator - to be something for everyone. But I don't think they went about doing it right.

One of the reason why Apple is successful is because they have a tight control on both the device as well as the API. They have 2 sizes (iPhone and iPad), compared to a million combinations that Java has to satisfy.

I feel the foundations of MIDP are strong - it is modular (profile-based). However, it is not agile. They could have expanded the concept and created 'feature packs' that would satisfy different types of inputs and screens, with a base foundation. This way, the developer would simply create a base application, and additional feature packs that would be applied based on the device (the download mechanism can determine the packs to download and install based on the need). Such a mechanism could have led to multiple manufacturers rallying behind a single API instead of each developing their own kit (such as Nokia, RIM, and Sony currently do).

It is simply unpardonable that a language that started out as the champion of all things that are not desktop or web failed to finally meet the dynamic demands of its target market.

At the end, I don't think the end is near yet. The foundation of Java is strong. There is a strong user base that can rally behind it when a call is given. All it needs is a good leader who has a long-term vision - I just can't help but see the parallels between Java and the struggles that the two major democracies (US and India) seem to be having. I hope that sanity prevails!

How Sun could have made money off Java

I started out my career as a Java developer. While I have branched out to Microsoft SharePoint recently, I still do a lot of architecture and development in Java and consider it to be quite handy.
I have always wondered why Sun could not make money off Java after having come up with such a cool and now ubiquitous software. True, they tried their luck with optimizing their servers for Java and so on, but none ever stuck and they have ultimately got themselves acquired by Oracle.
Recently, I started working on a project that essentially is a desktop application (Swing components) with Webstart as the deployment mechanism. I have used JGoodies intermittently for creating rich layouts before and got a chance to use it more thoroughly this time around. As a quick aside, JGoodies is an extremely well-written library and makes your screens look a lot nicer. Combined with a professional tool like JFormDesigner for visually editing the screens, I was able to cut down my design time from roughly 2 hours to 10 minutes!
While I was developing the application, I was able to think of two ways in which Sun could have really leveraged its expertise and thought leadership around Java without irking the open-source community that it has come to rely on so much.
Meta-libraries
First way in which Sun could've made quite a bit of money is by developing a set of higher-level libraries and selling them commercially at a moderate price. For example, developing database applications and managing objects in Java was cumbersome at best until Hibernate and Spring frameworks came into the picture. Sun could've come up with such high-level libraries on top of the core language and sold them commercially. Even now, Swing is relatively bare bones in nature and you have to do a lot of plumbing before anything meaningful is accomplished.
Ideally, if Sun had created such meta-libraries around various scenarios (such as database, desktop, web, etc.) and had developer and commercial licenses (say, $10-$20 for developers and $100-$200 for enterprise deployment), it could have formed a great revenue stream. Optionally they could have milked some more money for any major upgrades (say, from Java 5 to Java 6). I think these price-points would've been more than acceptable for both developers and companies. The core libraries and JVM themselves would still be free and available for all those who want to start from scratch.
Marketing via App Store
Recently Sun released the Java App Store taking cue from Apple and after sitting idle for almost a decade. The App Store is based on the Java WebStart technology, a core component of the Java Runtime Environment that has been around pretty much since the beginning (around 1999, I believe). WebStart is probably the most under-appreciated concept within Java, thanks in part due to the Internet boom that happened soon after its release.
Essentially, WebStart allows you to deploy Java applications via Internet on to your system without needing any .EXE files. Better, applications deployed via WebStart are Java applications that are handled and run by the JRE in the client computer and hence, can be run on almost all platforms that have JRE (which is pretty much every OS out there).
Sun could have used this technology to build an eco-system around the Java technology while marketing the virtues of the language more successfully. Eventually they could've built premium services around the app store creating another revenue stream.
Oh, well!

Performance Tuning in Java

Recently, a friend of mine asked me on how he can improve performance in an open-source Java application that he was working on. I was reminded of the past (i.e., circa 2003) when Java was being bashed for poor performance and when a number of books were written on improving performance of all most all parts of Java.

While it is true that an application running within a container will be naturally less performant than a native application, the question one should ask is whether the optimal performance they are getting is good enough for them, as opposed whether it is the best. When it comes to performance, in most cases, 'works for the situation' is more useful than being the fastest horse in the racetrack.

Coming back to the issue in question, I think there are some basic steps one can take to drastically improve the performance of a Java application with relatively minimal effort. In this case, the application was a data extraction tool that was writing thousands of small (1 - 5KB) XML files. Based on this, the following recommendations can be provided.

JVM Garbage Collection parameters
If the size of the XML is small, it is reasonable to assume that the application may be using DOM instead of SAX. As DOM is an in-memory XML model, it would create a lot of objects that live for a short period of time.

By default, a JVM has two parameters - Xms and Xmx, which is the minimum or start-up memory the JVM will use and the maximum amount of memory it can possibly use (beyond which an OutOfMemoryError will be thrown). By default, Xms parameter is around 40MB to 64MB for Windows systems and the Xmx is around 128MB - 256MB. The JVM would start with the Xms memory and if it hits the limit, will then keep increasing the memory till it reaches Xmx value in steps. The problem is that this incremental increase comes at a performance cost, especially if you start really low compared to the needs of the application. If you have a reasonable idea of your application's minimum memory requirements, or if your system has enough memory to spare, it's a good idea to boost this to a much bigger number.

In this case, the numbers were boosted to 512MB for Xms and 1024MB for Xmx. Additionally, it is a good idea to increase the -XX:MaxPermSize value from the default 32MB to a more respectable 128MB or even 256MB (which was the value used in this case).

Once these changes were done, the application, which took 1 hour 15 minutes to run, went down to 28 minutes - and all without changing a single line of code!

Choose your OS wisely
The second improvement came not within Java, but outside. This particular application was creating around 300,000 files under a single folder. Windows typically does not handle huge file volumes nicely within a single folder. The optimal value seems to be somewhere around 2,000 files. UNIX based systems on the other hand, have no such issues.

The application, thanks to Java's portability, was moved to a UNIX based system. The running time went down from 28 minutes to 13 minutes - again without changing a single line of code!

There are a number of other tweaks that can be made, and most without changing the actual application itself - hopefully for another blog down the road...

So, the bottom line is, don't blame the language/library without spending some time to fine tune the performance - more importantly, sometimes it just takes a few minutes of effort to make a big difference.

Java Store - Good concept but late arrival

Sun unveiled Java Store yesterday at the JavaOne conference - yet another example of playing catch-up. As Java user for more than 10 years now, I think it's a great concept, but can't help but feel that this would've been a great hit probably circa 2004.
It was a time when the Java brand was getting beaten up by almost everyone as being slow and cumbersome compared to native applications. A Java Store with some really good applications would have gone a long way in establishing trust and keeping the web insanity that happened at that time (almost everyone wanted a web app at that time - whether it fit the scenario or not).
Now, it just seems like Java is copying Apple's idea and desparately trying to hold some ground on their own. Given the popularity of iPhone applications and their foray more and more into non-trival application space, they probably are. Sun missed this exact boat a few years back when they introduced J2ME. While half the blame should be shared by the telephone companies that wanted to monopolize the mobile application arena, Sun could've come out with an 'app store' that would've lent a lot of credibility to mobile applications. But they didn't and it was up to Apple to come up with the concept.
I wonder if Sun will get any mileage out of this. I have always felt that Java WebStart was a great concept and probably a very under-appreciated concept within the Java stack and am very happy to see that brought to fore with this initiative. But the question is, "Is it too little, too late?"

Exception handling/safety mindmap

I have been following Jessica Hagy's Indexed for a while now. While some of the graphs may be a bit too esoteric for me, I am amazed at the way she converts fairly complex issues into simple graphs.

This, combined with my *conversation* about exceptions and threads I blogged about earlier, made me think. Why not attempt something similar, not with graphs, but small, manageable mind maps on core concepts that I can later use to refresh my memory?

I think mind maps, if used properly, are a great way to crunch knowledge into visual chunks that are easy to remember. After all, our brain is supposed to work this way, so why not remember it in a way it's easy for the brain?

So, here's my first attempt. This is about the concepts around exceptions.

Don't whine about Java being slow

I came across the article Java is too slow! via The Server Side today. I think most Java programmers in the late 90's and early 00's have seen one Dick or the other in their career. I made a post there on why you can't whine about Java being slow and thought of expanding on that here.

I think Java got a bad rep mainly becuase of yuppie programmers (and yes, many from India with more interest in catching on to the next wave than learning fundamentals of programming) spewing out junk code. The big firms were no exception either. I vividly remember IBM's Visual Age for Java, the pre-cursor to the now popular Eclipse - a kludgy, super-slow IDE that made users cringe and forced them away from Swing and over to writing web apps for desktop systems.

I have implemented a number of highly transactional systems as well as GUI applications (yes, Swing) in Java that have run extremely fast. The top reasons I have found for Java being slow are Lack of application of fundamentals of programming (such as taking care of the number of objects you create). Spring to some extent reduces this risk, if used correctly. The impact is the highest in Swing applications where layouts are nested too much just because someone didn't spend enough time to do the design properly. In some cases, even moving some unwanted "new"s out of loops will help things significantly. Not using the right software and instead reinventing the wheel. I have used MQ Series for a distributed system that happily processed around 100,000 records or more per hour without any complaints. The average transaction was around 7ms. And it was all written in Java. Not using threads optimally. One program that used the Java Advanced Imaging packaged processed and uploaded 50,000 images (each image around 200K) per hour easily when multiple threads were used (you have to of course check how many threads are optimal). Not using a profiling tool to check your code. In the glory days of Borland, I used their OptimizeIt software - an excellent tool to detect object leaks, something that's ignored due to Java's garbage collection. Finally, not tuning the JVM. After optimizing your code, you need to tune the GC so that it works for your application. Just leaving the defaults as they are (or just modifying the ms/mx values doesn't cut it). It's like expecting an auto-gear car to win in a drag race. I am sure there are a number of other reasons I can come up with, but these 5 tend to be the most common. It looks like Java itself succumbed to the slowness by creating stupidities such as the original J2EE, where there was way too much architecture/framework and way little code. Thankfully, the open source community - with Spring, Hibernate, and others - seem to have brought some sanity back and hope it continues before another wave of stupidity (possibly int he form of overuse of annotations) sets in.

Java Exceptions

Last week, I had an interesting discussion with Sachi, Greg, Seshu and Muthu regarding exceptions. I had used an exception in my code to handle application level error conditions like so:

public void doStuff() throws AppExcecption {
// Some code
if(error) {
throw new AppException("Cannot handle this automatically.", errorCode);
}
}

and somewhere else

public void controller() {
try {
Data d = doSuff();
writeData(d);
} catch (AppException e) {
writeErrorCode();
}
}

Sachi's contention here was that I should not use exceptions unless I have an exceptional case. My initial argument was that an application error was indeed an exceptional case as far as the application is concerned and hence, it's perfectly fine to throw an exception.

He was arguing for something like this:

public ErrorCode doStuff(Data d) {
// Some code
if(error) {
return errorCode;
}

return null;
}

and elsewhere

public void controller() {
ErrorCode code = doStuff(d);
if(code == null) {
writeData(d);
} else {
writeError(code);
}
}

So, finally I decided to try his approach and found that it was also equally readable, and in fact, made me do some refactorings that made the code clearer.

While I still argue that in some cases throwing an exception like I mentioned is valid, I have softened up a bit. Here's my final thought:

1. Do not throw exceptions.
2. If you have to throw exceptions, try handling them as soon as possible.
3. If you cannot handle them right away, throw them to the caller.

The steps above have also been mentioned in the Robust Java - Stelting book.

powered by performancing firefox

One more Java web tier?

A new web tier has been sprung from the Spring camp called Spring Web Flow. At first, I was a bit wary of yet another web flow - after your typical JSP/Servlets, Struts, WebWorks, and now this.

But one thing surprised me though - the simplicity of the architecture. The intereseting aspect is that they are equating the page flow of a website as a state diagram. So, in effect, a website (or more specifically, a specific flow of web pages to accomplish something) is like the state diagram of a class. Each state is represented by a page and each event is an action that is performed by the user.

It's so simple that it drives me crazy why I didn't think about it when I read about the GoF state pattern!

With a nice architecture, I hope it keeps its promise. I'm thinking if I should use it in my Carnatic Search Engine.