Monday, May 21, 2007

Amazingly bad APIs

I actually don't hate the Java language (at least it's killing off c++), but every time I use it I'm blown away by how hostile the APIs are.

For example, let's say that we want to scale an image file. That's a fairly common thing to do, so it's probably only one line of code, right? Wrong. It turns out that they haven't even bothered to include a reasonable API for doing this. Instead, you have to take the awkward APIs that they do provide, and then write about 50 lines of code to do what you really want. This article that explains what the "right" way is.

Once the image is scaled, you'll want to save the smaller version too. Luckily, that's only a few lines of code. Of course that's only if you're happy with the default settings -- what if you want to use a different JPEG quality level? Here are the docs -- see if you can figure it out (the solution involves 4 additional classes).

For comparison, here is some Python code which loads an image, scales it, and then saves it at a non-default quality level:
from PIL import Image

i = Image.open("/tmp/c.jpg")
i.thumbnail([220, 133], Image.ANTIALIAS)
i.save('/tmp/c-thumb.jpg', quality=90)
Doing the same thing in Java requires closer to 100 lines, not because the language is bad, but because the APIs are terrible.

How did this happen? My theory is that the people who made the Java APIs just sat in a room making stuff up (and possibly drawing UML diagrams). Meanwhile, the Python people were actually writing software and creating libraries to make their life easier.

There is a lesson in this: If you are building a platform, you should also be writing applications for that platform, and the platform should be designed to make life very simple for those apps. It's my understanding that Ruby on Rails came out of a process such as this (they extracted the common tasks out of their web apps). If you just sit around staring at the clouds while writing your APIs, you'll probably end up with something bad.

I also have a second theory about what happened to Java: it attracted people who like to write piles of code filled with endless abstractions. When I worked at Google, one of the engineers on our Java mailing list suggested that we ban the keyword 'new'! I was puzzled by this suggestion, but after a little research I learned that one of the hot fashions in Java is to use factories instead of 'new' (not that factories are bad, but "sometimes good" does not imply "always best"). Better yet, you don't reference the factories directly (not enough abstraction), instead you have some framework that injects the factories into your classes according to what is written in an XML configuration file. In this way, you can make your code perfectly unreadable, but more importantly, it is very abstract.


Update: Several people have interpreted this post to mean that I'm opposed to the factory pattern. That is, of course, silly. Factories, when the situation calls for them, are fine. For example, the python code above is using a factory. What harms code is the blanket application of the factory pattern in every possible situation. To put this another way, Tabasco sauce is delicious on eggs, but I still don't put it on ice cream. (but don't think I haven't tried!)

Update Two: It's not about Java, really! My point here is that your APIs should emerge from the needs of real applications, and that they should make common tasks super-easy. Bad APIs are common in many languages, Java just happens to be the language that I'm using right now.

Why didn't I simply hunt down+install some third-party image library? Because that's a hassle too (probably more work than copying code off of some web page), and all I want is to generate nice looking thumbnails.

37 comments:

  1. "My theory is that the people who made the Java APIs just sat in a room making stuff up (and possibly drawing UML diagrams). Meanwhile, the Python people were actually writing software and creating libraries to make their life easier".

    That's probably exactly what happened.

    I know enough java to hold my own with it, but do more useful work in python.

    That is a horrible, horrible API.

    ReplyDelete
  2. I think the problem with the APIs isn't that they're abstract, it's that they're not abstract. You need to know (some of) the internal details to know how to use them.

    About 'new', it's tempting, when you realise that the factory pattern (and DI) is useful sometimes, to see whether it's useful all the time. It's not. If you have one interface with one implementation, whose instances are provided by a factory, your code is more complex than it needs to be. Get rid of the interface, make the implementation public, and get rid of the factory. Of course, if you have two implementations, then what you're doing might be fine.

    You won't lose anything - allowing behaviour and interface to evolve separately leads to leaky abstractions anyway.

    ReplyDelete
  3. How difficult is this with the ImageMagick library? I agree with you to a point, that the GPL Java folks should bundle or easily make available the useful extensions.

    ReplyDelete
  4. How did you come to a conclusion that Java is killing off C++? Yes, Java has completely pushed almost everything else in a boring, dark world of enterprise, becoming new COBOL, but who cares about enterprise software?

    99.9% of all open source Linux development is still C or C++ and there is nothing coming to replace them. Well... except maybe D.

    ReplyDelete
  5. Well, on one side the imaging APIs for Java have a problem: different layers were born in differen moments and for different reasons. We have the old Image that was focused on async loading; then BufferedImage and Java2D; and there's JAI too. This sort of confusion probably has a part in the scenario.

    On the other hand, I don't agree fully with the criticism. We can say in the same way that JDBC is overly complicated for persistence (and in fact it is). The point is that there are higher level APIs (some in the JDK, some by third parties) that sits upon JDBC and perform most of the common jobs more easily.

    You might be interested in looking at mistral.tidalwave.it (look at the white paper first). One of its aim is to provide a simpler and cleaner interface over the imaging APIs.

    ReplyDelete
  6. @zeka... don't look only at Linux - look at the whole market and Java has been steadily above C++ since many years (numbers from jobs offers).

    Second, I'm not a big expert of the Linux world, sure Java isn't yet very popular (but things are changing with the GPL license), nevertheless I've got a lot of doubts about that 99.9%, considering the large portions e.g. of Python and Mono you can find...

    Unless you're talking about the kernel, device drivers, X11, in this case we agree - but this is a very specific field...

    ReplyDelete
  7. The major malfunction with Java is the "there is exactly one way to do it" mentality. This looks like:

    1. Take some area of interest, say directory access, XML processing or cryptography. Or Java 2-D.

    2. Create an abstraction of the behavior of said area that covers anywhere from 50-90% of the functionality inherent the domain.

    3. Create a fake provider API that bridges real implementations (which always exist, since #2 is their lowest common denominator) with this artificial abstraction. Make sure that any feature that is too vendor-specific or complicated is thoroughly obfuscated or made inaccessible to the end-user, regardless of its usefulness.

    4. License or other wise bundle some implementation in the default SDK so something works out-of-the-box (for some simplistic value of "works").

    Excessive use of factories are a symptom of this diseased mentality. The usability of these kinds of APIs is necessarily deficient.

    ReplyDelete
  8. The JDK does contain some very bad APIs but it also has very good ones. Apart from this image resizing issue, the Java 2D API is actually very good. Having done my share of Python code, I must say I'm also angry at some parts of Python's API (inconsistencies, useless APIs, missing APIs, etc.)

    ReplyDelete
  9. Surely to do the same as your Python example, you would just need:

    BufferedImage jpg = ImageIO.read( new File( "/tmp/c.jpg" ) ) ;
    BufferedImage tmp = new BufferedImage( 200, 133, BufferedImage.TYPE_INT_RGB);
    Graphics2D g2 = tmp.createGraphics();
    g2.setRenderingHint( RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BICUBIC ) ;
    g2.drawImage(ret, 0, 0, 200, 133, null);
    g2.dispose();
    ImageIO.write( tmp, "jpg", new File( "/tmp/c-thumb.jpg" ) );

    Not the "closer to 100 lines" you say you'd need...

    I'm not arguing that it couldn't be nicer...all I'm saying is it isn't quite as bad as you make it sound...

    ReplyDelete
  10. I have to agree there are definitely some areas in Java where you have to write a book when all you want is one sentence.

    However, the factory pattern that you describe as making your code "perfectly unreadable" - here I have to disagree. I have used APIs that use factories, and generally they're very easy to write to and better yet, they allow you to build large and complex systems in a modular way. Complaints I have heard about Python and PHP is that yes they're effective to get something quick and dirty done, but when you scale up the size of your app, things start getting pretty ugly.

    ReplyDelete
  11. In Adobe ColdFusion 8 (which compiles to Java byte code):

    <cfimage action="resize" source="/temp/c.jpg" height="220" width="133" destination="/temp/c-thumb.jpg" />

    Just 1 line of code. Of course Adobe has done a lot more than just wrap JAI, they've ported a lot of thier image libraries to Java from C++ so the quality and flexibility is high.

    ReplyDelete
  12. I'm all in for less Java and more applications:

    http://laurentszyster.be/less4j

    ReplyDelete
  13. The same way you used PIL in your python example, here's ImageJ:

    ij.plugin.JpegWriter.setQuality(75);
    new ij.io.FileSaver(new ij.io.Opener().openImage("/path/to/image1.jpg"))
    .saveAsJpeg("/path/to/image2.jpg");

    Two lines.

    There are ways to use java that manage to avoid the java API altogether. If that is good or bad, I have little idea.

    ImageJ is on the public domain, a 1Mb jar file for scientific image processing, with literally hundreds of user-contributed plugins.

    ReplyDelete
  14. This comment has been removed by the author.

    ReplyDelete
  15. It is the circle of life...

    I like C as a language. But C sux because if you want to use libc to scale JPEG images it takes 100 lines of code. The people who sit around staring at the clouds, dreaming up APIs will probably end up with C. There is a lesson in this: If you are building a platform, you should also be writing applications for that platform, and the platform should be designed to make life very simple for those apps. It's my understanding that BASIC came out of a process such as this.

    ... 'round and 'round she goes...

    ReplyDelete
  16. Tim, your code is shorter because it does the wrong thing. (wrong scaling algoritm and wrong jpeg quality level)

    David, there's nothing wrong with the factory pattern. See my update.

    Albert, that's a good example of why the problem is the the libraries and not the language, though I think you forgot to scale the image.

    Richard, I think perhaps you completely missed the point. As Albert shows, the Java libraries don't have to bad.

    ReplyDelete
  17. My bad. Two more lines would be needed: one to catch the reference to the ImagePlus returned by the Opener, and another to call setProcessor(null, imp.getProcessor().resize(w, h)); on it.

    One extra comment: I despise java (the language) for its overboard verbosity, but the free non-official libraries are very useful. To get the most from it, I use jython: python code that lives inside a JVM and has access to all java libraries (even private fields if one sets it to do so).

    In jython, your example code using ImageJ would be:

    import ij

    i = ij.io.Opener().openImage("/tmp/c.jpg")
    i.setProcessor(None,i.getProcessor().resize(220, 133))
    ij.plugin.JpegWriter.setQuality(75)
    ij.io.FileSaver(i).saveAsJpeg("/tmp/c-thumb.jpg")


    To make it as short as your pure python code, one would have to add a new method to the ij.io.FileSaver specifying the jpeg quality to use. Why it's not present is because ImageJ is (I speculate) used as a GUI a lot more than for (headless) scripting, i.e. lack of current demand.

    ReplyDelete
  18. Anonymous10:17 PM

    The Python example can be written on one line (by using 'resize' instead of 'thumbnail'). It's not the line count that's important here, really...

    (the 'thumbnail' method has an advantage over 'resize', though -- if the source image is large enough, it reconfigures the JPEG reader to use a different DCT stage, to avoid decoding data that's not actually needed. This improves performance a lot, and also reduces memory requirements. None of the Java translations I've seen seem to do the same thing...)

    ReplyDelete
  19. Good point. Which is why I started writing API docs and sample code first. See my post on this.

    ReplyDelete
  20. Anonymous2:10 AM

    (and yes, the 'thumbnail' method also preserves the aspect ratio, of course. the resulting image will fit into a 200x133 rectangle; it won't necessarily be exactly that size).

    ReplyDelete
  21. I think it's important to realise the difference between platforms and frameworks. Java is a platform and Sun try and create APIs to enable you to do as much as possible on that platform, but there is still a need for frameworks to make the apis easier to use. take the servlet spec, in many ways it is a great api for using http, but coding web apps using the servlet spec can be tedious so people use frameworks to introduce consistency and simplicity.

    in your case, there are frameworks that make image manipulation easier, the problem is that they aren't endorsed by sun or any other body so people taking up java start off by learning (and suffering at the hands of) the sun apis and then start using/writing frameworks as they learn about the community/industry. this differs from a language like ruby where the frameworks often come first.

    ReplyDelete
  22. This comment has been removed by the author.

    ReplyDelete
  23. Or if what you really want to do is image processing, you might actually try the image processing API...

    http://java.sun.com/javase/technologies/desktop/media/jai/

    If all you bother to look for is a hammer, you'll have a tough time turning screws.

    ReplyDelete
  24. JAI is amazingly horrible and the documentation is useless. ImageJ looks really nice. I was looking for something like it recently and didn't find it, so hopefully it will become more well known. That said it looks to me like the jpeg writer would not be suitable for using in multiple threads.

    ReplyDelete
  25. Yes... just yes.

    PS: Python rules, try Django.

    ReplyDelete
  26. Just for the record, I've posted back here.

    PS JAI is extremely rich and powerful; its problem is the documentation.

    ReplyDelete
  27. You got close to my opinion on how bad APIs happen. I see the following things in my practice:

    .o. API designers tend to create things that are easy for them to DESIGN rather than things that easy for people to USE. This tends to create broad, flat, hairy, thin, bottom-up designs that expose all the options of the underlying thingummy as if they were all equally important.

    .o. Since all options must be set, they must all be set EXPLICITLY (said facetiously). This means the user must KNOW EVERYTHING before he can DO ANYTHING. It's also the opposite of "intelligent defaults."

    .o. There is a giant faction of people (about 50%) who actually distrust abstraction, hierarchy, grouping, and composability. These people prefer a long list of special case to even ONE level of abstraction. They view EXPLICIT=SIMPLE. Of course, their view is logically coherent, so it's very difficult to argue against. It's also crazy and wrong :)

    ReplyDelete
  28. I am not sure why exactly that you have the opinions that Java API's are hostile. It seems like a broad general statement. Were you referring to just the imaging API's? What other Java API's have you worked with that you share such an opinion?

    It is clear that you have very deep knowledge of image processing concepts and their implementation in some languages. However, although it may be true in the case of imaging, I don't believe that it's true in general for Java API's. In fact, I find them to be the most robust in general compared to almost any other language and implementing most software applications.

    If you find these API's to be deficient compared to other languages, why not participate in making these API's more robust? Java is open source and I am very sure that your expertise in this area could make an immediate impact in making the Java 2D, imaging, or whatever API could make them as powerful to your liking.

    ReplyDelete
  29. There was interesting write up on harvested Vs Non Harvested framework from Martin Fowler (http://www.martinfowler.com/bliki/HarvestedFramework.html) which suggested any framework that comes as by product of some real world application building exercise is more valuable than the framework created by committe focussing solely on the framework. (Spring, Struts are good examples of non harvested frameworks.)

    Having said that it's too much to expect from Sun to make the APIs perfect. The frusturation on this from James is completely correct when he calls for more participation & collabaration. (http://blogs.sun.com/jag/entry/participate)

    ReplyDelete
  30. I'm thoroughly amazed at this: in almost every class I teach, when I talk about programming to Java interfaces, at least one person objects, saying that that means we have to introduce factories all over the place for those interfaces.

    How did this happen? When did all these programmers learn that "interfaces mean factories"? Who taught them this nonsense?

    ReplyDelete
  31. There has been quite a few suggestions about using free small libraries. These or similar should be in the java core, not external libraries.

    At the moment I'm working as contractor for very large organization. In order to use almost any external libraries, the trouble for getting them approved would practically take more time and effort than to writer the needed functionalities by myself. Now if these were in the java -core, I could use them right away, but as external libs... nope.

    ReplyDelete
  32. Hey Paul, Last couple of your posts revolve around uploading files, processing images etc. I am wondering where is this coming from, what is it that you are trying to build/solve and if you can share it with others at this point in time.

    ReplyDelete
  33. I think the important part is to use good abstractions. I found the java library has often the right level of abstraction. Perhaps image scaling is just an example that's not the strong side of this library?

    ReplyDelete
  34. I think the biggest problem with Java API both being too abstract and not enough abstract is the community process. Instead of half a dozen smart guys knowing logic, model theory, like that, there's a crowd of mediocrities dictating the style and the essense of how things should be done. They want getters and setters, they want to control implementation, etc. Democracy. There are lots of advantages in democracy, but don't expect a new Galois theory being born by democratic voting.

    ReplyDelete
  35. Anonymous10:25 AM

    Have you seen Visual Basic 6? Has some cool features ;)

    ReplyDelete
  36. I'm still quite satisfied after a decade of Java. But some APIs in Java are not nice. Today I tried to do some attachments with java.mail. I couldn't simply attach a stream or even a file. No I needed a DataSource and a DataHandler which takes the DataSource in the constructor... Then I realized MimeBodyPart got a new Method attachFile() with JavaMail 1.4 which is actually kind of a shortcut for above back and forth. Hey didn't we need this back in JavaMail 1.0 times? When I realized that my File object, which is good enough to open stream, do attachments the old way is choked upon by attachFile being an illegal path .. I stopped coding for tonight :-(

    A good API should make simple things simple and complex things not too complex. With Java I often feel simple things are tough to do, only to realize later that the complex things do not get much more complicated any more. Sigh.

    ReplyDelete
  37. Anonymous1:31 PM

    http://www.comesolvego.com/2008/04/29/resize-images-with-java-high-quality-and-working-solution/

    ReplyDelete

Note: Only a member of this blog may post a comment.