Twas the night before Festivus...

And all through the FriendFeed Global International World Headquarters, not a creature was coding, not even Bret..



But before that, we were all very busy...


And not just busy posing for pictures -- we're also creating new products! I'm thrilled to announce the release of our latest platform initiative, AirGrievance.

Brilliantly wrong

Being a little bit wrong is easy, but it's much more interesting to be completely wrong.

Google Blogoscoped had an entry yesterday about my statement that, whenever possible, everything should have undo.

The comments on that story had the usual mix of no-so-great usability suggestions (putting options and settings everywhere), and a few good ideas, but it also had one comment that was so remarkably wrong that it brightened my whole day:

While an Undo feature could be useful, isn't this just coddling people who should otherwise be paying closer attention to what they are doing? A mistake is a mistake, and people need to learn to live with the consequences of the mistakes they make.

This comment may have been a joke, but I really like this "tough love" approach to usability, because in just a few words it perfectly captures the exact opposite of what we should be doing.

To design great products, we must truly empathize with our users, and understand that if they are having problems using our products, is more likely our fault, not theirs. This isn't the same as the disdainful or patronizing attitudes too often expressed by engineers and other technical people. Our users aren't dumb, they just have better things to do than waste time understanding poorly designed software.

As for "undo", in general, the more we can lower the costs of making mistakes, the faster we can move. This applies to everything from interfaces (I type fast because I have a backspace key) to societies (people are more likely to start new companies in cultures that are accepting of failure).

Is there more to life than money?

Whenever there is a discussion about joining a startup vs working at regular job, someone will defend the boring 9-5 job by saying, "there is more to life than money".

These people are, of course, right. There is plenty of evidence that happiness is only loosely correlated with wealth. People seem to derive a much greater sense of satisfaction from good relationships, and having a sense of purpose and meaning in their life. Money matters too, but not as much once the basic need for food and shelter are addressed, and those aren't big issues for most people deciding between joining a startup or a big company.

Furthermore, most people who do join startups will never experience a huge payday. Google distributed billions of dollars to thousands of employees, but that was truly exceptional. Even moderately successful startups that eventually sell for $50 million dollars or so will only make a couple of people rich.

If all you care about is money, I doubt that joining a startup is the right way to go. You'll probably make a lot more at a hedge fund, or by becoming a lawyer, or something like that.

Maybe it now seems like I'm defending the boring 9-5 job, but it's actually just the opposite. Those 8 hours/day are a huge chunk of of your waking life, and don't forget that you'll probably also spend a few hours preparing, commuting, and "unwinding". Even worse, bad or boring jobs can sap our energy, so that at the end of the day all we feel like doing is sitting in front of the tv.

That's no way to live, if you can help it. Why surrender such a huge chunk of your life just to get some money? For some people, that's the only option, but for those fortunate enough to be smart and educated, there's a better way.

Instead of throwing away your "working hours", why not make every minute count? Why not find work that you can actually enjoy, work that's fun and meaningful?

Of course that's easier said than done, but it's not impossible.

For reasons that will have to be left for another post, the structure and systems in big companies tend to make work meaningless and life unpleasant, at least for me. Of course smaller companies can be awful too, but they have a greater potential to be good.

If your job isn't working for you, if it isn't making you happy and energized, then why are you still there? I can't guarantee that you'll find something better, but perhaps you should at least try something new. Don't just give up on life.

I truly enjoy writing code. I love creating new products and features. I like getting feedback from users and finding ways to solve their problems. I like the game of business. I like helping people. Of course there is always some amount of unpleasant work that must be done, but that can be contained (my rule is that work should be no more than 10% awful).

That's why, even though I don't financially need to work, I choose to work (and end up staying up until 4am pushing new code). It's why we created FriendFeed, to have a great place to work, a place where we can build great products and have happy users. Of course I'd also like to earn a few billion dollars, and I plan to make all of our employees very wealthy, but that's more like a bonus. I don't believe that you can be happy and make great products and treat your users right if all you think about is money.

There's more to life than money.




My first Google pay-stub (including note from Heather). Money isn't everything, but this zero-dollar check did make me a little nervous.

There's no such thing as a "social network"

Of course there are many things that people call "social networks", but that's more confusing than enlightening. The "social network" aspect of these products is just a mechanism, not the purpose, and their purposes are often more different than similar. It's like saying that something is a "Javascript". Many websites use Javascript, but hopefully the Javascript is only there in order to accomplish some other purpose. LinkedIn and Twitter, for example, both use Javascript and social networking mechanisms, but clearly they are very different products and have very different uses. Facebook likes to call itself a "social utility", but to me that's just as meaningless.

That said, there is something "social networky" that unites these otherwise-dissimilar "social networking" sites. The two features that seem to define the social network aspect of a product are:

1. Some kind of page or profile for each user. The contents of this page vary wildly, but it always includes a name and often a picture.
   
   
2. "Friend" links among the profile pages, which may or may not be bi-directional. These "friends" only sometimes correspond to real-life friends.
   

Those two features alone are very simple but also very useless. Real products need more functionality in order to somehow deliver value to their users. It is this other functionality that defines the real purpose of a product, not the "social network", which exists only to enable or enhance the core purpose.

So what ARE the purposes of these many "social networking" products? (in random order)

1. Enable people to send messages to their friends. This functionality basically defines email, which can be considered one of the earliest "social networks", though most email systems lack browsable profiles. Annoyingly, many other social networking products reimplement this function, with mixed results.
   
   
2. Enable people to send messages to non-friends. There are two aspects to this feature: discovery (finding the people) and control (preventing unwanted messages). Email is weak on both aspects, and for many people, this is one of the most useful features of products such as Facebook and LinkedIn (I can easily find and contact people that I don't know).
   
   
3. "Live" address/phone book. Sites like Facebook have fields for phone, email, etc, and since everyone maintains their own page, the information is generally up-to-date (vs the bad-old-world of everyone having to broadcast "please update your address books" every time they move).
   
   
4. Less private communication. Email and IM are often too private. Life would be much less interesting if all of our conversations took place one-on-one in closed rooms. Part of what makes parties, offices, and other social environments interesting is that we can observe other people interacting, overhear conversations, and often join-in. Features such as the "wall" create a kind of public or semi-public email, allowing our friends to overhear conversations. Twitter is often used as a public IM/chat.
   
   
5. Passive communication about my life. I don't want to interrupt or spam my friends every time I have some little bit of news, take some new pictures, or get a random thought, but I have no problem blogging or Twittering those things, or uploading to photo sharing sites (which all have social features now). Similarly, I enjoy seeing this information from my friends (but I wouldn't want them calling me on the phone to tell me about their lunch every day).
   
   
6. Passive sharing of non-personal content. I'll occasionally see an interesting or amusing web page and want to share it with friends. I used to put these things into my IM status, but now I add them to my FriendFeed. Email, Twitter, Facebook, and Digg/Reddit are also sometimes used for this purpose (though obviously Digg is less "friend" oriented).
   
   
7. Self expression. This is most prominent on MySpace, where user profiles are highly customizable and can have embedded music (which starts playing as soon as I visit the page). This behavior is very similar to decorating your house or bedroom in "real life" -- there seems to be some human instinct to define our identity through decorations, fashion, music, art, etc. Self expression is an element of all products which have browsable profiles, but on most sites identity is expressed more in terms of content (favorite movies on Facebook, job history on LinkedIn, tweets on Twitter, etc).
   
   
8. Background information. If I meet someone new (or am about to meet them), I sometimes checkout their Facebook or LinkedIn pages to learn more about their background, see if we have common friends, etc. Obviously this also ties in with self-expression and identity in more social settings -- people "friend" each other after meeting and their MySpace page (or whatever) becomes part of that "first" impression.
   
   
9. Dating. One of the earliest and still most popular uses of sites like MySpace, Facebook, Orkut, etc. It's less "explicit" than on dating sites, but by putting themselves online, people can see and be seen, plus get background info, see who their common friends are, etc. The Facebook newsfeed even tells us when friends break-up (or start dating).
   
   
10. Jobs. Job hunting and hiring are essentially the "professional" analog of dating and seem to work in somewhat similar ways.
    
    
11. Finding old friends. By enabling friend-of-friend and school class browsing, it's much easier to locate old friends, and therefore people do more of it. This also lets people track down old friends without appearing crazy (they can "bump" into them on classmates.com, or whatever -- no private detective needed).
    
    
12. Keeping in touch with "unclose" friends (related to "passive communication about my life"). It's fun to know what's going on with people who we've known in the past, even if we aren't close friends. Profile browsing and newsfeeds make this easier. This can also make unclose friends into closer friends, as we may coincidentally be in the same city at the same time, be attending the same events, or just start chatting.
    
    
13. "Micro-socializing" (I need a better term here). A lot of wall posts, poking, winking, commenting, Twittering, etc can go into the category. The comments on FriendFeed generally fall into this category as well -- someone shares a link or posts some photos, then they and their friends will end up chatting about it. This is somewhat analogous to the traditional "water-cooler" conversation.
    
    
14. Word-of-mouth recommendations. Shared links, news, Yelp reviews, some Twitters, and the like provide us with information about what our friends are doing. Word-of-mouth can be very powerful since we typically trust our friends much more than random people. The new Facebook "Beacon" advertising system is trying to use this effect to sell things.
    
    
15. General amusement for bored people. Clicking around on friends and friends-of-friends and even random people's pages can be kind of interesting. As sites provide more content (such as the links and videos on FriendFeed) or games (many popular Facebook apps are games) there will be even more of this.
    
    
16. Publishing. This especially applies to blogs, twitter, flickr, and the like, but being able to subscribe to the content produced by our favorite authors and artists as well as friends and family is very powerful for both the producer and the consumer.
    
    
17. Group sharing and socializing. This was traditionally done via mailing lists and that's the model behind Google/Yahoo Groups, but of course Facebook, Orkut and the like have have some basic group functionality too, and Ning seems to specialize in it by enabling each group to create their own social network.
    

This list is certainly not complete, but I'm way over my 30 minute limit... I look forward to seeing your suggested additions. I'll probably post an improved list at some point.

I also hope to explain how my company, FriendFeed, fits into all of this. It definitely has social aspects (the word "friend" is right in the name!), but it certainly isn't a social network in the style of MySpace or Facebook.

We all have tunnel vision

Supposedly people can hold about seven "items" in their mind at any one time. I was never sure what that meant -- what qualifies as an "item"? I recently realized that it means our brains have roughly seven "registers", similar to the registers on a microprocessor. These registers don't store much information, they really just hold pointers to something in longer term memory.

Here's the important realization: There's a million important things going on in the world and in our lives, but we're really only aware of seven of them. This means that we all have a very narrow and limited understanding of the world and our own lives. The seven things on our mind all seem very important, while everything else is just kind of forgotten.

We must be very careful about what gets loaded into those seven registers. I like to read reddit occasionally, but I've found that it can be a little dangerous. Some days it is filled with bad news -- my seven registers get filled with scary and depressing things, and it feels like the world is crumbling.

On the other hand, I can go for a walk outside and after a little while I begin to notice how nice the trees are, the various smells of nature, the construction progress on the house down the street, and other pleasant (for me) things, and the world seems like a really nice place.

This seven register limitation also makes people very subject to manipulation. If you can control what is getting loaded into their attention, you can largely control what they think and how they feel. For example, if people keep talking about Iran and how scary they are and debating what to do about them, then pretty soon Iran will seem like the biggest, scariest problem in the world, and no solution will seem too extreme. The truth is that there are probably 100 more important problems, but it won't seem that way because all seven registers are loaded with the same topic. The subject of the debate is more important than the content.

At any one time, a million things are going wrong, the world is falling apart. At any one time, a million things are going right, the world just keeps getting better. Those are both true statements, but one will feel much more true depending one which seven things are loaded into your attention now.

The same thing can happen inside of a company. We can easily become obsessed with one issue or threat, and it ends up taking all of our attention and energy even though it's not the most important problem. And because it seems like the only problem, our response is often completely irrational. I have some amusing Google anecdotes, but I'm almost out of time for this post, so they will have to wait...

How do we deal with this? I'm not certain, but the first step is to simply be aware of it. Take a break. Go for a walk. Try to clear out your attention. Maintain balance. Keep four good things in mind at all times (make a list). We shouldn't ignore the bad things, but we mustn't allow them to overwhelm us, because that's what will happen if the bad completely pushes out the good.

27 minutes.

Update: To be clear, the actual value "seven" is not important -- it's just an idea that many people are familiar with. The point is that our attention has a very limited capacity, and therefore we are always missing out on the bigger picture.

Back to blogging again, I hope

I have about a 1000 posts queued up in my brain, but these things take forever to write, and now that I'm working again, I don't have forever to waste. So, in the spirit of "Good enough" is the enemy of "At all", I'm going to start blogging again, but with an absolute limit of 30 minutes per post. If after 30 minutes I'm not happy with what I've written, I'll delete the post and move on for the day. If anything seems less thought-out or more confusing and inflammatory, this is why (and I didn't mean whatever you thought I meant, I meant something intelligent and non-insulting).

Meanwhile, check out My FriendFeed.

The next big adventure

We have a new startup: FriendFeed!

It's still early beta, and I'll write more about the product and company as soon as I get a chance. Until then, read the NY Times article and checkout our "about" page.

Quick: Read this if you ever store password data

Thomas Ptacek has a good post explaining why your password storage scheme is probably bad.

His post has a lot of words, so I'll summarize:

• Storing passwords in plaintext is obviously bad (most developers already know this)
  
• Using a salted-hash such as MD5 or SHA-1 isn't much better. (too easy to brute force)
  
• Your other clever password storage ideas are probably bad too.
  
• Use bcrypt instead. It was created by real cryptographers for just this kind of thing.
  

Here's bcrypt for Java. It has a nice simple API:

// Hash a password for the first time
String hashed = BCrypt.hashpw(password, BCrypt.gensalt());

// Check that an unencrypted password matches one that has
// previously been hashed
if (BCrypt.checkpw(candidate, hashed))
    System.out.println("It matches");
else
    System.out.println("It does not match");

I'm sure other popular languages have similar libraries.

16GB iPhone "SLR" includes high-quality 8MP camera

And it still fits in my pocket:

Reality, page 191

...replaces all objects of belief with one single thing: reality itself. We believe only in this universe. We don't believe in the afterlife. We don't believe in the sovereignty of nations. We don't believe in money or power or fame. We don't believe in our idols. We don't believe in our positions or our possessions. We don't believe we can be insulted, or that our honor or the honor of our family, our nation or our faith can be offended...

We just believe in reality. Just this.

...doesn't ask you to believe in anything you cannot confirm for yourself. It does not ask you to memorize any sacred words. It doesn't require you to worship any particular thing or revere any particular person. It doesn't offer any rules to obey. It doesn't give you any hierarchy of learned men whose profound teachings you must follow to the letter. It doesn't ask you to conform to any code of dress. It doesn't ask you to allow anyone else to choose what is right for you and what is wrong.

...is the complete absence of belief. ...is the complete lack of authority. ...tears away every false refuge in which you might hide from the truth and forces you to sit naked before what is real. That's real refuge.

Reality will announce itself to you in utterly unmistakable ways once you learn to listen. Learning to listen to reality, though, ain't so easy. You're so used to shouting reality down, drowning it out completely with your own opinions and views, that you might not even be able to recognize reality's voice anymore. It's a funny thing, though, because reality is the single most glaringly obvious thing there is... Yet we've forgotten how to recognize it.

What does it mean to own a "right"? Was Gandhi a thief?

From the Wikipedia entry on Gandhi's 1930 Salt March:

The British monopoly on the salt trade in India dictated that the sale or production of salt by anyone but the British government was a criminal offense punishable by law. Salt was readily accessible to coastal area dwellers, but instead of being allowed to collect and use it themselves for free, they were instead forced to purchase it from the colonial government.

In other words, the British government owned the right to produce and sell salt in India.

And another page says:

On April 5, 1930 Gandhi and his satyagrahis reached the coast. After prayers were offered, Gandhi spoke to the large crowd. He picked up a tiny lump of salt, breaking the law. Within moments, the satyagrahis followed Gandhi's passive defiance, picking up salt everywhere along the coast. A month later, Gandhi was arrested and thrown into prison, already full with fellow protesters.

Was Gandhi stealing salt from the British government, or was he simply breaking the law which gave the British the exclusive right to produce salt?

Since Gandhi wasn't actually taking the salt away from the British, and the ocean contained a practically infinite supply of salt (so it can't be claimed that he was taking it from anyone else), I'd argue that he was not stealing anything.

Did Gandhi steal the legal "right" to produce salt? Not really. A right has no substance or physical reality, so it is difficult to outright steal one (you could perhaps trick someone into signing a contract transferring the right, but clearly that's not the case here).

Gandhi's actions DID decrease the value of the British right to produce salt, but many actions could have that effect. For example, he could have convinced the people of India to switch to a lower salt diet, thereby decreasing the demand for salt and the value of the British monopoly. I don't think that anyone would call that stealing.

In fact, I've never seen anyone claim that Gandhi was a thief. It seems like a somewhat silly argument.

So what's my point? Owning a "right", which is a form of "Imaginary Property", is not the same as owning real property. If someone violates your exclusive rights, they may be breaking the law, but they are not stealing. To claim otherwise is silly and dishonest.

Update: Some people have interpreted this post as meaning that I'm opposed to Imaginary Property, or that infringing on other's legal rights is ok. That is not the case at all -- IP can serve an important and beneficial role in society. My only point is that Imaginary Property is not the same as Real Property, and that infringing on someone's legal rights isn't the same as theft. Pretending that they are the same has caused a lot of unnecessary trouble.

The first thing that you need to understand about humans

Humans aren't rational -- they rationalize. And I don't just mean "some of them" or "other people". I'm talking about everyone. We have a "logic engine" in our brains, but for the most part, it's not the one in the driver's seat -- instead it operates after the fact, generating rationalizations and excuses for our behavior.

I first realized this fact several years ago when I read something about "split brain" people. These people have had their corpus collosum, the structure which connects the left and right brain hemispheres, cut or damaged, leaving the two halves of the brain largely unable to communicate with each other. I don't recall where I read that, but a quick Google turned up this similar article:

"A split-brain patient shown a photograph of Hitler only in the right hemisphere, for example, might exhibit facial expressions indicating anger or disgust. But when asked to explain those emotions, the patient will often invent an answer, such as 'I was thinking about a time when someone made me angry.'" (Newberg and D'Aquili 2001, p. 23)

Kenneth Heilman offers another, more concrete example, writing about the research of Dr. Michael Gazzaniga and his colleagues. In one experiment, they showed sexually suggestive pictures to a woman with callosal disconnection, flashing them only on the left half of a screen so only her right hemisphere could perceive them. The woman giggled and blushed, but when asked why she was doing so, she replied that she was thinking of something embarrassing (Heilman 2002, p. 129).

Are these people lying? In one sense of the word, perhaps; but it seems clear that there is no conscious intent to deceive. Rather, researchers have concluded, what is happening is that the right hemisphere, upon seeing an image with strong emotional connotations, generates the appropriate response. However, due to the callosal disconnection, it cannot transmit the associated sensory data to the left hemisphere and its language centers. The left hemisphere perceives a change in the body's state, but does not know why - and so it "fills in" the missing details, fabricating a logical reason for the emotional reaction. This happens at a subconscious level, so that the person genuinely believes the verbal explanation they provide. In the language of psychology, this filling-in process of unconscious invention is called confabulation.

The important realization is that this process of "confabulation" is not limited to people with brain damage -- everyone does it -- people with "split brain" are just a little more obvious.

It's tempting to think that "most people" have this problem, but that you or I are different, to think that our actions are all logical. That is a mistake -- denying the truth is irrational and dangerous. By accepting that people are fundamentally irrational, we can deal with ourselves and others in a more rational and effective manner. We can learn to manage our irrational selves (somewhat). If, however, we insist that all of our actions and feelings are rational, then we will never be able to deal with them honestly, and are more likely to cling to irrational beliefs and limitations. If we are going to be honest, then we must admit the possibility that everything we know and believe is, in fact, incorrect.

The fascinating (and easy to read) book "Influence: Science and Practice" has some great examples and explanations of how people actually work. Here's a great quote about the effect of attractiveness, and how unaware people are of their true decision making process:

A study of the 1974 Canadian federal elections found that attractive candidates received more than two and a half times as many votes as unattractive candidates (Efran & Patterson, 1976). Despite such evidence of favoritism toward handsome politicians, follow-up research demonstrated that voters did not realize their bias. In fact, 73 percent of Canadian voters surveyed denied in the strongest possible terms that their votes had been influenced by physical appearance; only 14 percent even allowed for the possibility of such influence (Efran & Patterson, 1976). Voters can deny the impact of attractiveness on electability all they want, but evidence has continued to confirm its troubling presence (Budesheim & DePaola, 1994).

A similar effect has been found in hiring situations. In one study, good grooming of applicants in a simulated employment interview accounted for more favorable hiring decisions than did job qualifications—this, even though the interviewers claimed that appearance played a small role in their choices (Mack & Rainey, 1990). The advantage given to attractive workers extends past hiring day to payday. Economists examining U.S. and Canadian samples have found that attractive individuals get paid an average of 12-14 percent more than their unattractive coworkers (Hammermesh & Biddle, 1994).

Equally unsettling research indicates that our judicial process is similarly susceptible to the influences of body dimensions and bone structure. It now appears that good looking people are likely to receive highly favorable treatment in the legal system (see Castellow, Wuensch, & Moore, 1991; and Downs & Lyons, 1990, for reviews). For example, in a Pennsylvania study (Stewart, 1980), researchers rated the physical attractiveness of 74 separate male defendants at the start of their criminal trials. When, much later, the researchers checked court records for the results of these cases, they found that the handsome men had received significantly lighter sentences. In fact, attractive defendants were twice as likely to avoid jail as unattractive defendants. In another study —- this one on the damages awarded in a staged negligence trial -- a defendant who was better looking than his victim was assessed an average amount of $5,623; but when the victim was more attractive of the two, the average compensation was $10,051. What's more, both male and female jurors exhibited the attractiveness-based favoritism (Kulka & Kessler, 1978).

Other experiments have demonstrated that attractive people are more likely to obtain help when in need (Benson, Karabenic, & Lerner, 1976) and are more persuasive in changing the opinions of an audience (Chaiken, 1979).

Understanding our irrational nature is also critical to product design. How can you expect to make something for humans if you don't understand how they think or make decisions?

Another book, "The Culture Code", has some great examples based on the author's experience helping companies improve the design and marketing of their products.

The people at Chrysler had indeed asked hundreds of questions; they just hadn't asked the right ones. They kept listening to what people said. This is always a mistake. As a result, they had theories about moving the Wrangler in multiple directions (more luxurious, more like a traditional car, without removable doors, enclosed rather than convertible, and so on) with no clear path to follow.
...
When I put groups of consumers together, I asked them different questions. I didn't ask them what they wanted in a Jeep; I asked them to tell me about their earliest memories of Jeeps.
...
I returned to those wary Chrysler executives and told them that the Code for Jeep in America is HORSE. Their notion of turning the Wrangler into just another SUV was ill advised.
...
The executives weren't particularly moved. After all, they had vast research that told them consumers said they wanted something else. ... I asked them to test my theory by making a relatively minor adjustment to the car's design: replacing the square headlights with round ones. Why? Because horses have round eyes, not square ones.
...
They tested the new design and the response was instantly positive. Wrangler sales rose and the new "face" of the Wrangler became its most prominent and marketable feature. In fact, the car's logo has incorporated its grille and round headlights ever since.

Using surveys and focus groups to design your product is a great way to produce boring and mediocre products that are the same as everything else.

Another great quote from "The Culture Code" connects us back to the "split brain" example that I started with:

What do Americans look for in a car? I've heard many answers when I've asked this question. The answers include excellent safety ratings, great gas mileage, handling, and cornering ability, among others. I don't believe any of these. That's because the first principle of the Culture Code is that the only effective way to understand what people truly mean is to ignore what they say. This is not to suggest that people intentionally lie or misrepresent themselves. What it means is that, when asked direct questions about their interests and preferences, people tend to give answers they believe the questioner wants to hear. Again, this is not because they intend to mislead. It is because people respond to these questions with their cortexes, the parts of their brains that control intelligence rather than emotion or instinct. They ponder a question, they process a question, and when they deliver an answer, it is the product of deliberation. They believe they are telling the truth. A lie detector would confirm this. In most cases, however, they aren't saying what they mean.

The reason for this is simple: most people don't know why they do the things they do. In a classic study, the nineteenth-century scientist Jean-Martin Charcot hypnotized a female patient, handed her an umbrella, and asked her to open it. After this, he slowly brought the woman out of her hypnotic state. When she came to, she was surprised by the object she held in her hand. Charcot then asked her why she was carrying an open umbrella indoors. The woman was utterly confused by the question. She of course had no idea of what she had been through and no memories of Charcot's instructions. Baffled, she looked at the ceiling. Then she looked back at Charcot and said, "It was raining."
...
Even the most self-examining of us are rarely in close contact with our subconscious. ... Therefore, we give answers to questions that sound logical and are even what the questioner expected, but which don't reveal the unconscious forced that precondition our feelings. This is why polls and surveys are so often misleading and useless.

Of course this understanding is important to more than just product design. It's critical to anything involving humans, which is everything involving us. How does it apply to your life and work? Are you still making decisions with the assumption that we are rational?

Quick: All actions should have 'undo'

I just slipped and added a misspelled word to the Firefox dictionary. Guess what the process is for undoing that mistake?

Some apps "fix" this problem by prompting with an "are you sure" type dialog before performing the action. That's very annoying, and it doesn't always work since I might hit space or enter or something and accidentally "ok" it. (this happens to me a lot when the dialog pops up while I'm typing)

I really like the solution used in Gmail:


After an action is performed, state what the action was and give an opportunity to undo, but in a non-intrusive way. More people should copy this approach, including Gmail, which needs it on "Send" (this will require adding a short delivery delay, like 10 sec, but it's worth it).

Quick: Maybe you can hack computers

But do you know how to hack people? It's only code.

Quick: Job culture

Here's a good link about the "Job culture":

The cultural assumption of a true free enterprise system would be: “Individuals are responsible for their own lives and labors. They trade as equals, but are beholdin' to nobody.”

Free enterprise isn't anything like big-corporate capitalism. We've been told the two are equivalent, but that's just another bit of cultural brainwashing.

Think about it. Job holders by definition aren't capitalists. Job holders, no matter how well paid they might be, function merely as the servants of capitalists, just as medieval serfs functioned as the servants of lords. They are beholdin'. They function in a climate of diminished responsibility, diminished risk, and diminished reward. A climate of institutional dependency.

...

The daily act of surrendering individual sovereignty – the act of becoming a mere interchangeable cog in a machine – an act we have been conditioned to accept and to call a part of “capitalism” and “free enterprise” when it is not – is the key reason why the present Job Culture is a disaster for freedom.

James Madison, the father of the Bill of Rights, wrote:
“The class of citizens who provide at once their own food and their own raiment, may be viewed as the most truly independent and happy. They are more: They are the best basis of public liberty, and the strongest bulwark of public safety. It follows, that the greater the proportion of this class to the whole society, the more free, the more independent, and the more happy must be the society itself.”

Madison was speaking specifically about independent farmers, but he was also a believer in the independent entrepreneur – and for the same reasons.

Madison (and his like-minded friend Jefferson) knew that people who are self-sufficient in life's basics, who make their own decisions, whose livelihood relies on their own choices rather than someone else's, are less likely to march in lockstep. Independent enterprisers are far more likely to think for themselves, and far more capable of independent action than those whose first aim is to appease institutional gods.

Living in the Job Culture, on the other hand, has conditioned us to take a “someone else will deal with it” mentality. “I'm just doing my job.” “The boss makes the decisions.” “I'm just following orders.” But if someone else is responsible for all the important choices in life, then we by definition, are not.

I reminds me of my favorite part of Paul Graham's essay on "Why to not not start a startup":

16. A job is the default

This leads us to the last and probably most powerful reason people get regular jobs: it's the default thing to do. Defaults are enormously powerful, precisely because they operate without any conscious choice.

To almost everyone except criminals, it seems an axiom that if you need money, you should get a job. Actually this tradition is not much more than a hundred years old. Before that, the default way to make a living was by farming. It's a bad plan to treat something only a hundred years old as an axiom. By historical standards, that's something that's changing pretty rapidly.

...

Now we look back on medieval peasants and wonder how they stood it. How grim it must have been to till the same fields your whole life with no hope of anything better, under the thumb of lords and priests you had to give all your surplus to and acknowledge as your masters. I wouldn't be surprised if one day people look back on what we consider a normal job in the same way. How grim it would be to commute every day to a cubicle in some soulless office complex, and be told what to do by someone you had to acknowledge as a boss—someone who could call you into their office and say "take a seat," and you'd sit! Imagine having to ask permission to release software to users. Imagine being sad on Sunday afternoons because the weekend was almost over, and tomorrow you'd have to get up and go to work. How did they stand it?

I have a plan to fix this, but it's a little difficult to describe because people have a hard time thinking outside of the surf mindset, the assumptions that underlie the "job culture". It will take some time.

Three types of ideas - bad ones are often the best

Product ideas can be divided into three categories:

1. Obviously good ideas that are very difficult to implement. Efficient cold-fusion, flying cars, and a lot of other sci-fi ideas fall into this group.
   
2. Obviously "good" ideas that seem possible but haven't happened yet. Video phones and HDTV were in this category for a long time. I think this happens when people get excited about technology and overestimate the benefits (and possibly underestimate the cost). I just don't care that much about having a video phone.
   
3. "Bad" ideas. Many of these ideas are truly bad, but some of them will in hindsight turn out to have been very good ideas. I put them in the same category because they are difficult to distinguish without the benefit of hindsight. Some examples are the personal computer ("why would anyone need a computer?"), Google ("there are already too many search engines, and besides, search engines don't make money"), and Blogger ("can't you just use Geocities, and besides, are there really that many people with something worth saying?"). More recent (and still controversial) examples include Facebook and Twitter.
   

I expect that a lot of people will argue with the fact that I've grouped "truly bad" ideas together with "seemed bad but were actually very good" ideas. Everyone thinks that they can tell which ideas are the good ones, but observation suggests that they can't. I do believe that some people are better at picking winners than others, but at best they have maybe 50% accuracy. I've seen a lot of very smart people (such as at Google) be very wrong about these things.

For example, I remember when the "upload videos and put them on the web" incarnation of Google video was first being developed. Nearly everyone inside of Google, including me, was very skeptical that anything worthwhile would ever be uploaded. The common prediction was that it would all be movies and porn. Of course there was some of that, but the skeptics were completely wrong about the lack of worthwhile content. Uploaded video is one of the most important developments of the past several years. Unfortunately Google Video was burdened with an incredibly bad upload process (it included installing a windows client to do the upload!), and YouTube, which was started AFTER Google Video launched, took over. I believe that mistake was partially due to the negative expectations.

Here's my point: The best product ideas are often found in the "bad ideas" category!

If you're a super-genius researcher looking to dedicate your life to a really important problem that you may never solve, then the category one "cold fusion" ideas may be a good option. If you've found some clever way to dramatically reduce the cost of implementing a category two idea (like Skype did to the video phone), then that might be a good option. However, the real "low hanging fruit" is likely to be found with the category 3 "bad ideas". You'll have to deal with annoying skeptics and haters who say that you are wasting your time, but sometimes you'll create something hugely important, or at least moderately successful (and they won't).

Realizing that good ideas and bad ideas are often nearly indistinguishable, there are a few more lessons to be learned:

• Instead of endlessly debating whether an idea is good or not, we should find faster and cheaper ways of testing them. This is one of the reasons why open systems such as the Internet or market economies develop faster than closed systems, such as communism or big companies -- individuals and small groups are able to build new things without getting approval from anyone.
  
• Your product idea is generally worthless, because to most people it's indistinguishable other bad ideas. Very few people will want to buy (or steal, or take for free) your idea because they already have their own bad ideas which they like better. (There are other reasons as well, but this is one of them. The way to make your idea valuable is to demonstrate that it's not bad by building a real product -- this also demonstrates your ability to execute.)
  

Obviously there is a lot more to say about these two points, and I haven't done enough to explain or justify them right now. Maybe in another post...

Not getting health insurance is a really bad idea

Seriously. If you can possibly afford insurance, then there's no excuse for not having it. I'm especially thinking of young people doing a startup and not buying insurance in order to "save money". That's a really dumb optimization, and suggests that you aren't very good at making trade-offs. A quick search shows that in California you can get a decent plan for $77/month. If you can't find $77/month, then maybe you aren't ready to do a startup.

I'm reminded of this issue right now because I just passed someone in SF laying face down on the pavement, not moving. His bicycle was nearby and I'm guessing that he had just been hit by a car. I hope that he does ok, but it didn't look good. (people were already helping him, and the police were just a few cars back, so I didn't stop)

Even if you are young and healthy, bad things can still happen. My brother was perfectly healthy until he got cancer at age 33. When my daughter was in the hospital, it cost $12,000/day -- she was there for almost 3 months.

The simple fact is that the health care system in the US isn't setup for people who don't have insurance. If something bad happens, you don't want to be one of those people. I'm not looking to debate our system, so please don't bother. Right now, the system is the way it is.

My point is simple: Do what you can to get insurance. Now.

Quick: Unintended consequences

For some reason, I like reading about unintended consequences. It seems that reality always has more depth and complexity than we would like to admit. This is part of the reason why people who claim to have the answers, generally don't.

This comes to mind because I was just reading a little about the effects of pesticides on plants:

Pesticide residues in plants are regulated to protect human health. They are measured when plants enter the market. However, this procedure does not consider that pesticides modulate secondary metabolism in plants when they are applied. The question arises whether it is conceivable that pesticide-induced changes in the chemical composition of plants influence human health. The examples resveratrol, flavonoids, and furanocoumarins indicate that plant phenolics may have subtle effects on physiologic processes that are relevant to human health. These effects may be beneficial, e.g., due to the inhibition of the oxidation of macromolecules and platelet aggregation or by their pharmacologic properties. Depending on concentration and specific chemical composition, however, plant phenolics may also be toxic, mutagenic, or cancerogenic. The consequences of a modulation of plant phenolics on human health are complex and cannot be predicted with certainty. It may be that the modulation of plant phenolics at the time of application and not the usually low level of pesticide residues at the time of consumption is critical for human health.

Put another way, pesticides can actually alter the nutritional properties of plants by removing environmental stress, stress which would have caused the plants to produce nutrients that may be important to humans. I hadn't considered that.

Quick: Big screen development

This is a good setup for working together, and the price of 1080p televisions is dropping.

Quick: DBE

People sometimes ask what "Don't Be Evil" really means. Eventually I'll write down my thoughts, but for now this excellent post from Seth Godin explains a good part of it:

Let me be really clear, just in case. If you think that the world would be a better place if everyone owned a handgun, then yes, market handguns as hard as you can. If you honestly believe that kids are well served by drinking a dozen spoonfuls of sugar every morning before school, then I may believe you're wrong, but you should go ahead and market your artificially-sweetened juice product. My point is that you have no right to market things you know are harmful or that lead to bad outcomes, regardless of how much you need that job.

Along the way, “just doing my job,” has become a mantra for blind marketers who are making short-term mistakes in order to avoid a conflict with the client or the boss. As marketing becomes every more powerful, this is just untenable. It’s unacceptable.

If you get asked to market something, you’re responsible. You’re responsible for the impacts, the costs, the side affects and the damage. You killed that kid. You poisoned that river. You led to that fight. If you can’t put your name on it, I hope you’ll walk away. If only 10% of us did that, imagine the changes. Imagine how proud you’d be of your work.

There are a lot of worthwhile things in the world -- why waste your life working on something evil?

The dogmatic programmer - when software becomes religion

Do you believe that there is only one "right way" to do things? Do you ignore evidence that other solutions may also work? Do you dismiss the possibility that there are reasonable trade-offs to be made between the "right way" and some other way? (do you leave angry comments on my blog?)

Do you think I'm kidding? Read these comments about GET vs POST. There's no question that POST is the "right" way and generally safer, but sometimes it's annoying. Even though GET isn't "supposed" to work, it often can be made to. Don't believe me? Google does billions of dollars a year in GET based transactions in the form of CPC ad clicks (which can cost over $50/click). A lot of other things work this way too, including most non-ajax webmail (for changing read/unread state, at the very least).

If you're writing missile guidance software or something then, please, please, please do things the right way -- it's better safe than sorry, right?

However, if you're writing little web apps, then there's a good chance that you can bend a lot of the rules and produce a better product sooner. I'd rather have something imperfect but useful and popular than something "perfect" but unfinished and unused. Sometimes, it's better sorry than safe.

When is it safe to bend the rules? Just ask yourself, "What's the worst that could happen?" If the worst case isn't too bad, then you're probably ok. Minor bad things will happen no matter what, so it's often better to put your energy into general problem recovery, rather than imagining that you can simply avoid all problems. (this is especially true on the web)

Quick: Vibe

Seth Godin on "the vibe":

Have you ever been at a banquet or in a boutique or at a concert or a meeting or a company where the vibe was incredibly positive?
...
If vibe is so important, why does it sound flaky to worry about it? Who's in charge of the vibe at your place? Could it be better? A lot better?

This is a very real and under-appreciated part of work. It was one of the first things I noticed when I started at Google after working at Intel -- the place just felt so much more alive. Now it's something that I watch for when visiting startups -- my guess is that upbeat and energetic companies will outperform the ones that feel oppressive and hopeless.

Quick: Wolves

A great story from Steve Yegge. It sounds like Steve recently realized that there are wolves at Google. Can you see the wolves in your organization? How are you protecting yourself? (or are you being eaten alive?)

"Have you ever noticed that all our folklore about wolves has certain themes and patterns to it? I've never met a wolf, but I give them fair credit for being wily creatures. I'm a big man, not one to fear a big dog's bite, but I would give a wolf a wide berth, because they're reputed to be so crafty. And crafty, friends, I am not. A craftsman, yes, but I realize now that I am more sheep than wolf.

"The landowner in my tale is a good, affable fellow; I could see it within minutes of meeting him, and I stand by him today still. He's a man with a clear vision and a good heart, and he hired crew members who, for all our talents, know nothing of wolves or wizards.

"The problem with setting up an untended, untrained flock of sheep is that there are wolves aplenty, and as sheep are not adequately prepared to recognize or deal with wolves, they will all eventually be eaten. This seems obvious to us. Nearly as obvious is that this idea applies equally to government, with the minor change that the wolves are savvy politicians and the sheep are incompetent ones. Eventually the sheep are all eaten, so our assumption is generally that the government is filled entirely with wolves — an assumption that has proven to be essentially axiomatic, across time and nation.

"What's far less obvious is that the idea applies outside flocks and governments, with the same consequences. If you set up any organization of saintly do-gooders, with no politics anywhere in sight, then the presence of a single master politician can go unrecognized for long enough for every single sheep to be eaten and replaced with a wolf. At some point the shepherd looks down and perceives that he now has a flock of wolves, and he wonders how it happened. But a shepherd rarely witnesses the process while it's in motion, because the wolves are so sneaky.

"I was being eaten alive on my mansion project, and yet I was utterly oblivious to anything but the need to survive, which due to various odd circumstances required working without sleep and eating marshmallows without respite, apparently until either I was dead or forever happened, whichever came first.

Email blackmail is unnecessary

From Slashdot:

Under the guise of fighting spam, five of the largest Internet service providers in the U.S. plan to start charging businesses for guaranteed delivery of their e-mails. In other words, with regular service we may or may not deliver your email. If you want it delivered, you will have to pay deluxe.

It's tempting to think that such schemes, although distasteful, are necessary to fight spam. They are not.

A good reputation system can be used to provide reliable delivery to high-volume, non-spamming senders, and it can be done without punishing non-profits, mailing lists, startups, or others who can't afford to pay extra delivery fees.

How does that work? Brad Taylor of Google published a paper about the Gmail reputation system, which is quite effective (pdf or html).

... spammy domains get nicely clustered around a reputation of zero. We know these domains are spammy because if our users disagreed with us, they would unmark spam on them and the reputation would no longer be zero. Nonspam domains are more loosely clustered in the 90-100 range. There ends up being a smattering of domains in be- tween. Many of the messages in the in-between category are from legitimate bulk senders, and the lower than 90 percent reputation is a reflection of less-than-ideal sending practices. But not all bulk senders are the same. Some have very high SPF and DomainKey reputations. For example, eBay's DomainKey reputation is 98.2, enough to be whitelisted by Gmail. This is the "holy grail" of bulk sending, and it is all done without requiring any payment or extra effort on the part of the sender other than just having good mail hygiene.

Unfortunately Google does not currently publish this reputation information, though I'm hopeful that they will eventually. (but I know that the team there is already hard at work on a number of other important projects)

Optimizing everything: some details matter a lot, most don't

Optimization is something that all engineers and product designers (all people, really) should understand. However, their words and behavior suggest that most either don't understand it, or haven't really internalized the lessons of optimization. (and by the way, I'm talking about CS-style "make it better" optimization, not "make it optimal" optimization)

Like everything, optimization is a complex topic, so I'm going to simplify it into three easy steps:

1. Find the problems ("profiling")
   
2. Sort the problems according to benefit-of-fixing/cost-of-fixing ("ROI")
   
3. Fix them, in the order of highest benefit/cost to lowest
   

Most programmers probably think of optimization primarily in terms of CPU profiling, but it really applies to everything. (and in fact, this post is about the general process of optimizing, especially for things other than code -- for specific cases of hardcore game engine optimization or whatever it might be totally wrong)

For example, people who continually "pinch pennies" (perhaps driving around town to save a few cents on gas) but then waste thousands of dollars on something stupid like a fancy car or credit card interest are probably not optimizing their spending very well. They are putting most of their effort into low reward activities (saving a few cents on gas) while losing much more money in other places. Big, dumb, Dilbertesque companies are also notorious for this kind of thing, cutting back on pencils while running super-bowl commercials, or something.

This is also the kind of activity that I was talking about in my post titled "Wasting time on things that really don't matter". It's not that there is zero difference between two slightly different ways of testing for the empty string, it's just that your product probably has 100,000 more serious problems, and so time spent worrying about the empty string style is time NOT spent fixing those more important problems. Put another way, fix all of the other problems first, then debate the empty string.

Of course you may enjoy debating the empty string (and in fact there are some interesting arguments), and that's fine. The zeroth step in optimization is to decide what you are optimizing for. If you're looking to maximize the amount of time spent on endless debates, or perhaps code "beauty", then maybe the empty string debate really is the highest benefit/cost activity (because the benefit is so high). If, on the other hand, you are trying to maximize the success of your product, then the empty string debate probably has negative value and should be strongly avoided.

The empty string comparison is obviously a small detail, and it's tempting to think that we should not bother with such small details, but that too is a huge mistake. Another important lesson of optimization is that some small fraction of the details will determine the majority of the outcome. This is commonly referred to as the 80/20 or 90/10 rule, stating for example that "90% of the time is spent in 10% of the code", but it could just as easily be 99/1 or something like that (and for code that has already been well optimized, this 80/20 rule may not hold -- it's just a heuristic). In code it may be just a few lines in the "inner loop", or perhaps it's the interest rate on your mortgage ("what's a few percent?"). I once profiled a server at Google and discovered that something like 90% of the time was being spent on a single line of code: a call to sscanf (repeated millions of times) that someone had snuck into the scoring code (that's the 99999/1 rule).

Determining which details matter a lot (and which don't) is also the key to designing successful products. In the classic slashdot iPod post from 2001, they summarized the newly released iPod thusly: "No wireless. Less space than a nomad. Lame."

History suggests that those weren't the important details, since it's 2007 and there still isn't an iPod with wireless (not until the 29th, at least), and most iPods have less than the original 5GB of storage. Apple did, however, get a few very important details right: it's easy to get your music onto an iPod, it holds "enough" music (unlike the early flash players), and it fits easily into your pocket (unlike the early disk-based players).

So how do we determine which details are important? Observation. (and some judgment, but I think that's mostly formed via observation) If you're optimizing an application, observe which resources are being exhausted or causing the delay (it's not always CPU -- it's often disk seeks), and then observe where they are being consumed. If you're trying to optimize money, first observe where most of the money goes to and comes from. To optimize a product design, observe which things are most "useful", and which are most painful or annoying (waiting or thinking, usually).

If you get all of the important details right, you'll be so far ahead of everyone else that the other details probably won't matter. But remember, your time is finite, so time spent on unimportant details is time not spent on important details! Most engineers are very detail oriented people, and this is good because they can really focus on the important details, and it is bad because they can really focus on the unimportant details! I suspect that learning to tell the difference will be worth your time.

By the way, to get even more details right, decrease the cost side of the benefit/cost equation so that you can do more things in less time (by finding easier solutions, avoiding endless debates, etc).

Java running faster than C

Note: A lot of people seem to be taking this post to be the "Ultimate C vs Java shootout". It's not. Performance is a very complex topic. My only real point is this: Java (which used to be slow) has reached the class of "fast languages". For the majority of applications, speed is no longer a valid excuse for using C++ instead of Java.

I just saw this page comparing the performance of several languages on a simple Mandelbrot set generator. His numbers show Java being over twice as slow as C, but then I noticed that he's using an older version of java and only running the test once, which doesn't really give the JVM a chance to show off.

I quickly hacked up the code to run 100 iterations 3 times and then used my standard "go fast" flags (there may be better flags, but I'm lazy). Here are my results:

$ java -server -XX:CompileThreshold=1 Mandelbrot 2>/dev/null
Java Elapsed 2.994
Java Elapsed 1.926
Java Elapsed 1.955

$ gcc -O8 mandelbrot.c
$ ./a.out 2>/dev/null
C Elapsed 2.03
C Elapsed 2.04
C Elapsed 2.05

C still wins on the first iteration, but Java is actually slightly faster on subsequent iterations!

Obviously the results will be different with different code and different machines, but it's clear that the JVM is getting quite fast.

This test was run with Java 1.6.0-b105 and gcc 4.1.2 under Linux 2.6.17 under Parallels on my 2.33GHz Core 2 Duo MacBook Pro. Here is the hacked up code: Java and C.


For extra fun, I also tried running the JS test using the Rhino compiler:

$ java -cp rhino1_6R5/js.jar -server -XX:CompileThreshold=1 org.mozilla.javascript.tools.shell.Main -O 9 mandelbrot.js 2>/dev/null
JavaScript Elapsed 21.95
JavaScript Elapsed 17.039
JavaScript Elapsed 17.466
JavaScript Elapsed 17.147

Compiled JS is about 9x slower than C on this test. If CPU speed doubles every 18 months, then JS in 2007 performs like C in 2002.


Update: A few more cpp flags have been suggested. -march=pentium4 helps a little, but it's still slower than Java.

$ gcc -O9 -march=pentium4 mandelbrot2.c
$ ./a.out 2>/dev/null
C Elapsed 1.99
C Elapsed 1.99
C Elapsed 1.99

Adding -ffast-math puts C in the lead, but I'm not sure what the downside is. The gcc man page says, "This option should never be turned on by any -O option since it can result in incorrect output for programs which depend on an exact implementation of IEEE or ISO rules/specifications for math functions." That sounds like an optimization that Java might not use.

$ gcc -ffast-math -O9 -march=pentium4 mandelbrot2.c
$ ./a.out 2>/dev/null
C Elapsed 1.66
C Elapsed 1.67
C Elapsed 1.67

Update: Several people have claimed that the performance difference is due to fputs (including the top rated comment on reddit, aumusingly). That is not correct, at least not on my computer. I tried replacing the print calls with a trivial function (but with side-effects), and it actually helped the Java more than the C:

C Elapsed 1.88
Java Elapsed 1.554

Many people have pointed out that '-O8' is more than enough 'O' levels. I know, and I don't care -- it's just as good as '-O3' or whatever.

Wasting time on things that really don't matter

It's easy to waste a lot of time debating things that aren't very important. Maybe that's fun if you don't have anything better to do, but when you're actually trying to accomplish something it can be deadly.

This post on "coding horror" is a great example of this phenomenon, or more specifically, the comments are a great example. At least half of the comments are from people who are debating which of these two lines of code are better:

if (s == String.Empty)
if (s == "")

Here's my answer: It's not important!

It's easy to get dragged into these debates because we all have opinions (and the one form really is better!), and of course the more time is spent talking about the issue the more important it seems.

Here's my trick for killing these stupid debates: Let's list all the issues and problems that we are facing right now. Now, where does this issue rank on that list? Is it in the top 10? The top 100? The top 1000? Are we spending more time talking about this one minor issue than we are spending working on any of our top ten issues? (You don't necessarily have to make the list -- just suggest that it exists and then guess where on the list the issue appears)

Sometimes that works. Sometimes it brings a little perspective.

And yes, this misallocation of time (aka "worrying about dumb stuff") is probably a top 10 productivity killer.


Update: Here's a good link (via news.yc) which offers one explanation for why people often spend the most time on the least important issues:

Parkinson shows how you can go in to the board of directors and get approval for building a multi-million or even billion dollar atomic power plant, but if you want to build a bike shed you will be tangled up in endless discussions.

Parkinson explains that this is because an atomic plant is so vast, so expensive and so complicated that people cannot grasp it, and rather than try, they fall back on the assumption that somebody else checked all the details before it got this far. Richard P. Feynmann gives a couple of interesting, and very much to the point, examples relating to Los Alamos in his books.

A bike shed on the other hand. Anyone can build one of those over a weekend, and still have time to watch the game on TV. So no matter how well prepared, no matter how reasonable you are with your proposal, somebody will seize the chance to show that he is doing his job, that he is paying attention, that he is *here*.

Update Two: Wasting time is fine, and even inane discussions of the best way to compare empty strings can be interesting. My real point is this: if we are actually trying to get something done, then we need to kill this stuff, or it will kill productivity. If, on the other hand, we're ok being in time-wasting-mode, then silly debates are probably harmless enough. I picked on the blog comments above not because there is anything wrong with them per-se (what are blogs for, if not inane debate?), but because it reminded me a lot of the silly debates that I've seen on work mailing lists (where they do cause harm).

Amazingly bad APIs

I actually don't hate the Java language (at least it's killing off c++), but every time I use it I'm blown away by how hostile the APIs are.

For example, let's say that we want to scale an image file. That's a fairly common thing to do, so it's probably only one line of code, right? Wrong. It turns out that they haven't even bothered to include a reasonable API for doing this. Instead, you have to take the awkward APIs that they do provide, and then write about 50 lines of code to do what you really want. This article that explains what the "right" way is.

Once the image is scaled, you'll want to save the smaller version too. Luckily, that's only a few lines of code. Of course that's only if you're happy with the default settings -- what if you want to use a different JPEG quality level? Here are the docs -- see if you can figure it out (the solution involves 4 additional classes).

For comparison, here is some Python code which loads an image, scales it, and then saves it at a non-default quality level:

from PIL import Image

i = Image.open("/tmp/c.jpg")
i.thumbnail([220, 133], Image.ANTIALIAS)
i.save('/tmp/c-thumb.jpg', quality=90)

Doing the same thing in Java requires closer to 100 lines, not because the language is bad, but because the APIs are terrible.

How did this happen? My theory is that the people who made the Java APIs just sat in a room making stuff up (and possibly drawing UML diagrams). Meanwhile, the Python people were actually writing software and creating libraries to make their life easier.

There is a lesson in this: If you are building a platform, you should also be writing applications for that platform, and the platform should be designed to make life very simple for those apps. It's my understanding that Ruby on Rails came out of a process such as this (they extracted the common tasks out of their web apps). If you just sit around staring at the clouds while writing your APIs, you'll probably end up with something bad.

I also have a second theory about what happened to Java: it attracted people who like to write piles of code filled with endless abstractions. When I worked at Google, one of the engineers on our Java mailing list suggested that we ban the keyword 'new'! I was puzzled by this suggestion, but after a little research I learned that one of the hot fashions in Java is to use factories instead of 'new' (not that factories are bad, but "sometimes good" does not imply "always best"). Better yet, you don't reference the factories directly (not enough abstraction), instead you have some framework that injects the factories into your classes according to what is written in an XML configuration file. In this way, you can make your code perfectly unreadable, but more importantly, it is very abstract.


Update: Several people have interpreted this post to mean that I'm opposed to the factory pattern. That is, of course, silly. Factories, when the situation calls for them, are fine. For example, the python code above is using a factory. What harms code is the blanket application of the factory pattern in every possible situation. To put this another way, Tabasco sauce is delicious on eggs, but I still don't put it on ice cream. (but don't think I haven't tried!)

Update Two: It's not about Java, really! My point here is that your APIs should emerge from the needs of real applications, and that they should make common tasks super-easy. Bad APIs are common in many languages, Java just happens to be the language that I'm using right now.

Why didn't I simply hunt down+install some third-party image library? Because that's a hassle too (probably more work than copying code off of some web page), and all I want is to generate nice looking thumbnails.

Flash, Silverlight, and JavaFX all look nice, but what I really want is better file upload

One of the most annoying limitations when writing web apps comes from the <input type="file"> form element, which is the standard way of uploading files.

Here are a few of the problems with <input type="file">:

• You can only select a single file per input element, and there is no way to select entire folders.
  
• Web browsers just pop-open the standard file open dialog, which is often not very good. For example, the dialog probably just shows a list of files, not thumbnails. This can make it difficult to pick the correct file if they all have names like IMG_3485.JPG, and it's therefore easy to upload the wrong file.
  
• There is no way to actually read the data locally without first sending it to the server and then having the server send it back.
  
• There is no way to do any local processing on the data, such as scaling images to make them smaller.
  

Flash apparently solves the multiple file problem (kind of), but it's my understanding that it still has all of the other problems (I'd love to be wrong about this though).

Java applets can actually do most of these things if they are signed and the user agrees to allow the applet access to their filesystem. Unfortunately, Java applets don't actually work, and after 11 years of not working, I'm not expecting that to change. Facebook has a nice image uploader applet that lets you browse your photos locally and then scales them before uploading to make the whole thing go fast. The only problem, other than the annoying "do you want to trust this applet" prompt, is that it crashed my browser the second time I tried it.

To me, the great thing about flash is that it fills in basic capabilities missing from the browser, such is video, sound, camera and microphone access, and they did it with a reasonably small and stable browser plugin that I'm not afraid to allow onto my computer. A few months ago, I accidentally upgraded one of my computers to Flash 9 -- I don't remember how exactly, I just remember that it was simple and non-eventful.

Here's my idea: I want to create a small, reliable browser plugin that does for file uploads what flash does for audio and video. It should work with all the modern browser/os combinations, and install with minimal effort and fuss (browser restarts avoided if possible). It would be free and open source, but hosted in a common location for easy install.

Popular Ajax libraries such as jQuery, YUI, and Dojo can add support in such a way that people who don't have the plugin just get the regular browser upload control (and maybe a link to install the plugin), while those who do have the plugin get a better ui. Clearly, getting people to install a new plugin isn't easy, but once any site convinces them, their experience automatically gets better on all other sites that support the plugin. There isn't a chicken&egg problem because the plugin simply provides an enhancement to the already existing (but very poor) browser functionality, so sites can easily add support without fear of losing people who don't have the plugin (unlike building a whole app in silverlight, or whatever).

I'd like to hear from anyone with experience writing browser plugins. How difficult is the technology side of this? (making it small, reliable, and stable -- crashing browsers is not ok) Also, are you interested it writing this plugin? I'm serious about this, and willing to pay quite a bit of money to make it happen.

Here are a few more details on how I imagine the plugin working:

It would not have any in-browser UI -- it would simply be an object that can be called from javascript and that would pop open a nice file browser with thumbnails and all that (similar to windows explorer, perhaps). It would also provide upload-related utility functions, such as image resizing. Since the file access is read-only and limited to what the user selects, security shouldn't be a big problem, assuming of course that the plugin implementation is secure (and it must be).

Here is some JS illustrating how I imagine using this plugin (UploadHelper is the object provided by the plugin):

// Have the user choose files or directories to upload.
// Returns undefined if user clicks Cancel. Doing this with a
// callback instead of a return value would also be ok.
var files = UploadHelper.chooseFiles(
              {filter:'*.jpg,*.gif,*.png', allowDirectories:true});
if (files) {
  var s = 'You selected: ';
  for (var f in files) {
    var file = files[f];
    s += (file.name + " (" + 
          file.imageinfo().width + "x" +
          file.imageinfo().height + ", original size = " + file.size);

    files[f] = file.scaleImage({maxwidth:100, maxheight:100, quality:5});
    s += " scaled size = " + files[f].size + ") ";

    // If I could somehow insert this image on the current web page
    // too, that would be really awsome.
  }
  alert(s);
  
  // using jquery XMLHTTP wrapper
  $.ajax({
    type: "POST",
    url: "/upload",
    data: UploadHelper.encode(files, 'multipart/form-data'),
    contentType: 'multipart/form-data'
  });
}

var files = UploadHelper.chooseFiles({filter:'*.txt'});
if (files) {
  // We can also read the content of the selected file
  // The ability to get chunks of a file also means that it's possible
  // to restart failed uploads, and to chunk large files into multiple
  // POSTs.
  for (var f in files) {
    var file = files[f];
    var d = file.getData(0, 50); // get the first 50 bytes
    if (file.size > 50) d += "...";
    alert(files[f].name + ": " + d);
  }
}

Update:

Several people have suggested that I should instead try to get this feature added to web browsers or (better yet) the web browser standards. That's a fine long-term solution, but realistically it will take years to happen. I would rather have an immediate (months, not years) fix in the form of a plugin, and then let the web browsers add it as a standard feature on their own schedule. This is somewhat similar to the path followed by XMLHTTP.

Sriram Krishnan of Microsoft makes a few good points:

• The standard file open dialog in windows already has a thumbnail view. Unfortunately, this feature is hidden under a completely unremarkable icon and despite having used windows for many years, I've never seen it before today. This is a great example of why simply adding a feature isn't good enough -- it also needs a good ui or else it may as well not even exist. Furthermore, not existing seems to be the status of the thumbnail view on OSX -- hopefully that will be fixed in 10.5.
• This feature request could be implemented in Silverlight. That's great news! MS should write and release a little Silverlight control that provides these functions to JS. I would be very hesitant to write (or rewrite) my apps in Silverlight, but if I could simply embed a little Silverlight module the same way that I would a flash mp3 player, then I'd definitely use that. This would be a very effective way for MS to get people started installing and using Silverlight (start with something small and optional).

Silverlight sounds pretty nice. Google or Adobe had better have an open alternative on the way, or else MS may be able to recapture the application platform.

Update Two:
I believe that within a few months we will have a few reasonable solutions to this problem!

When people don't know how to change something, they often start searching for a way to justify failure, rather than thinking about how they could try doing something different to make it work.
...
If we pretend that you can teach anyone anything, we'll find out where it's not (yet) true. But if we think that when someone isn't learning it means they can't be taught, no one will even try.
...
If you take the attitude that anything is possible, you'll find that a lot of things that were previously thought impossible actually do become possible.

Page 125

Beliefs, intelligence, and failure

From the book "Hard Facts, Dangerous Half-Truths, and Total Nonsense" (via this blog, via an email from a friend):

A series of studies by Columbia University's Carol Dweck shows .... that when people believe they are born with natural and unchangeable smarts ... [they] learn less over time. They don't bother to keep learning new things.

People who believe that intelligence is malleable keep getting smarter and more skilled ... and are willing to do new things.

That's a pretty important fact. It's also an excellent example of how your success is determined by your beliefs/mindset/frame. If you believe that you can't get smarter, you won't!

I searched around a bit and found another interesting quote:

...beliefs about the nature of intelligence have a significant impact on the way they approach challenging intellectual tasks: Students who view their intelligence as an unchangeable internal characteristic tend to shy away from academic challenges, whereas students who believe that their intelligence can be increased through effort and persistence seek them out.

Students who hold an "entity" theory of intelligence agree with statements such as "Your intelligence is something about you that you can't change very much." Since they believe their intelligence is fixed, these students place high value on success. They worry that failure-or even having to work very hard at something-will be perceived as evidence of their low intelligence. Therefore, they make academic choices that maximize the possibility that they will perform well. For example, a student may opt to take a lower-level course because it will be easier to earn an A. In contrast, students who have an "incremental" theory of intelligence are not threatened by failure. Because they believe that their intelligence can be increased through effort and persistence, these students set mastery goals and seek academic challenges that they believe will help them to grow intellectually.

Not only does a belief in fixed intelligence prevent learning, it also makes people more risk averse! It's easy to see how some cultures or micro-cultures make people successful, while others keep them mired in failure.

I expect that this same basic principle applies to most things, whether it's skills with computers or skills with people. Obviously "nature" plays some role, but beliefs, knowledge, and determination are usually more important. Furthermore, I suspect that you can learn to enjoy these activities, and that finding the fun is the key to mastering the skill.

And you thought all the good domain names were taken?

Using only the magnetic letters stuck to my refrigerator, we were able to discover these promising, yet unregistered domain names!

spitbomz.com
xombiz.com
lisp.tv (imagine the media empire that could be built around that!)
xzombi.com
QykDr.com (quick doctor)
nevrqyk.com
WakiDr.com
FuziXombi.com
phuzix.com
mowbul.com
t-mubal.com
WakyXombieFujPHD.tv


And venturing beyond the refrigerator, we also found these gems:
un4tun8.com (I went ahead and registered this one)
4nic8.net
t-mobull.com
grrrrrrrr8.com (that has eight 'r's)

There's a goldmine in replacing 'for' with '4' and 'ate' with 8.

Don't limit yourself to good ideas...

Whose reality are you living in? Whose reality would you rather live in?

Mental frames are the biased and limited way in which information is perceived or understood. Because the human brain is inherently biased and limited, we are always in some mental frame. That frame determines how we relate to and understand reality. As far as you can tell, that frame is reality.

This is why the world seems better when we are happy, and worse when we are sad. It is also a reason that people on "the other side" of an issue seem so stupid, misinformed, and out of touch with reality. Not surprisingly, mental frames are often discussed with respect to politics. However, their scope and importance goes well beyond that -- they ultimately determine not only our perceptions, but our whole mindset about what is valuable, practical, or dangerous, and what behaviors are responsible and acceptable.

If someone pursues their passions, are they boldly living life to the fullest, or are they simply being frivolous and irresponsible? The answer, of course, depends on your frame.

Perhaps you might prefer to be objective about reality and escape these limiting frames. Unfortunately, that's not really an option, at least not while we are stuck with these monkey brains inside our heads. Although it is nice to attempt objectivity, we must accept that we are human, and therefore limited. To deny that and claim true objectivity is to deny the truth and be stuck in a very limiting and annoying frame.

Mental frames help to explain why some people needlessly stay in bad jobs, bad relationships, or other bad situations -- in their reality, it makes sense. Clearly, some frames are better than others (from my perspective :).

Here's the good news: You can switch frames!

Many influences can shift your frame, such as reading books or taking a walk in the park, but one of the most powerful influences is the people around us. We tend to synchronize frames with our friends, family, co-workers, and anyone else we encounter (including the people on tv), though obviously some of those people are more influential than others.

Therefore, if you want to change your reality, change your surroundings. Find people with a more attractive reality, and live with them. This is very important. When you spend a lot of time with people, their reality becomes your reality.

For example, if you are interested in startups, but work in a big company, you are in danger. If you stay there too long, you will be drawn into the big company frame shared by the other "lifers". Startups will all seem too risky, frivolous, or impractical, and you'll spend the rest of your life in that big company (and posting bitter comments on TechCrunch).

Similarly, if you dream of pursuing some other career or lifestyle that is not shared by the people around you, then you either need to accept that it's not going to happen, or you need to change your situation.

Please note: I'm not suggesting that you completely cut off contact with people outside of your desired frame (that's how cults operate, btw). To the contrary, it's good to keep in contact with a wide variety of people -- that will help provide perspective and keep you from becoming trapped in your new frame (which might not be as great as it seemed from the outside). What I am suggesting is that your target frame should have significant representation in your life (like 50%), or at the very least, some minimal representation (5%) so that it doesn't completely fade from your reality.

Update: To better understand frames, think of a time when you were excited about an idea or possibility (or anything else), but when you shared that thought with someone else, they somehow ridiculed, doubted, or otherwise criticized it. How did that make you feel about the idea? Were you a little less excited, more doubtful, or perhaps even somewhat embarrassed about it? If so, you've just entered further into their frame -- their reality is becoming your reality. If you want to nurture your dreams, it's better to share them with people whose frame is compatible with the dream.

Read my previous post on reality. What am I doing to your frame?

Update Two: One of the commenters on news.yc suggests that if everyone really were in their own frame/reality, then it would be impossible to communicate or build products for other people. This is a good point, and it would be true if our frames were completely disjoint. Fortunately, they are not -- we always have something in common. However, the more our frames differ, the more trouble we have communicating. This is why it can be so difficult to communicate with a broad audience, such as on a blog, and why I've decided to write primarily for those with similar frames (because it's easier).

For more thoughts on how your mental frame affects your life, there are some interesting posts on Steve Olson's blog (though he uses the term 'belief system').