Joshua Go

Ways to modularize your Ruby code

2010-08-08T16:00:00.000-07:00

In this post, I'll recommend several ways to properly organize code in Ruby projects. I'll also explain my reasoning and how I arrived at each solution.

Much of this builds on the excellent work of others in the Ruby community, and I've linked to these other writeups as appropriate in case you want to know more.

This subject is of particular interest to anyone who is packaging a gem library, but it should be handy to anyone who wishes to organize code within any Ruby project.

Namespacing with modules

Let's say you've got an isolated piece of functionality that doesn't depend on anything else, such as a simple key (random string) generator. You intend to always call it with its full namespace, so you don't get it mixed up with similarly named functions. To make something that you can call using Rigatoni::KeyGenerator.generate_key(n), you'd use the following code.

module Rigatoni
  module KeyGenerator
    def self.generate_key(key_length = 5)
      puts "generate_key() called with length #{key_length}"
    end
  end
end

Note that I defined the function as self.generate_key() — the self keyword is crucial.

That's a subtle but critical difference. If I didn't include the self,
I'd have to include Rigatoni::KeyGenerator first and then run the
generate_key() function, which isn't what I want to do.

If I want to save myself a little typing while having some semblance of a namespace to qualify my method call, I can do this:

include Rigatoni
KeyGenerator.generate_key()

This is the best way to modularize a self-contained piece of Ruby code that's meant to be called independently.

Extending functionality through modules

The most common way in which I've seen Ruby modules used is to extend the functionality of existing classes. This is where they're used as mixins to extend a Ruby class.

Even then, there are two ways to extend the functionality of a Ruby class with modules: instance methods and class methods. By default, including a module in your class definition will give you new instance methods.

If you want to define class methods in a module, you have to jump through some extra hoops.

module Moo
  module Ham
    def self.included(base)
      base.extend(ClassMethods)
    end
    
    module ClassMethods
      def foo()
        puts "foo called"
      end
    end
    
    def bar()
      puts "bar called"
    end
  end
end
    
class Bacon
  include Moo::Ham
end

The module we define above is Moo::Ham. We have a dummy class, Bacon, which includes Moo::Ham. It includes both class and instance methods, which we can run with the following example code.

# Class method.
Bacon.foo

# Instance method.
b = Bacon.new
b.bar

This is a longtime Ruby idiom, and John Nunemaker unpacks this in his post, Include vs. Extend in Ruby.

Namespaced classes (for state-dependent modularity)

In the examples up until now, we've dealt only with methods that could be run independently without needing something already in place.

Let's say all you're using to modularize your code is Ruby modules. Then you start to notice that a lot of the methods have the same argument passed in. Either that, or you find yourself setting up or populating some variable again and again in order to perform the work.

module Moo
  module Ham
    def some_func_foo(access_key, api_key, x)
      # Set things up with access_key and api_key.
          
      # Perform work with x.
    end

    def some_func_bar(access_key, api_key, y)
      # Set things up with access_key and api_key.
          
      # Perform work with y.
    end

    def some_func_baz(access_key, api_key, z)
      # Set things up with access_key and api_key.
          
      # Perform work with z.
    end
  end
end

When you start to notice these things, it's time to turn your module into a class.

module Moo
  class Ham
    def initialize(access_key, api_key)
      @access_key = access_key
      @api_key = api_key
          
      # Do other stuff here to set up what you need.
    end

    def some_func_foo(x)
      # Perform work with x.
    end

    def some_func_bar(y)
      # Perform work with y.
    end

    def some_func_baz(z)
      # Perform work with z.
    end
  end
end

This keeps us from duplicating code. Note that we end up having to instantiate a class because the methods depend on the initial setup work being done, but our code is leaner and meaner this way.

# Old way. Gross.
Moo::Ham.some_func_foo(access_key_one, api_key_one, x)
Moo::Ham.some_func_bar(access_key_one, api_key_one, y)
Moo::Ham.some_func_baz(access_key_one, api_key_one, z)

# New way. Nice.
mh = Moo::Ham.new
mh.some_func_foo(x)
mh.some_func_bar(y)
mh.some_func_baz(z)

The major takeaway is that modules are not the only way to modularize our Ruby code.

Handling dependencies

Other times, you'll have methods that aren't state-dependent and which don't belong in a class. But they'll have a different kind of dependency: on external gem libraries being present.

Say you have a module, Foo, which you define in a file called foo.rb.

# foo.rb
module Foo
  ABACAB="abacab"

  def print_foo
    puts "foo"
  end
end

Then say you have another module, Bar, which depends on Foo.

# bar.rb
module Bar
  def self.bar
    require 'foo'
    include Foo

    puts ABACAB
    print_foo()
  end
end

You try to be a good citizen by pulling in Foo only when you need it. So now you're ready to run Bar.bar() from another script, run_bar.rb.

# run_bar.rb
require 'bar'

Bar.bar()

But then you run it, and you get an error with baffling and mixed results. The line referencing the constant ABACAB ran perfectly fine; the call to print_foo() failed. Let's move the require and include of Foo outside Bar's module definition and see what happens.

# bar.rb
require 'foo'
include Foo

module Bar
  def self.bar
    puts ABACAB
    print_foo()
  end
end

When we run run_bar.rb again, we see that this works for us.

It seems a little wasteful to pull in Foo, but given the results of our little experiment here, we've got no choice: if we want to call methods in the Foo namespace, we've got to pull in Foo at the top — outside the module definition and outside the method definition.

Besides, if we're pulling in bar.rb, our real intent is to go after the full functionality that Bar provides. The functionality that Bar gives us depends on Foo anyway, and won't work at all in its absence. We're not really being wasteful.

Summary

There are two major ways to modularize our Ruby code: modules and classes. Despite the name, modules are not the only way to modularize. Use classes if the proper behavior depends on state.

Start with a module by default. If you find yourself creating too many methods that take in the same parameter again and again, turn your module into a class which populates the initial values when you instantiate it.

When a module depends on other libraries, pull these other libraries in at the top of the file, outside the module definition. Calls to methods in these libraries won't be found otherwise.

JavaScript: Checking for undeclared and undefined variables

2010-07-14T13:59:00.001-07:00

Up until now, the JavaScript I've written has typically been in pursuit of a bigger goal. This means that I worked around language quirks on the spot and moved on. I never stopped to consider the nitty gritty details of the language for commitment to long-term memory, because I wanted to get a working end product.

Recently, someone asked me about the best way to check for undefined JavaScript variables. I found myself at a loss, which was alarming to me since doing these kinds of checks is immensely important, practical, and a frequent fact of everyday programming no matter what programming language you're using.

The fact of the matter is that most of the time, I know where my variables are coming from. Something like this would work:

    var x;

    if (x)
        x.some_method();
    else
        alert("we can't use x");

In the vast majority of cases where I had to perform a null check, I knew that the variable was declared somewhere because it was of my own making.

What if you're counting on something having been declared somewhere else and being there? Let's try this.

    if (y)
        y.some_method();
    else
        alert("we can't use y");

In this case, you should get a ReferenceError thrown. You'll see this if you're using the Firebug JavaScript console or the developer tools in Safari or Chrome.

After much experimenting and looking around the web, here's my definitive, reliable, robust, cross-browser way to perform this kind of check:

    if (typeof(z) != "undefined")
        z.some_method();
    else
        alert("we can't use z");

It also works for when you're the one in charge of declaring a variable, too.

    var w;

    if (typeof(w) != "undefined")
        w.some_method();
    else
        alert("we can't use w");

So that fixes our problem. But look closer, and you may have a couple of nagging questions.

Why in the world would you be referencing a variable you didn't declare? If we only ever had to deal our own code, we should always know where the variables that we're using are coming from. But that is not the case for many of us; we rely on using others' code, third-party modules, or frameworks all the time.

An easy example is if you want to send debugging output to console.log in Firebug. When Firebug is disabled, console isn't a declared JavaScript object. (Safari and Chrome have their developer tools more integrated so this problem doesn't appear.) It helps to first check if there's a debugging console available before you start writing to it.

You could also find yourself in a situation where you're expecting some object to be initialized because it's an external library. If it's loaded over the network, and loaded separately from the rest of your code, you'll want to recover gracefully if it's not available for some reason (like the remote server is down).

Why do we have to compare to the string "undefined" instead of the keyword? Because typeof() always returns a string.

    console.log(typeof(j));             // "undefined"
    console.log(typeof(typeof(j)));     // "string"

This seems too simple/short. Will it work in all web browsers? This will work in any modern web browser that supports JavaScript. Just to be thorough, I've personally verified that it works as expected in Internet Explorer 8, Firefox 3.6, Safari 4, and Google Chrome 5.0.

Checking for undefined variables in JavaScript in this way has been around for a long time, and several other sources suggest this method. I wrote this up mainly as an extended explanation for why things are the way they are, and to explicitly address quirks like typeof() always returning a string.

What if it could also be null in some cases? Then you add a check for null. I've limited the check to checking for "undefined" to keep the explanation focused, but yes, in reality you'll want to do an additional check.

    var k;

    if (typeof(k) != "undefined" && k)
        k.some_method();
    else
        alert("we can't use k");

The check for null is pretty straightforward; it's "undefined" that trips us up. Note that, as written, this code will only proceed to check for the null condition if k has been defined. This is because of short-circuiting: if the first condition is true, there's no need to evaluate the rest of the other parts of the conditions.

When it comes down to it, the key insight is that JavaScript has "undefined" variables in two senses: first, when they're not even declared, and second, when a variable has been declared but hasn't been assigned a value. This is when it really is undefined in the strict sense.

We also saw that typeof() will always return a string, which means you need to compare its result to a string when you're running a conditional test.

The status quo

2010-05-21T23:13:00.001-07:00

Sometimes there are good but non-obvious reasons for keeping the status quo. Competing interests may have, over time, been balanced and counter-balanced to form a functioning ecosystem. And as with any ecosystem, reducing the functioning whole into its constituent parts in the foolish pursuit of extracting isolated benefits is an intractable problem.

Other times, the status quo represents a deeply flawed system, fundamentally broken at the core. Such a system is characterized by people in power whose actions are driven primarily by the overriding interest in preserving their own favorable position, to the recurring detriment of the less favored.

The tough part is looking at a situation and making the call as to which one of these models applies. In some cases, the status quo may be made up of both a well-balanced system with benefits to all, as well as a rotten portion entrenched for no good reason but the preservation of the holders of power. Fixing what's broken is a separate problem, and we should be concerned with fixing the problem only after we have correctly identified it.

Slowing down as an immersive experience

2010-04-17T22:08:00.001-07:00

Through my entire time as a student, I never used Cliffs Notes or SparkNotes in place of assigned reading. I made it a point to read every book I was supposed to, down to the last word.

The trouble for me was that everyone else who resorted to these "study guides" knew enough about the real books to do well in their classes. From the perspective of efficiently using my time to reach the objective of a good-enough, basic understanding for writing essays, discussing the books in class, and impressing teachers, I lost out.

I remember one book in particular. Crime and Punishment really broke a lot of people, many of whom were like me and had, up until then, insisted on reading the book and not the summary booklet. But I was determined. I was hellbent on savoring that book and milking it for all it was worth — calculus, biology, and economics be damned.

As expected, I ended up with the same general recollection of the book's contents as the more reasonable folks who resorted to the Cliffs Notes.

But what I remember most poignantly is the feeling I had while reading that book. I became immersed in it to the point that I'd feel the cold sweat, delirium, and ever-present sense of dread that hounded Raskolnikov (the protagonist in the story) as he ran from the authorities and lived each day with the burden of his guilt.

I highly doubt that it was even possible for anyone who read the Cliffs Notes to experience that.

Whenever I approach any new text, it's in pursuit of that kind of total immersion. I know that I'm not going to remember every detail of what I read, but my ultimate takeaway from all the things I read is not the knowledge to be gleaned, as if I were some kind of one-man strip mining operation for facts. It's in putting myself in a position to be shaped and influenced; I want to see how the mind of another person works by temporarily forcing my mind into the mold of their thought process.

In order to do this, it's absolutely necessary to slow down.

Slowing down allows real life to happen, as events inject themselves into my reading experience. If I take a long enough and serious enough text and stew over it, its applicability quickly becomes apparent when I frame its ideas in the context of whatever I happen to be dealing with in my life.

On the flip side, the shortcomings of a text also make themselves apparent when I slow down and let life happen in between. The opportunity afforded by this perfect setup allows me to weed out and carefully qualify newfangled notions, because when it comes to novel and interesting ideas, I tend to give them the benefit of the doubt and adopt them a little too eagerly. A tempering influence helps, and a tempering influence provided by direct observation is the best that anyone could ask for.

Slow down to savor the richness of a text, and make time for ideas to run up against real situations. That way, you'll get to see how valid these ideas really are.

JavaScript: Variables in regular expressions

2010-02-11T07:12:00.000-08:00

Often, you want to look for a particular pattern within a string. Let's say you know that you want to look for the string "revenue" inside a given string.

function match_string_for_revenue(string_for_searching)
{
    return string_for_searching.match(/revenue/gi);
}
var our_string = "We are looking for ReVeNuE.";
alert( match_string_for_revenue(our_string) );

You can store the pattern as a regular expression in its own JavaScript variable, which makes your code more readable and its intention better known.

function match_string_for_revenue(string_for_searching)
{
    var pattern_to_look_for = /revenue/gi;
    return string_for_searching.match(pattern_to_look_for);
}
var our_string = "We are looking for ReVeNuE.";
alert( match_string_for_revenue(our_string) );

But what if you want to generalize this matching and be able to search for any given pattern? Can you make it vary based on a parameter given, instead of keeping it in the code?

/* NOT GOING TO WORK */
function match_string(string_for_searching, valuable_substring)
{
    var pattern_to_look_for = / + valuable_substring + /gi;
    return string_for_searching.match(pattern_to_look_for);
}
var our_string = "We are looking for ReVeNuE.";
alert( match_string(our_string, "revenue") );

Well, that didn't work. What happened? That last alert() should have given you a null, which means that it wasn't able to match the pattern as expected.

Are we stuck? No. We just have to work around this using eval().

function match_string(string_for_searching, substring)
{
    eval("var pattern_to_look_for = /" + substring + "/gi");
    string_for_searching.match(pattern_to_look_for);
}
var our_string = "We are looking for ReVeNuE.";
alert( match_string(our_string, "revenue") );

This method will also work with Ruby and any other language that gives you the option to evaluate the contents of a string as code in that language.

Sometimes you'll find yourself having to generate code dynamically from a given string because there's no other way but to write real code in that language in a way that goes beyond merely passing in a string. If the language doesn't support having a string in there during the course of its normal operation, that's where eval() can come in handy.

Important Update (July 14, 2010): It's important, as Matt Austin points out, to consider the security implications of using eval(). See his comment below. Additionally, coderrr rightly pointed out that there's a more proper way to do it using RegExp:

function match_string(string_for_searching, substring)
{
    return string_for_searching.match(new RegExp(substring, "gi"));
}

var our_string = "We are looking for ReVeNuE.";
alert(match_string(our_string, "revenue"));

This is the way I'd recommend doing it in the future. The specific use of eval() that triggered my idea for this post was free from the security concerns Matt raised, but seeing as how this post describes general usage, please use eval() sparingly or consider security implications of the kind Matt described.

The costs of configurable settings in your web application

2010-02-02T07:50:00.000-08:00

By and large, the received wisdom when it comes to configurability and flexible settings for web applications is that more is better. Making something configurable frees up the engineer from having to make mundane (but necessary) changes that can be left to other people. It's desirable to change the state of software without having to rebuild or redeploy the code. There are a lot of obvious benefits to making software configurable, and these benefits are plain to all.

What I'm going to focus on here are the costs of configurability, based on my experience and observations over five years of writing web applications full-time. These costs apply to other kinds of software beyond the realm of web applications. I just choose to limit the scope of discussion to web apps because these costs manifest themselves the most when there's a fast development cycle and rapid iteration.

The following is intended to be a comprehensive, itemized breakdown of the costs of making an application setting configurable by the user. By itemizing and breaking these out separately, my intention is to make it easier to count the costs. For any given web application, it will be easier to see which of these apply, and to what extent.

The up-front cost of developing it as a setting. It comes as no surprise that the up-front cost is higher and requires more time than hardcoding, and is thus the most immediate and salient concern for the lazy programmer. If you're treating it as an investment and the setting is important, then of course it will pay off later. But if you're moving quickly and pushing out new features every week or even several times a week, the up-front costs of the things you're doing will be very important.

The cost of developing visual flexibility to accompany a setting. If your setting involves anything that has an impact on the visual presentation of any pages, be sure to consider that. For example, if you want to support different widths for a column of text, it is probably in your best interest to delineate discrete acceptable values rather than allowing continuous values. You'll know how to handle widths of 100, 150, and 300, for example, and you'll have to test them out to see how they look, but you probably don't want to allow just any width. Other situations may warrant a limited range on values; these values can be continuous, but restricted within a certain range. For example, if you have a setting for the maximum length of a summary text field, you'll probably want to test how your layout handles the minimum and maximum lengths of the summary text.

The cost of attention when sifting through all the other configurable settings. Even if it's relatively easy to change a setting, your love of settings probably means you've got even more settings. If you take the simple approach in laying these out in a list of settings for you to scroll through, there's a cost you incur. The cost of paying attention to this setting in the midst of all your other settings is one reason to stop and think whether you really want it to be configurable, but it's also one way you can force yourself to be more discerning and make only the most critical things configurable.

The cost of informing all stakeholders. This becomes more important as the size of your organization grows. The part to be concerned about here isn't in letting everyone know just that the setting exists or where to find it; that's a simple matter and thus negligible. It's in telling everyone involved and making sure that they understand what the setting does, whether it has any side-effects, valid values for the setting, and whether it's a critical setting that should be changed with caution. It's surprising how easily people forget, especially if it's a rarely used setting.

The cost of determining a default fallback value. How failsafe do you want your code to be? Chances are you'll have some situations — such as bringing up a greenfield installation, restoring from backup, and database unavailability — when you won't have access to the setting. If the setting isn't available for some reason, what kind of default value do you fall back to and how do you proceed? Will this default continue to hold true and be valid as your software matures?

The cost of abstraction and the cognitive load associated with translating it to concrete results. To make some things configurable, it may be necessary to generalize and turn the setting into an abstraction. Take the problem of content management functionality on your custom webapp. You may end up having to manage embedded Flash SWFs, images, simple blocks of text, or any combination of the above. Many content management systems call these "page elements" to encapsulate their full versatility, but this kind of abstraction is another step away from merely "image block" or "text block." What's the URL for the "graphical element" — in other words, the URL for the image? In using a general tool for a specific case, you end up having to refer to things in the most general terms first and translating them into their specific names, rather than calling them just what they are. This costs you time. Time is money, and it adds up.

Now, given these costs, what is one to do? In software engineering and in politics, we resort to compromise. For example, we could hide a setting from users so that only developers or sysadmins can change it. This way, you maintain the configurability that you want and avoid many of the unnecessary costs listed above.

It may also help to know the nature of each setting: are the acceptable values discrete or continuous? If they're discrete, such as officially supported sizes for visual elements, you can impose restrictions on values that you accept so that it's harder for the users of your application to shoot themselves in the foot.

Ultimately, in any discussion of costs, no matter how focused and limited in scope, we must remember to consider the benefits. Neither the costs nor benefits considered alone give us a complete view of any situation, and each must be considered to give context to the other. The difference is that costs are difficult and unpleasant to pinpoint and isolate. They don't receive the attention they're due, but who could blame us? We prefer to look on the bright side and think instead about reaping benefits. We enjoy watching our projects and our companies grow. But it's by being aware of our costs and knowing what hampers us that we free ourselves up to build more of what we want.

The reasoning behind Lumberjack

2009-12-16T13:08:00.001-08:00

So why, in 2009, in a world of RIA frameworks, web-based applications, and a wide variety of blogging engines to choose from, would I write a desktop application in Java targeted exclusively for one company's proprietary blog platform?

First of all, it was tempting to write this as an Adobe AIR application. It would have fit my requirement that it be cross-platform and run as a desktop application, but I've never written anything with Adobe development tools before. Given the limited time I had on weekends to work on it, I wanted to get something written as quickly as possible rather than spending all my time learning a new platform. With Java, I could just hit the ground running, and it was just a matter of referencing the Swing-specific documentation. It boiled down to what was expedient and familiar because it would allow me to build something quickly.

With respect to the issue of making this application web-based, the main point is that I didn't want to start up a browser just to create new posts on Blogger. There's the Blogger Dashboard for that. Now, it's true that one could write a slimmer, lighter, faster-loading web-based client for Blogger without all the heavy clutter of the Blogger Dashboard, but it would still require that I start up a web browser; in the end, I wouldn't end up using it much. I wanted to build something that I would use and keep on using. (I have also been writing web applications for the past five years, and thought it would be fun to write something that ran on the desktop for once.)

Finally, there's the issue of this program being a client specifically for one company's proprietary blogging engine. Why not make it work for WordPress, TypePad, Posterous, and all the other major and popular blogging platforms? For that matter, why am I still using Blogger when there are so many newer, slicker platforms to choose from? I have to admit that I was tempted to switch out from Blogger — WordPress and Posterous in particular have impressed me the most — but when it comes down to it I'd rather have my blog on Google infrastructure than anywhere else. With that said, not everyone feels this way, so they choose newer, snazzier blogging software — which is how Blogger ended up neglected by hobbyist software developers. To this day, I still can't even log in to my Blogger account using Drivel, a desktop client with support for all sorts of blogging engines.

Introducing Lumberjack, a desktop client for Google Blogger

2009-12-15T18:59:00.001-08:00

Earlier this year, I mentioned that I was working on a desktop application, a client for Google Blogger, written in Java with the Swing GUI toolkit.

I recently found some time to work on the enhancements that I said I wanted to do during my last update, and I'm ready to release the application to the world. Get a ready-to-run file at the Lumberjack download page. Just download the JAR file, save it, and run it; it will work as long as you have a Java runtime installed. If you're running Windows, Mac OS X, or Linux, you probably do.

If you have trouble logging in, you may have to solve a Google CAPTCHA first. You should only have to do it once.

If you're a developer, you'll find the source code at the Lumberjack GitHub repository. Simply run ant to build the application. (Yes, I put a lot of effort into getting this one-step build working.)

Patches, questions, and suggestions are most welcome. Just leave a comment on this post.

Idealism and retreat

2009-12-14T18:58:00.001-08:00

When I'm by myself, when it's quiet with no distractions pulling me in a million directions, I automatically start to dream and envision great things. All of my scattered realizations and fleeting memories somehow just coalesce to give me more clarity about myself and what I'd like to do. It's as if a master blueprint is forming in my mind. It's not a time for questions about implementation or feasibility, but for consideration of what should be done simply because it's the right thing to do. There's no cacophony pressing from without, clamoring that it can't be done or that it's not worth the time, effort, or attention.

Organic thankfulness

2009-11-27T12:10:00.000-08:00

I've always had trouble keeping my sentimental cycles synchronized with what the calendar officially sanctions. This year is no different. I got into a very thankful mood early — well before Thanksgiving Day rolled around. Days before Thanksgiving proper, I was working on writing the bulk of my post-wedding "Thank You" cards.

True to my high ideals and impractically grand ambitions, I didn't want to just write a generic, canned response to everyone; I wanted each card to be sincere and personalized for each recipient. For I follow the golden rule: "Do unto others as you would have them do unto you." If I am deeply insulted and take personal offense whenever someone sends me a generic message disguised as a customized one, why would I inflict the same on others?

Faced with the task of writing so many customized notes to so many people, it was hard to even get started — but good old engineering experience soon kicked in. Whenever I find it hard to get started on something, I try to build momentum by doing the easy part first. Then, to maintain that momentum and to keep my focus tight, I periodically stop and ask myself what the main objective is, and re-align accordingly.

So, I started with what was easy. I simply read the card that each person sent, so I could see what was written. That would give me something to riff off of. And sure enough, this worked in giving me something to start off with. But the unexpected (and much more helpful) side effect of reading these cards was that they just naturally made me feel thankful. Writing the response just seemed like the natural next step — in contrast to painfully extracting whatever random remnants of goodness I had left in me, or emotionally manufacturing shoddy imitations of good cheer for putting on paper.

The trouble with letting nature run her course and supply material for us is this: what she supplies seldom fits our purposes exactly.

And so, I had to rein in my overflowing thankfulness to keep it focused on thanking people specifically, rather than letting my responses meander all over the place.

I remember one specific instance of an older relative, widowed for a couple of years now, whose long and happy marriage had instilled in me a deep and lasting belief that marriage in this day and age could work. She and her late husband had so many excellent qualities that I could easily have filled her card by heaping random praise on them. But by focusing on thanking her first, I actually had the chance to keep the note shorter and more meaningful by zeroing in on how they inspired me through their marriage specifically, rather than listing their virtues as individuals.

I'm almost done writing these cards. Each card gives me a chance to take a natural sense of gratitude and then channel it in a way that renders it more meaningful by doing a better job of making it known.

Automated summaries and excerpts

2009-07-05T23:45:00.000-07:00

You've seen them around the web: they're the blocks of text appearing under the headlines, giving you a little more information on what the linked article is about. If the headline didn't tell you enough, the summary or excerpt is supposed to serve as a sort of fall-back mechanism to tell you a little bit more.

The websites of such longtime print titans such as The New York Times, The Economist, and The Wall Street Journal tend to have good summaries beneath their headlines. They know how summaries should be written — by hand. They are, after all, the professional producers of such content and have a vested interest in getting that content viewed.

Those who are a few steps removed from the production of such content, on the other hand, make it readily apparent that they just don't care. Two of the most surprising offenders are Google and Apple — ironically, two media darlings who appear regularly in feature articles.

A quick jaunt over to Google Finance gives us an example of their automated summary text. If news articles could get circumcised, then Google Finance summaries would be the equivalent of the unwanted foreskins.

decades of dollar ..." id="BLOGGER_PHOTO_ID_5355221299523276386" />

Where do we even begin when trying to enumerate what's wrong with this? First of all, the summary text is much too long to reasonably hold the attention of the typical reader who is scanning the page; it's unreasonable to even call it a summary.

And even if you read the entire thing, it doesn't make sense because it trails off. Decades of dollar what?

And then there's the extraneous information: the name of the author, the location where the article was wired from, and the name of the wire service. I'm sure the author is a swell guy, that Paris is a lovely city, and that Reuters is excellent at what it does, but what is all that doing in the summary text?

Oh, but Google is all about automating stuff, even if it turns out a little ugly. Let's look at what Apple does; we know they've got a better sense of design than anyone around.

Sorry to be the one to tell you this, but the headlines on the Apple start page don't fare much better. Here's just one entry from the entire embarrassing list.

Well, if anything could be called an improvement, I suppose we can give this one a little bit of credit. The summary is at least short enough to read and doesn't contain extra cruft. But it still doesn't make sense when considered as a self-contained sentence. Barbecue Glazed Alaska Salmon with what? Why should I click “Learn more” when you can't even bother taking the time to edit the summary to give me a coherent sentence up front? Is this entire article made up of sentences that are going to trail off and leave me hanging just like the summary did?

Why, in the middle of 2009, do we still see these amateur-looking automated excerpts on websites run by widely respected technology companies?

I'm not the one making decisions inside the conference rooms in these companies, so I can't say for sure. But I can venture a guess. These two companies are highly respected, to a large extent, because they're also highly profitable. Assuming that people don't really care whether the summaries are readable or not, it certainly makes sense to cut costs and forgo hand-editing of summaries. And being technology companies, of course they'd want an automated, technological solution; when all you have is a hammer, everything looks like a nail.

So these world-class companies knowingly produce these barely passable, poor excuses for excerpts. What does that mean? What's the implication? From this, we can conclude that these excerpts aren't meant to be read. If they aren't meant to be read, then they aren't meant to be taken seriously. They're merely filler text, a cheap imitation of what big-boy news organizations have done for years. Maybe in the age of continuous partial attention, this makes sense for a new kind of audience.

As for me, I'm going to remain stubborn and demand only the best from my news sources — including summaries and excerpts. Is it really too much to ask for a complete sentence these days?

Finely tuned collective effort

2009-04-15T00:50:00.000-07:00

As someone who is rabidly individualistic, working with other people isn't something that's hard-wired into me. For this, I've been chided by friends and family members who hold collective effort up as a sacred cow. They cannot possibly fathom why anyone would question the merits of working together.

I remember reading (probably in the Harvard Business Review) that this teamwork mindset is prevalent especially among my generation, which grew up playing team sports and doing group projects in school. Contrast this with the modus operandi of previous generations of workers, who were much more individualistic: put your nose to the grindstone, pull your weight in the organization, and let your merits stand on their own.

In this sense, I am very much a traditionalist.

But I've been through some fiery projects in school and during my consulting days working with clients. Massive requirements and short deadlines have a way of focusing the mind and forcing the casting aside of closely held ideology. I've seen teams coming together to accomplish something that was more than the sum of the parts, and seeing that in action made me more open to the idea.

Still, I maintain what I consider a healthy skepticism towards a widespread and blind allegiance to the nebulous concept of collective effort.

I acknowledge that it produces tremendous benefits as many sets of eyes and differing perspectives hammer away to solve problems, and that work can be parceled out and done in parallel, resulting in undeniable time savings. Knowledge can be shared to increase the human capital of everyone involved; it makes everyone better off by increasing the raw capability of the team so that the team's maximum output doesn't merely hold constant.

But reaping these benefits does not come automatically.

Any time you pool resources to exert greater leverage, you also put yourself at risk of exercising power in the wrong direction or of misallocating those resources so that all you're left with is a colossal heap of waste. Capital intensive industries such as auto manufacturing earn billions in profit in good times, but when crisis strikes, the cost of all that unused capacity is crippling.

I don't want to come off sounding alarmist or overly pessimistic about teamwork gone wrong. Truth be told, it rarely ends in a blazing mess. Even the most dysfunctional team is just a misconfigured engine that nonetheless manages to sputter along, misfiring occasionally, but still operational. I'd guess that this is how most haphazardly assembled teams manage to get by.

If you choose to go it alone, it's like pedaling along on a bike: you can get in and out of places, but even the car with the misfiring engine can go faster than you can. Working alone feels much more elegant and affords the independent worker more agility. This is where the continued appeal of individual effort arises from. But for most undertakings worth their salt these days? A team effort is the smart choice, just like you need a car to really enjoy all that Southern California has to offer.

But again, let's be honest. Driving a car with a misfiring engine isn't very much fun. It's nerve-wracking, and the only people amused by it are the people who are laughing at you from a safe distance.

What can you do to pull off collective effort like a finely tuned engine?

Share knowledge. This makes you better equipped for the future.

Exploit different strengths and don't strive for homogeneity. This is where differing viewpoints and other sets of eyes can really come in handy.

Implement systems of coordination and common convention to maintain coherence. It's ridiculously easy for everyone to start doing things their own way.

Know the difference between parallel and serial tasks. As tasks in this knowledge-based economy become increasingly complex, it's not as easy as painting a room and asking everyone to take one wall.

Gather the right people for what you want to do. It does you no good to get an excellent tax accountant when what you need is a good plumber. No offense to tax accountants.

Question your assumptions: do you really need a team for what you're trying to do? Do you really need a large one, or would a small one suffice? And are you prepared to put in the hard work for everyone involved to get the maximum benefit out of the experience?

Enough with civility: confronting line cutters and queue jumpers

2009-03-28T17:41:00.000-07:00

Last Saturday, Sophia and I went to Disneyland to celebrate my birthday and to spend some time together, since our work schedules haven't really overlapped favorably in recent weeks.

We were in line for the Matterhorn, and as decent, upstanding Disneyland patrons, we took our places at the back of the snaking line.

It was a long line, but it was what one would expect for a Saturday at Disneyland. It was moving at a good clip — faster than the 405 near Santa Monica during rush hour, anyway.

About halfway through, a suspicious looking Asian guy wearing sunglasses sidled up next to me from out of nowhere. For a good three seconds, he stood there without saying a word. I thought that he was expecting to be recognized, but upon closer inspection I could recall no previous association with him.

When he finally said something, he said, "Hey man, you mind if I get behind you? The line is really long and I don't want to wait in the back."

Flabbergasted at the audacity of the request, my verbal faculties sputtered, and the first thing to come up was, "No."

As I realized that I had actually issued an unintentionally affirmative and welcoming response, I stepped in to clarify. "I mean, yes, I do mind. And I mean that no, it is not fine with me if you get behind me."

He responded, "Oh, come on. Why not?"

"Because we began at the back of the line, like everyone else, and so should you," I said with an annoyed and furrowed brow. (At this point, it was only one brow. That's how annoyed I was: unibrow annoyed.) I gestured to the very back of the line. "You should start heading over there. You shouldn't be here."

He kept a straight face. "Oh, alright." And that was that.

Or so I thought. I turned around five minutes later, and he was just standing there — right behind me!

I looked at the people right behind him to see if they were upset in any way. It was a wholesome-looking white American family, and they didn't seem bothered by anything at all. It looked like they were having a good time.

I am certain that the scumbag probably chose to ask me because white people would be more likely to think we were together since we were Asian — which is a reasonable assumption to make. But the guy was alone, and I felt a little sorry for him; maybe someone close to him died. You never know what the story is with people, I figured, so I decided to forget about it.

But five minutes later, the scumbag's wench joined him. At this point, I was fuming, and was very close to causing a scene. Sophia told me to just forget about it since we were there to have a good time, and I knew that anger tends to make me act irrationally, so I just fumed for a while and hoped it would pass.

It has been a week, and it has not passed.

While writing this up, I found out that queue jumpers get away with it most of the time. In the future, please do everyone a favor and cause a scene, and I will do the same.

Now, I understand that it may be difficult to think quickly of what to do. If you need some ideas, here are a few things you can do to people who cut in line.

Wait until you get to the front of the line and then tell the people in charge. Make the cutters return to the back of the line all over again.

Loudly and clearly proclaim to everyone behind the cutters of what they have done. Populist outrage is a powerful force, and public humiliation is a long neglected tool. Put them together and you've got a great combination.

Take a picture of the cutters. If they cut in pretending to know you, why not play along? Say loudly, "Oh, hey, what does your new driver's license picture look like?" Remember their names, and then post their names, their pictures, and what they did, so that employers can find them when they perform background checks. (Make the page SEO friendly so these dirtbags are easier to find.)

Get physical. The line cutters don't belong there, so you're just righting a wrong and putting things back as they should be. A simple shove should do the trick, but be prepared for some pugilism should you go this route. This is particularly well-suited for those of you who don't resort to violence — because it's your first choice, not a last resort.

It's about time these scoundrels get what they deserve. If it helps, just bottle up whatever road rage you have, and instead dish it out to someone who actually deserves it. Disneyland may be the happiest place on earth, but happiness will remain incomplete as long as we remain complacent about people who blithely dismiss the ideals of justice and fairness.

Vision and chaos

2009-03-27T12:39:00.000-07:00

Among the key elements of my father's network of enterprises are the fixer-upper houses which he rents out. As soon as my brother and I were old enough to be of use on these construction sites, Dad would take us along.

I hated the messiness of the building process. The floor would typically be littered with drywall chunks. Shattered roof tiles sat in piles on the front yard, and sawdust was sprinkled over everything.

When it was all done, though, with everything cleaned up, I felt accomplished for having been a part of bringing about the final outcome. It was more than easy to forget the messy process that brought about the end result: forgetting was automatic. It actually took me conscious effort to remember what it took to get there.

While I was in the thick of it, it was discouraging to see the mess in front of me, because it just didn't seem possible that everything could be made right again. All I saw was a seemingly intractable mess. My father, on the other hand, never seemed fazed by it. His vision of the end result was not clouded by temporary worry because he was certain of what we were working toward. He saw things not as they were, but as they should be.

Over the years, I learned to embrace the temporary mess, provided there was a plan and a vision for building something beautiful from it. Still, this sort of unflinching confidence doesn't just come at the flip of a switch. It took many messes and subsequent turnarounds to deeply ingrain this kind of optimism in myself. Even now, I need a conscious and intentional self-reminder not to be overwhelmed when confronted with a seemingly insurmountable task.

The boy knows he is a man when he can see not only what is in front of him, but what he's going to make of it.

My automobile's cooling system and its plastic parts

2009-03-10T01:22:00.000-07:00

I've learned a lot about my car's cooling system over the past couple of weeks. There's nothing like the prospect of a melted engine to focus the mind. Typically, I would be content to leave it to the mechanic, but the cooling system has many moving parts, and I'm the one who sees firsthand all the symptoms when driving it in various situations.

At the very least, anyone in my position would have to take careful note of which circumstances triggered certain events. Such diagnostic tips can help the mechanic narrow things down so that he won't charge you as much for diagnosing the problem. Ideally, we'd also prefer that he fix everything that's wrong with a component as vital as the cooling system.

I've had to watch the reading on the temperature sensor, for one. The key is to never let the needle hit the red zone at the top of the temperature gauge. If it does, your engine's head gasket and other crucial parts are in critical danger of melting, distorting, or breaking. The repairs for those problems are much more expensive than those to the cooling system.

When I went to the mechanic this morning, I took in various observations that would help him narrow down the problem and know where to look. I noticed that the fans were going full speed because of the higher running temperature, so I told him that the fans were extremely loud after a short drive. From various sources online, I made sure to observe any difference between city driving and high speed freeway driving, but there was none, so this meant there was one less option to consider.

With the cooling system in my car, things have been failing left and right in a sort of chain reaction as the increased running temperature of the car's engine puts a lot of parts under extra stress. Whatever parts failed and needed replacing were just worn out and should have been replaced long ago. Rubber rings had become as hard as plastic. One plastic pipe had become so brittle from age that it broke off; I had to re-fasten the hose clamp just to keep the engine running cool enough to drive to the mechanic. Metal parts such as the thermostat housing and the water pump showed signs of corrosion; in the case of the thermostat, it wouldn't open to let coolant flow as it should.

The mechanic told me some interesting tidbits while we were ruminating aloud on the absurdity of car makers — including Daimler and BMW — using so many plastic parts all over the cooling system. According to him, the move towards plastic parts is justified by lower cost of materials and making the car lighter so the engine doesn't have to pull as much weight. One thing he observed was the increasing failure rate of newer cars — and he said it wasn't unusual for people with new cars still under warranty to come to his shop with worn out plastic parts.

Suffice it to say, that made me very hesitant about paying a premium for a newer model Mercedes-Benz or a BMW. If I end up buying a new car soon, it may well be a Hyundai, a Honda, or a Toyota. If everyone's using plastic parts, I may as well pay less.

In any case, I'm surprised that my old car has lasted this long, considering the long distances I drive on a regular basis. It's a 1996 Mercedes-Benz C220. I've been very fortunate to have the car running within its prescribed temperature limits, despite all the hand-wringing and pulling over to the side of the road, fraught with worry.

Feisty Ford Focus

2009-02-26T23:58:00.001-08:00

People who drive small cars sure have colorful personalities. Yesterday, on the way to work, I saw a Ford Focus with a license plate frame that said, "Don't laugh. It's paid for." It was refreshing to see someone being flashy about their financial responsibility — a quality which is rarely flaunted.

Lessons from refactoring

2009-02-25T00:15:00.001-08:00

I enjoy the process of refactoring software — especially software that was written by other people. It forces me to understand things inside and out (which is a tough point to arrive at when all you do on that code is the occasional drive-by bug fix). On top of that, since the desired functionality is known already, it's all redesigning of the data model and writing code. There are fewer product-level questions that come up. The specification is pretty much set.

For this particular project, though, I have the unique challenge of making the new design pass muster with multiple parties. People are picky and they always have something to say, so I took special care this time to anticipate any objections and address them so they'd get a better sense of my thought process. I've still had to make changes — and I make them gladly — but I've found that having to explain each decision I'm making generally serves to clarify my thought process.

It certainly helps to have some time to think hard about these changes, too. It's a healthy thing to let a data model sit for a bit and soak into the minds of all stakeholders while their input is incorporated. It also helps when people are interested enough to give detailed and insightful feedback. Sure, it generally doesn't hurt to have more sets of eyes looking at a plan, but having a team of people analyzing it harnesses collective historical memory — great for backward compatibility — and makes the plan more bulletproof by ensuring that it addresses concerns from many points of view.

Blogger client as a Java Swing desktop application

2009-02-14T03:06:00.001-08:00

Ever since I started using Blogger, I've been looking for a decent, no-nonsense desktop client to write my posts. I personally find it to be a lot of hassle to load up a big, heavy web browser and log in to Blogger to make my posts. (To be fair, Drivel was quite excellent in the past, but doesn't look like it's maintained anymore.)

After a little poking around at Google's API documentation, I am pleased to announce that I'm writing this post from a desktop application of my own making. It's a Java desktop application using Swing, so it will run on Windows, Mac OS X, Linux, and any other major desktop platform with a Java runtime.

So far, I can select from a list of blogs, write in post titles (to be friendly to those search engines), write post content, and submit new posts. It's really basic, but it's quite a milestone and a major motivator for me to at least have gotten this far.

Soon to come on the feature front, I'm aiming for loading old posts to update, deleting posts, and smoothing out the user experience. (Right now, the app looks and feels like the weekend side project that it is. It doesn't even scroll when the text is too long.)

On the code quality front before I release this thing out into the wild, I'd like to do some release engineering, automate the build process, and just separate the various concerns (like GUI drawing code, network connections, and application logic.)

In any case, it feels good to have my software development methodology validated: get a dirty prototype up and running, and count on the morale boost to spur further development. So far, it feels pretty good. The hardest part is done and the challenge from now on is just finding spare moments to work on refining this thing.

Update 02/20/2009: I've implemented updating existing posts, and I'm updating from the improved client right now. Of course, to get to that point, it had to load up old posts. A cleanup of the code base before proceeding is probably in order, because I'm kind of going nuts trying to figure out where everything is and where new things should go.

Update 02/22/2009: I've created the build system to use Ant and package the app into a neat little .jar file. I also fixed a minor usability bug to make the app more pleasant to post with and give you feedback when a post has been submitted. The refactoring into a better architecture will have to wait; my mind's capacity for refactoring is mostly taken up by a big refactoring proposal I'm putting together at work.

Update 12/16/2009: It's open to the public! Please see the Lumberjack release announcement. The code has been cleaned up, made more maintainable, and a few features have even been added to make the application more pleasant to use.

The trouble with budget surpluses

2008-12-23T01:17:00.001-08:00

The trouble with a government (or any big organization) running a budget surplus and sitting on a big pile of cash is that people start asking questions like, "Why don't we do something with that money, especially since there's so much to fix?" At that scale, having some cash ready for a rainy day is not an excuse that people are able to handle, especially when the quoted figure is in the billions of dollars. Surpluses may seem like a lot of money, although when divided on a per-person basis, they seldom amount to much at all.

In this deficit-infested time, it's easy to rail against profligacy in our budgets. I'm not trying to engage in what-ifs here. I'm suggesting a way we can avoid this same problem in the future.

We've been told that it's wise to save up for the future, to have some cash on hand just in case — folk wisdom that was reinforced by the big collapses this year. We also know that it's hard for most people to hear about a surplus in the billions of dollars without wanting to spend it on something.

If an even moderately sized government were wise and careful in its spending, it would quickly accumulate a large surplus. That large surplus would then get people dreaming about capital improvements, social services, and other goodies that drain the public purse.

Those who advocate for smaller governments would say that the problem could be solved by striking the problem at the root — shrinking the government and therefore its potential to accumulate large sums of money. But pools of large capital certainly have their advantages, and whether by conscious choice or mere political expediency a large government must stay large, it could do better to quote the figures based on the population of the governed; that is, on a per-person basis. This might make billion-dollar surpluses less distasteful to the vast majority of us, whose visceral reaction to large dollar amounts placed before us is to spend imprudently.

Small business owners: don't be a jerk

2008-12-11T23:59:00.000-08:00

My girlfriend Sophia is an assistant manager at Abercrombie and Fitch, and is thus bombarded with more than her fair share of rude customers. One story she told me this week was about a woman who went ballistic after asking to try on one of the mannequins' jackets and being told no.

Now, people are just crazy, and Sophia has told me many stories like this before. What was so different this time?

Well, the crazy lady played the business owner card: if it were her store, she would have gladly taken the jackets off the display mannequins. This know-it-all "business owner" then proceeded to hound Sophia for her full name and pressed her for her employee ID so she could file a formal complaint, and refused to go through the normal channels.

It is precisely this self-serving arrogance and provincial, narrow-minded ignorance that keeps small business owners from being taken seriously. As a former small business owner myself, I know that the burden is heavy: you've got to worry about employees, customers and growing your business. On top of that, you have the responsibility to make sure any legal paperwork is in order and that taxes are taken care of. But just because you're able to handle this does not mean that you know all there is to know, and that your way of doing things is the only way.

In response to this growing sense of self-importance, here are three things to keep in mind. (I use these reminders to keep myself in check, too.)

a) Rules and processes have a place, even if you choose to forgo them. As a small business owner, you can get by with fewer rules and processes in place. In fact, in most cases you do much better when you're flexible. But larger businesses have a much harder time being flexible; it's not impossible, just much harder. They have to manage everything more strictly in order to hold together the larger whole. A little sloppiness in your store can be passed off as "charming." In a national chain where customers expect extreme tidiness and consistency, that sloppiness is not charm. It is chaotic, and it is poor business.

b) Not everyone enjoys the same latitude to call the shots as you do. You may be your own boss, but most people have someone else to answer to. I've found that being a business owner, seeing the bigger picture, and having the power to remedy things has turbocharged my ability to take the initiative, even after going back to working for someone else. Still, despite having passion for my line of work and understanding its larger implications, I have much less scope to make important decisions. In large companies, even CEOs don't wield absolute power, because they have a board of directors and shareholders to please.

c) You are not special, so don't expect special treatment. A couple of years ago, I received a parking ticket by mistake. I knew that I had moved my car in time, and so I decided that I would write in to contest it. One of my co-owners suggested that I take a tough stance and mention that I was a business owner — as if that had anything to do with my guilt or innocence. I mentioned it anyway, thinking that a little reminder about my contribution to the community wouldn't hurt. Still, I didn't want to rely on that mostly irrelevant fact, so I put much more effort into stating the facts of the case. I drew a diagram of where I had parked, when I had moved my car, when I had been ticketed, and why it was a mistake. In the end I got the ticket waived, but I have a good hunch it had more to do with stating the facts than mentioning that I was a "business owner."

Conceptual tools for allocating finite resources

2008-12-06T19:39:00.000-08:00

I have lived and worked in Greater Los Angeles for over a year now. Since the area is one of the world's major population centers, it only makes sense that I'd come across plenty of opportunities to observe the interplay between two forces: lots of people and limited resources.

It's a common problem, but always a multifaceted one. Coming up with a solution means that planners (who could be church volunteers, business managers, or software engineers) must consider various factors. Having to stop and think about this can slow down decision making, and even when there's time to ponder and plan, thinking of a solution from scratch is more error-prone than referring to a proven set of guidelines and rules.

Pricing
Consider the situation when lots of people are elbowing each other to buy concert tickets. There's no increasing the supply of these tickets since a concert hall or stadium can only hold a set amount of people without the Los Angeles Fire Department throwing a hissy fit.

Many eager economics students and libertarians automatically want to apply the law of supply and demand here: why not just raise the prices? In fact, that's what some organizations have done: Britney Spears concert tickets for Staples Center start at $150.00. (How do I know this? My girlfriend told me. Really.)

As the purveyor of tickets, you can do that when you have some idea of how much demand there will be. But what if demand shoots up while your supply stays the same? When it looks like you're about to sell all your inventory and are hours away from turning away your customers and disappointing them, you could double your prices. But then, your customers would be angry that you're gouging them. Your company would appear disreputable for "arbitrarily" changing prices on consumers.

If you're looking at it from an economist's point of view, there's no problem there: you're just changing the price to meet demand. But from a public relations vantage point, you run a very real risk of alienating your customers. It'll look like you're exploiting them, when all you're doing is ensuring availability (while making a few extra bucks).

On top of that potential PR nightmare, there's a downside that even economists must acknowledge. It's the phenomenon that economists call "sacrificing equity for the sake of efficiency." That's all fine and high-sounding, but what does that mean? It means, "It's not fair." And the customer will feel this way. Your costs haven't changed: it still costs you the same to make it, doesn't it? This gives your customers a reason not to trust you, and therefore not to come back.

Changing your prices is certainly an effective tool to curb or increase demand, but raising them is an extremely delicate matter in the age of consumers who expect posted prices to stay the same.

Waiting list: first come, first served
A common alternative to raising prices is to let people buy stuff on a first come, first served basis. The folks at Ticketmaster do this, in case the $150.00 concert tickets sell out (which they often do).

As you may recall from your economics class, this is considered more equitable, but not as efficient: the people selling tickets could make a lot more money just by raising the prices. (If you do not recall from your economics class, shame on you for not paying attention. But hey, now you know. Trust me on this one.)

As far as fairness is concerned, lower prices with waiting lists are an improvement over simply raising the price. But if you're not one of the first served, you probably won't feel like it was very fair. Maybe you had to be at work that time of day, or don't have time to wait in line.

Lottery
A different spin on fairness is to let people enter a lottery to let them buy what they want at lower prices. If you were to enter such a lottery and your number got drawn, you could then buy what you're after. Typically, submissions to this lottery are accepted during a specified time window. So if you have to be at work when the lottery first opens, that's not a problem. You have just as good of a chance as the first person who entered that lottery.

This was part of Saddleback Church's approach when they hosted the Saddleback Civil Forum with Barack Obama and John McCain during the presidential campaign. Since Saddleback Church is very high-profile, it was important that a purely pricing-based model was avoided. Taking an exclusively pricing-based allocation of tickets to see the candidates would have looked too much like favoritism towards the rich, a Biblical no-no.

Then again, Saddleback Church also had to cover its costs, so it actually opted for a brilliant hybrid approach that consisted of various tiers, each with a different price. It was a multi-tiered lottery. Higher-priced tiers would likely yield a smaller pool and a higher chance of being selected. You can't please everybody, but such a creative and enlightened approach likely pleased quite a few free market die-hards while at the same time placating those who were concerned about equality. If only governments and businesses were as wise.

Look where you're baby stepping

2008-10-25T15:59:00.000-07:00

When you're starting out from scratch, it's hard to predict what you're going to need. I just moved into a new apartment, and my kitchen cupboards are empty because I've decided that I'm not going to even try to predict what I might need in the future. The way I intend to re-acquire the supplies I'll need is by doing stuff.

I mean, sure, there's cooking oil, salt, and pepper. But for a little more exotic Chinese cooking, I'll need some sesame oil or star anise. Beyond the basics that I'm certain I'm going to actually use, it's impossible to predict what other odds and ends I will personally end up using. And soliciting advice from others in the past hasn't been good for me: what works for other people often does not work for me.

For me, the only way I have been able to know — with certainty — what I'll need is to actually start trying to do things. If I'm trying to figure out which ingredients I'll need to cook a meal, I just start trying to cook. If anything is missing, its absence becomes readily apparent.

These are kind of like kitchen use cases, if we're going to resort to software development parlance. It's like I'm going agile, or adopting Test Driven Development with my cooking processes. I could theoretically start out with nothing at all — to illustrate the method theoretically, I actually should start from nothing. I would soon find that I need a cutting board, a knife, and whatever ingredients my recipe calls for. For a basic meal I would also find that cooking oil, salt, and pepper come in handy.

After cooking a basic meal, I know with confidence that I have everything I need to cook — wait for it — a basic meal. That common use case is taken care of and I can cook basic meals in the future with confidence. Now, what if I were to go ethnic? I'd soon run into some cases where I'd need to stop and get some more ingredients.

The idea is that, after a while, I would eventually be able to confidently prepare a broad range of meals. This confidence is rooted in the knowledge that I have verified that I have everything needed — by actually having done it in the past.

Of course, it's possible to carry out this kind of method to absurd extremes. If you know for sure you're going to need it, then make a note and take care of it. Be reasonable in what decisions you leave for later and which ones you can make now. In the cooking example, it's great for didactic purposes to start from scratch, but in practice it's dumb not to have the basics like cooking oil or a cutting board.

What about when developing software? You can leave scalability concerns for later if you're just starting to write a web application, but no matter what, you are going to need a machine and a relational database. (If you're writing something, like an offload server, because it needs to be scalable, then you damn well better take scalability into account.) To maintain your sanity, it might even help to sketch out a data model. Take baby steps, but know where each step will end. Don't be content with merely knowing that your foot will be in the air and then end up somewhere on the ground at some point. After all, this nebulous "goal" can just as easily be achieved by tripping and falling.

Getting more out of a sentence

2008-09-18T23:18:00.000-07:00

There's a very simple, effective, and systematic way to amp up the amount of insight you get from the things that you read.

It's probably best to explain with an example. The following is from Ed Catmull's article for the Harvard Business Review, "How Pixar Fosters Collective Creativity."

We must constantly challenge all of our assumptions and search for the flaws that could destroy our culture.

How much thought can this spark in your mind?

If we just read this sentence several times, each time with an emphasis on only one word, we get a new angle on the general idea being conveyed. Something different is emphasized.

We must constantly challenge all of our assumptions and search for the flaws that could destroy our culture.

The emphasis is on the we. This tells me that whatever this sentence is talking about involves a team effort. Everyone on the team must be involved. It's a "we" rather than a "me." But does that necessarily have to be the case? It starts with somebody, right? Why not with individuals?

It's okay — a good sign, even — to ask questions about the truth or applicability of a passage. If you're a reader with a healthy sense of skepticism, those kinds of questions immediately arise. A lot of the same questions will come up: is it necessarily so? Is it true all the time? The more you ask these kinds of questions, the more you hash things out and figure them out for yourself.

Let's look at another word.

We must constantly challenge all of our assumptions and search for the flaws that could destroy our culture.

This time, the emphasis is on the word "search." This tells me that the flaws are often hidden; otherwise, they wouldn't require searching. Now, I'm an imaginative guy, so then I imagine myself searching frantically and then getting tired or discouraged. When I'm tired or discouraged, I ask questions. Why am I searching, anyway? What exactly am I searching for, again? How do I really know when I've found it?

That's only two words, and I could go on for a while, but I think the general method has been made clear. The biggest plus going for this method is that it is systematic; I can just emphasize each word until I'm plumb out of new thoughts. When I'm done with all the words in the sentence, I know I'm done.

Personally, I've found that this process also helps me internalize ideas. Even if I don't remember the exact wording of a sentence, the dance of thoughts anchored around a single idea tends to leave a lasting impression. I imagine this is because the thoughts that come to mind immediately tend to be those of the most concern to me. When it comes to things that truly concern me, I have an easy time remembering.

Ruby on Rails: optimizing a slow-rendering page

2008-09-17T22:27:00.000-07:00

Recently at work, I was faced with the problem of fixing a slow-rendering page. The page is part of an internal content management system, where the content includes video, images, and text. We have over a thousand pieces of content being managed with this tool and the slow page was a listing of all the content along with key overview-type information for each content package.

Everyone was fine with this page loading slowly, since we knew that there was a lot of content. The problem arose when the page would only show around 50 items and then stop loading; the server configuration was set to automatically stop requests that took too long to process.

I knew that the best course of action would be to start measuring. I took a look at the logs to see how long the request was taking and the approximate tasks where time was being spent. At JibJab, we take pride in caching the hell out of everything. Only a tiny fraction of time was spent hitting memcached. And it looked as though memcached was doing its job: almost no time was spent hitting the database. But a ridiculous percentage of the time was spent "rendering" the page.

That meant the onus was on my code to speed things up. Since the slowdown had become more and more apparent as we added more content, my guess was that the loop was the main source of the problem. Anything that would be only a tiny bit slow on its own would lead to a delay over a thousand times greater.

The first thing I removed was a call to cycle(), which I was using to zebra-stripe the table display for easier reading. In its place, I put in an index variable that would be modulo-two to determine its index into an array of two choices (even row or odd row). To keep the same feature, I had to dirty up my code if I wanted it to run more quickly. And indeed that shaved off about one-fourth of the rendering time.

But the page was still taking over ten seconds to render, so I continued my hunt. A pattern I saw many times within each iteration of the loop was an old friend, link_to. I am not terribly attached to this particular friend, however, so I got rid of it and replaced it with some good old <a> tags, which bought me a good second or two.

So far, I felt pretty good about what I had been able to do to extract render-time savings. But there was still room to shave the time down, and luckily for me, Inflector.humanize was being called in each iteration like there was no tomorrow. I decided that there indeed would be no tomorrow for these expensive-looking helpers, and after removing all five calls inside the loop, I got total render time down to a very respectable two-second neighborhood.

The lesson learned? Sometimes code has to be a little messier so it can be fast enough to be used. Specifically, watch out for those beautiful-seeming Rails helper functions; they do a lot of work for you, but sometimes you're just better off doing the work yourself.

Bilateral versus multilateral trade agreements

2008-09-16T23:14:00.000-07:00

News sources everywhere seem to lament the collapse of the Doha round of trade talks because countries would then have to resort to bilateral or regional trade agreements. For any supporter of free trade this would be understandable, but if we acknowledge the short term havoc wreaked by rapid changes to trade flows that do not give workers and business owners time to adjust, a gradual and stepwise approach to the opening up of markets seems to be a more prudent way to go.

One of the major problems cited by those who favor a broad trade policy is the inconvenience of having to keep track of different rules for different countries. I would go about solving this by building a hash table of trade rules. Basically it would mean that given a country and a product as input, we would get the policies regarding that product as output. This would be a large table, but we would ask and receive only what we are interested in.

There's also the advantage of isolated policies having limited effects, rather than opening up to the entire world in one go. Maybe there is something I'm overlooking, but I don't consider the use of bilateral agreements to be an indicator of failure. I see great potential for reaping the benefits of free trade with less of the shock typically expected by any opening up.

The collapse of grand schemes for international trade deals is really a familiar scenario that just happens to be playing out on a macro level: a project's planners have decided that they have bitten off more than they can chew, and decide to reduce the scope of the project. Everyone does this, and while we may feel disappointed at envisioning the results of the grand project and having to instead settle for something less, we eventually resign ourselves to the reality that we are doing all that we can, and it is a whole lot better than nothing at all.

Colored bash prompt

2008-09-04T16:53:00.000-07:00

The following will set a bash prompt to highlight the name of the directory in yellow, and in bold.

export PS1="[\u@\h \[\033[1;33m\]\w\[\033[0m\]]$ "

This will also work:

export PS1='[\u@\h \[\e[1;33m\]\w\[\e[0m\]]$ '

The escapes have to be in single quotes, however.

During a previous attempt, I wasn't terminating everything correctly, and so when my command exceeded the length of the row, it wouldn't wrap around and start a new line. It was not just a Mac OS X Terminal problem; I tried it in rxvt (installed via MacPorts) as well as the xterm that came standard with Leopard's X11.

Doctors and programmers

2008-09-03T23:15:00.000-07:00

One evening, I was watching Mystery Diagnosis on the Discovery Health Channel with my brother. The show is about people who have strange ailments, but doctors can't figure them out.

It got me to thinking about how people in my profession approach problem solving. The approach that computer programmers take is up to us, and it's pretty open-ended, but it's widely agreed that we use measuring tools — debuggers and profilers — to examine whether each little step is doing what it's supposed to be doing. It's also a widely accepted quip that one should never try to guess where a slowdown or a bug is happening, because one is usually wrong about it.

But that sort of guessing without detailed measuring seemed to be exactly what the doctors were doing. For one guy, they drew blood sample after blood sample, but it turned out to be a problem on the genetic level.

I wonder if there's some level of professional ego that keeps doctors from doing what would otherwise be considered sensible. If it took a lot of lobbying to get them to use checklists, I wouldn't be surprised if they think they've gone through too much training to actually rely on detailed measurements.

To be fair, though, doctors probably don't have much scope to apply deductive reasoning on the entirety of the systems they're working on. There are no "development" or "testing" patients where it's okay to mess up and start over. Everything they work on is in live production mode.

Freeways that trick you

2008-08-29T00:54:00.000-07:00

I don't like it when freeways trick me.

There's I-10 west into downtown LA, when it splits off to US-101 and I-5. Six lanes of I-10 turn into a single pathetic lane if you want to stay on it to keep going to Santa Monica. Most of the rest goes to US-101. Context over consistency, to serve the massive hordes going to Hollywood? Perhaps. But this is the mighty I-10 freeway, encompassing the width of the country and passing through Phoenix, San Antonio, Houston, New Orleans, and Jacksonville. It's not some little country road in the middle of nowhere. Someone should tell this freeway, "Hey, you're not done yet. You reached LA, but the real end is Santa Monica."

And then there's I-5 north to SR22 west in Santa Ana. When you're driving on I-5 and you choose the "Exit Only" lane to switch freeways, you should be rewarded for your dedication. But it is not like that at all. It really does mean "Exit Only" — and not to switch freeways. It takes you off the freeway and you have to switch into the other lane if you want to get on SR22 west to Long Beach. Well, you must be saying, it does say "Exit Only" so what were you expecting? Well, says I, it's clearly marked SR22 west to Long Beach, while it should really be marked as a regular exit, not a way to switch freeways.

Meetings are not always toxic

2008-08-27T22:14:00.000-07:00

I'm a big fan of 37signals in general, but I remember a time when I went a little overboard with youthful zealotry for one of their philosophical tenets: meetings are toxic.

Back then, my business partner Steve and I had just landed our first major client, and this client asked for a kick-off meeting.

I curtly dismissed it, telling them I didn't need a meeting — leaving them baffled with my strange behavior (and Steve smacking his forehead).

Since then, I've found kick-off meetings with partner companies and clients to be crucial for greasing the wheels of personal exchange. It gets people talking to each other, which is worth pursuing because cross-company communication is a tricky thing. Plus, the stakes are higher: if you have to coordinate with another company, chances are that you're working on something pretty important. I've found in the vast majority of cases that it doesn't take extensive preparation to the standard I would expect of myself if I were to prepare for an intra-company meeting. People just like to talk and get a feel for each other. If meetings and conference calls are good for anything, they at least open up the communications channels.

After thinking about it for a bit, I realized that I initially learned to conduct business in a contrarian way, since I got most of my tips from the Web. But since then I've begun incorporating more traditional business practices because that's how most people still do business. I have a hunch that most people still do business "the old way" because it works.

Am I anxious that I could be blindly following new ideologies or anything that remotely seems to make sense, to my detriment? Would I have been better off doing business the traditional way and then finding out about these counter-cultural methods? I don't think so. Whichever end of the spectrum I may have started on, the important thing is to continually evaluate the merits of each idea I encounter, whether old or new, and knowing why I do things the way I do. The truth lies somewhere in the middle, and it's reachable starting from either side.

The introverted leader

2008-08-24T13:27:00.000-07:00

During the course of my two years attending Discovery Christian Church in Davis, I had the chance on several occasions to eat lunch with the pastors, namely Aaron Brockett and John Richert.

I still remember the most surprising thing that Aaron ever mentioned to me. He said that a lot of the leaders at Discovery, himself and John included, were natural introverts. From the outside, it seemed that they were anything but.

They greeted people left and right, confidently gave their sermons, and coordinated the big picture very smoothly. One of the things that caught my attention at the beginning, and subsequently hooked me in to attending regularly, was how the services on Sunday were so well-run.

As for me, I have found that I am quite happy to be a follower, unless there is a leadership vacuum — either no leadership at all, or very poor leadership. To be fair, I've found that very poor leadership can come from both extroverts and introverts.

In extroverts, the leadership pitfalls have to do with the tendency to make decisions without thinking very hard about them; not being aware of how everyone else feels; or not stopping to ask what everyone else thinks.

With introverts, what I've observed includes the lack of dynamism to keep everyone interested; indecisiveness while trying too hard or too long to build a consensus; and a lack of forcefulness.

Being the introvert that I am, I naturally got to thinking. What, I asked myself, can I do to leverage my natural strengths while avoiding the pitfalls of my natural weaknesses?

My natural strengths are things which come easily to me: thoroughness in exploring issues; attention to detail and a concern for total correctness; and having a passion for learning new things on my own, which makes it easy to become knowledgeable in various areas. I see these kinds of traits as "hard-wired" where my thirst for knowledge or obsession with detail give rise to good things. Like a microprocessor is hard-wired to perform basic operations like addition and subtraction, I'm hard-wired to do certain things because I'm comfortable with them and I know how to do them.

But the desirable traits that extroverts possess that I don't have — outspokenness, dynamism, gregariousness, forcefulness — I have to run them in emulation mode. I have to figure out some software to run on what's hard-wired in me so that I can do pretty much the same thing. Software operations are slower and require more instructions than hardware-level instructions. If we're going to continue carrying our analogy over to people, this means that I've got to put more effort into expressing qualities that are generally characteristic of natural extroverts.

But it's possible to do so, and this is good news.

For the introvert, it just takes a lot of piecing together. For example, the extrovert can just "be forceful," while the introvert has several mental steps to take. First, he must recognize that forcefulness will be necessary to avoid wasting everyone's time, and it's better for everyone. Then, he must double-check that the position he is strongly advocating is correct, because he knows his confidence in a position relies on knowing that it is as correct as can be, given all the data. Then there's recognizing that the best that anybody can do is make a decision based on all available information, and that's really the most that we can do. There is no sense in second-guessing since more information is not available. Finally, there's actually saying what needs to be said, or acting out what needs to be done. This requires a conscious mental step for the natural introvert.

As anyone can see, it can be much more involved for an introvert to display extroverted qualities, but it is possible. Over time, the basic behaviors can be optimized and moved a little closer to being hard-wired, just as software can be extracted into microcode to sit somewhere between software and hardware. This way it runs more quickly and more readily, with less effort.

There's one last thing before I completely beat the dead horse of this analogy: even though things done in hardware run much more quickly than those which are run at the software level, the software level allows for much more richness and flexibility. What does this mean for natural introverts who are looking to be effective leaders?

My very introverted answer is that I don't know; I'm still trying to find out.

Where software developers fear to tread, or why open source fails

2008-08-08T23:08:00.000-07:00

Open source has had its triumphs and failures.

The most notable wins include Eclipse, Linux, Apache HTTP server, MySQL, and Mozilla Firefox. Developer tools and server software are generally areas of strength for open source. I wouldn't say that Mozilla Firefox is an anomaly. Rather, it shares something in common with developer tools and server software: the programmers who work on it are also its users.

Unfortunately, this is not the case with the gaps that open source has so far failed to close. A friend of mine recently asked me about my choice of spreadsheet software. We talked about it for a bit, and agreed that OpenOffice.org Calc is still clunky. I suggested Gnumeric, but I don't really hear anything notable going on with it these days. In short, I don't see an Excel killer in either of these. It's not for lack of vision that these spreadsheet projects have stagnated. It's for lack of sustained passion and developer interest in making these products world-class. Rather than get deep into using these spreadsheet packages, I often just resort to SQL queries and image generation libraries if I need a chart. And I'm guessing that other developers have also resorted to what was more familiar and efficient, given their esoteric knowledge of more powerful tools.

It's obvious to many of us that successful open source projects are characterized by programmers who are also users, but I would go a step further and say that the programmers must not merely be users; they must be the power users. There are two ways that a programmer could be a power user of his own software: out of enjoyment or out of necessity.

Open source software developers should be the power users of their own software. This sustains the passion to "scratch the itch" and fix bugs to produce polish. With proprietary software, programmers have the benefit of separate QA teams or product managers. In open source, it's overwhelmingly the case that they do not.

It's been said that with enough eyes, all bugs are shallow, but it's quite often the case that not all of the mouths will bother to report what the eyes are seeing.

The music at 24 Hour Fitness

2008-08-04T22:45:00.000-07:00

For my morning workout, I switch between two different 24 Hour Fitness locations. One is right by where I live, in Irvine, California, where people tend to be either white or some kind of Asian. The other gym, near Long Beach in the city of Carson, is on my way to work. Most of the customers I see there are either black or Filipino.

Each location plays different kinds of music.

At my neighborhood 24 Hour Fitness, the music that's played is the angst-ridden punk and alternative that people have come to expect out of Orange County. (To be fair, I like a lot of the songs they play.)

In Carson, it's a bigger gym with some sort of Magic Johnson approval. They make a big deal of being tied in with Magic Johnson, and his pictures are plastered over all four of the walls, on both floors. If I were Magic Johnson, I would be a little embarrassed to have so many pictures of me smiling statically at people as they work out. The music is hip hop and R&B. As both workout and leisure music, it works for me. (In fact, it recently helped me rediscover the genius of Usher.)

Noticing the difference, I got to wondering who makes the music selections. If it's an official policy that the music fit the ethnic demographics of the community where the physical gym is located, then I'm pleasantly surprised that they took a factor like that into consideration.

Mexican flower vendors

2008-08-02T23:24:00.000-07:00

One my drive home from work each day, I see Mexican guys trying to sell flowers, standing on the curbs by freeway entrances and exits that have stoplights. After seeing them all week, I got to wondering whether any of them ever make a sale: I haven't seen anyone buy flowers yet. And I'm sure they make a sale here and there; otherwise they wouldn't come back.

That got me to wondering whether there's some sort of demographic they target.

Given that the particular time of day is when most people get off work and then sit in traffic, the cynic in me thinks that the buyers are probably making their purchases for reasons other than being considerate. A man doesn't have to plan the purchase of these flowers in advance, and even if he did, it's not guaranteed that the seller will be there.

My hypothesis is that these traffic light flower vendors' target market is men whose wives or girlfriends are upset that they're coming home late from work, and these busy guys are grasping for any quick fix. In that kind of situation, there's a lot of potential to jack the price way up. Personally, I am not yet decided on whether I would ever buy flowers from a Mexican traffic light flower vendor, but I wonder if, next time, I should at least ask them for a price in order to get more information.

Foreign key cascade on update dangerous?

2008-07-27T18:14:00.000-07:00

In relational databases there's a feature called foreign key cascading. Typically, there's an ID associated with each row, or entry, in a database table. Tables may refer to other tables using a foreign key, which typically refers to the IDs of the rows being linked. There are options to "cascade" on two kinds of operations: delete and update. When a table refers to another table, the table pointing to the other table can be configured to take changes in the ID of the original table and integrate them automatically. This is what happens when ON UPDATE CASCADE is set on a foreign key. When ON DELETE CASCADE is set on a foreign key, deleting rows in the original table will also delete rows that are pointing to that row.

So, it's not hard to imagine situations where cascading on delete would be dangerous.

Now a while ago, I was talking to a database specialist about cascading updates on foreign keys. He mentioned that they're dangerous, so I took his word for it. But I've been thinking about it and I still don't see any way they can be dangerous; the only changes being propagated outward will be changes in the values stored in foreign keys; no rows will be deleted.

I've asked around to get second opinions and examples of situations where this would be bad; for now, I'm just going to avoid cascading updates for future projects.

Truth never changes — or, why we're solving the same problems

2008-07-20T10:54:00.000-07:00

"Truth does not change. What was true 2,000 years ago is true today, and will be true 2,000 years from now. New truths may be discovered, but that just means that it was true all along."

The above is paraphrased from a sermon given by Pastor Rick Warren at Saddleback Church. Very few things are really new, which is consistent with my observation of what's going on in the world, and if they are, chances are that there's some catch. What's new and doesn't last probably doesn't work or just isn't true. Such things lack conceptual integrity and fall apart eventually when reality hits.

(I've just started a new job at JibJab, on which I'll write more about later on, but that doesn't mean my mind is completely devoid of all thought dedicated to solving larger problems.)

When I see some new file storage startup, or some new effort to create an operating system, or several companies trying to push their respective next-generation platforms, I think, "Oh, great. Not another file storage startup. Not another operating system. Not another platform."

But I see these things happening again and again — first in Web 1.0, then Web 2.0 — because the problems haven't been solved. The truth in all of these situations is that the problems remain, or have resurfaced due to technological change or heightened consumer expectations.

Having a lot of competitors trying to tackle the same problem shouldn't be seen as, "Hey, there are just a bunch of me-too companies trying to tackle old problems." Instead, it should be seen as, "These are problems that continue to badger us. These are important problems."

The people starting these companies see the importance of addressing these "old" problems, but it's important that rank-and-file employees of these companies, as well as consumers and journalists, recognize the significance of the larger mission.

In order to appreciate these efforts for what they are, it helps to focus on the fact that solving the problem is a formidable task — rather than focusing on the fact that the company is merely another contender in a crowded field.

Proofs by induction and real-world business needs

2008-06-23T16:47:00.000-07:00

While I was a young and eager undergraduate computer science student, I was taught the method of proving things by induction. Basically, it's a way of showing that something is true for some simple cases, and then showing how it can be true for more complicated cases. There are two major parts to it: prove that the base case works, and prove that the inductive case works.

I see some applications of that kind of thinking in everyday business. Just because I can manage documents on my own doesn't mean that the way I do things will scale to other people. So, to build a solution that grows with the business, I'd have to show first that it works for me, and then assuming it works for X people, show that it will still work when X+1 people are in the picture.

The thing with most inductive proofs is that they show that something is true, or if we map it tightly, it would show that things are possible in business -- that something is computable and knowable. But in real-world concerns, efficiency is a big factor: while it's necessary to show that some method or process works, it is not sufficient. We've got to show that it's efficient, too. (Or in other words, "Duh! We know it's possible. I'm concerned with how quickly and cheaply can we do it.") So, I'd have to show that something works and is efficient for me, and then that if it works and it's efficient for X number of people involved, it will still be efficient for X+1 people.

Additionally, the method or process that we try to prove by induction would probably look like a step function to handle different domains. The reason for this is that a group of five people is going to present a different kind of situation than a group of 100 people, which will in turn be much different from managing an organization of 10,000.

Note to self: when trying to come up with a new solution, come up with something that looks like a step function, with different methods for different points in an input domain, and prove that it works and that it's efficient.

Put Apache Tomcat in its own directory, and your web applications in another

2008-05-26T10:54:00.000-07:00

I recently took my first look at Apache Tomcat 6.0. I was inspired by Tomcat: The Definitive Guide, 2nd Edition, which covers Tomcat 6.0. It mentioned a way to separate the actual Tomcat distribution and binaries from the webapp data, but described it for environments that had it pre-installed as an RPM or DEB package.

As I'm writing this, the latest version is 6.0.16. Here's what I had to do to get my webapp code in a separate directory tree from the Tomcat binaries as downloaded straight from tomcat.apache.org.

Create a directory, say /home/joshuago/domain.com, where all the webapps for that domain are going to be.

Set CATALINA_BASE to the directory above. (Don't set CATALINA_HOME.)

Move conf, logs, temp, and webapps directories to /home/joshuago/domain.com/.

Run apache-tomcat-6.0.16/bin/startup.sh script as usual.

Start developing at /home/joshuago/domain.com/webapps/ROOT/.

Change the directories as convenient or as suits best practices for your particular environment.

If you have multiple sites (and multiple instances of Tomcat) you'll need to edit conf/server.xml to make them listen on separate ports.

A fix for LaTeX "Missing $ inserted." console message

2008-05-15T22:09:00.000-07:00

While I was running pdflatex on a .tex file containing a math expression, I got this error message:

Missing $ inserted.

Powering through it would evaluate the math expression adequately, but I included a plus sign later in the same sentence. I followed the directions and made it convert to a PDF beautifully by adding a dollar sign at the beginning and at the end of the math expression. (It's not just at the end-of-the-line, like I would normally guess it meant in a Unix environment.)

So, what started off as 9*2^4 turned into $9*2^4$ and produced a beautifully formatted PDF.

I should also note that Norm Matloff has a great introduction to LaTeX.

EDIT: This one's even better. It just took a while to load and I got impatient.

Observations about making money from open source

2008-05-08T20:26:00.001-07:00

Open source is a great thing and I've personally benefited tremendously from it. It has let me deepen my knowledge, given me entire software platforms to invest my energies in, and has opened up a wide landscape of job opportunities. For a company though, it's pretty challenging to make money from it, and I recently I got around to thinking what successful companies in open source have done.

Arguably, Red Hat is the most successful open source software company out there. They came to mind not only because their product portfolio is mostly based on open source or that they've been around for a long time (relatively speaking), but because I actually use their products and find value that I willingly pay for. Good for them. But I wanted to figure out what it was they did in very general terms, because I'm definitely not going to try to take them head on.

Open source software has its strengths and weaknesses, where the strengths and the value produced often tend to accrue to the users and programmers of open source software, and the weaknesses plague them. Switch this around to the point of view of a company and you can actually get the opposite scenario: the strengths of open source software make it difficult for traditional software companies to make money, while the weaknesses in open source present an opportunity for an enlightened or creative software company to step in and fill the value void.

The strength in open source software is drawn from its unrestricted use and the consequences of unrestricted use. Anybody can just take the software, make changes to it, and repackage it. There are no restrictions on the use and distribution of the software. And since anybody has access to change the software, they often do — and the software often experiences a lot of change in a short period of time. In short, open source software is pretty great if you're a user: you can take it and do with it what you like, and there are always new features coming out so you never get bored or have to be stuck with the same old thing for long.

The weaknesses of open source software are analogous to its strengths. You could say that it's just picking up the other side of the same stick. Sure, open source software is easy to change and great for sharing, but when everyone's allowed to do this, there's a strong tendency towards chaos. Some efforts, like the Linux kernel or Mozilla, have very strong centralized cores that have survived challenges, but many others like XFree86/XOrg and FreeBSD/DragonFlyBSD have had very public internal debates. The fast pace of development also has its downside: frequent updates to existing software come out, but they come at such a quick pace and have not been well-tested. It's a hit or miss proposition when upgrading any reasonably important server to the latest version of a software package: it could break other software packages that haven't kept up. How does one get access to bug fixes or security patches and be assured that there are no unintended side effects?

Based on these observations and what Red Hat (and others in the industry) have done, here are a few guidelines on trying to make money from open source software.

Don't try to sell value that you didn't create

It's not a moral recommendation but a practical one: even if you tried to sell a repackaged version of what other people produced, nobody would buy it, because it's available elsewhere for free. You shouldn't just repackage the software and sell it like you would a traditional software package. Technically, you could repackage it for convenience, but you won't be able to make much money by charging high prices for your software. So there goes that plan.

Don't expect to sell value that someone else in the community can create/duplicate

Now, it's fine if you want to make enhancements to a piece of open source software to contribute something back to the community. Many companies do this: IBM, Red Hat, Novell, and Sun, just to name a few. Users really win big with open source software, because any feature in it will be available for free. Unfortunately, if you as a company try to tack something on to the project and sell it, your competitive advantage quickly erodes because you have to open up the source code (at least when using the GNU General Public License).

I say don't expect to sell value here because you can certainly try, and in some cases, succeed. But I'd say don't count on it. If you do want to try, make sure you enhance software that's covered by the BSD license, not the GNU GPL. If you choose the latter, you'll certainly have to open up the source code to any modifications you make. With BSD licensed software, you can keep it closed but include a note of acknowledgment in your software's licensing information.

Focus on what open source software finds difficult or impossible to address

With that said, open source software still has its weak points. All the openness can and does lead to fragmentation and inconsistency. Because people working on a project work only on what they want to rather than have a boss dictate items to them, a lot open source software lacks that extra polish that you'd expect from a consumer- or business-ready product.

I have a hunch that this is why open source programming tools are so clever and refined, while open source products meant for consumers seems to have a hard time getting past alpha or beta quality. Pretty much any open source program that a programmer would use on a regular basis and use heavily is bound to be very well done. Any rough edges are quickly smoothed out. But when a tool is built for non-programmers, the overwhelming tendency is for it to languish in mediocrity and stay under the radar of the wider community.

The business opportunities in open source software are rooted in the fact that open source developers overwhelmingly favor working on things that they want to work on. This means that the parts that aren't so fun are left undone. There is this desert where a programmer's passion only occasionally meanders — but it is wide open for businesses to step in, where another incentive which many of us are very familiar with, money, can take hold and flourish like native desert flora.

Where grassroots open source development stops, profitable enterprise begins. Here are some ways to make money from open source.

Client interactions and case-specific integration/support. It's fun to write new software, but not so fun to make sure that it works for every single person out there. Fortunately for the businessperson with an interest in open source, there are people who are willing to pay in order to get open source software working for them. Someone once said that open source software is only free if your time is worth nothing. There's truth in this, but that time can be dramatically reduced while avoiding being locked into closed source, proprietary software. Red Hat, Novell, IBM, and Sun make some decent cash through support contracts and servicing.

Subscriptions to updates that are well-tested and certified. There are software updates, and there are software updates that are guaranteed to work. I think Red Hat was extremely clever to set up their subscription system, and I admire it because I know that the updates coming from it aren't going to break anything. They do the hard work and make 95% margins on subscriptions. I can get on with my day rather than worrying about properly upgrading my server. It's unquestionably a win-win situation.

Coherent visual/brand identity that promises a certain expectation of quality. The open source community is vibrant, but it's also prone to fragmentation. Moreover, the traditional business practice of building a sustained competitive advantage through branding still stands in this new world of open source software. A brand still holds sway, and still serves as a focal point for trust. Novell, Red Hat, and Mozilla Firefox are highly respected open source flag bearers. It's really good once you get to that point, but, as in times past, building a brand requires a lot of hard work over many years.

Provide a stable platform: still open, but rarely changed because it has to work. There are two big problems with open source platforms: fragmentation and rapid change. Red Hat and Novell provide Linux distributions that can serve as stable platforms on which to build. They're infrequently updated and tightly specified. Ubuntu also has a Long Term Support (LTS) version of its releases, meant for the same purpose. It can be very rewarding to a company when it has control of a popular platform that holds a lot of developer mindshare. The vast majority of developers just want whatever they're building on to be predictable and for people to use their software. Providing a stable platform, trusted by developers and freely available to users, is a great way to establish thought leadership and make a name for your business. It yields a lot of advantages if you can get to the point where you are the trusted caretaker of a platform, but like branding, it's hard work and it's a long process only for the very committed.

Why the office of U.S. President should be available to all citizens

2008-05-03T12:36:00.000-07:00

Under our current Constitution, an American citizen is an American citizen — except when it comes to the presidency.

There are American citizens who will never be able to attain the highest office in the land, even with distinguished military service, community involvement, or a track record of political achievements. The one differentiating factor that isolates these citizens from the rest is that they happened to be born in another country, to parents who were not American citizens. This means that foreign-born American citizens — whether they were born in Mexico, the United Kingdom, Canada, Germany, China, or India — can never be President.

Nevertheless, the United States is one of the most accepting countries in the world when it comes to citizenship. Acquiring United States citizenship is a long and arduous process, but in theory it is open to all. And as many of us know, there's so much to be done and so much that could be done aside from coveting the presidency. In the political sphere, state governorships, seats in the Senate, and Supreme Court appointments are technically open to naturalized citizens. In the economic sphere, immigrants are increasingly providing the needed brainpower to develop new industries: a quarter of Silicon Valley startups were founded by entrepreneurs with a Chinese or Indian ethnic background.

Things are pretty great for the naturalized American citizen — but they could be better. There are a lot of good reasons to amend the U.S. Constitution to get rid of this last remnant of inequality, and not a single good reason to keep it as is.

Reasons to Amend

What we have now essentially delineates Americans into first- and second-class citizens. Granted, fundamental rights have been addressed thanks to the tireless work of the many who have come before us. And it's true that holding public office as a representative of the people should be viewed more as a privilege than as a right. But to lock people out of the presidency in the Constitution, for circumstances that were completely outside their control, creates an undeniable division of citizens into first- and second-class citizens. That's just the simple fact of the matter. Certain citizens get one more privilege than the others do. This distinction does not mean much to most citizens and does not impact them directly — running for President is not going to be a practical reality for the vast majority of Americans — but to be able to say, with absolute veracity, that our Constitution creates a system of first- and second-class citizens is just downright embarrassing when we claim to be the purveyors of equality and the beacon of democracy to the whole world.

What we have now does not effectively address the original concern with illicit foreign influence, and it is a blunt instrument in attempting to address this concern. The requirement for the President to be a native-born citizen was included in the Constitution during a fragile time for America, where the danger of foreign influence was very real, and whose potential consequences could have destroyed America early on. In short, it was justifiably paranoid and the drafters of the Constitution were right to include it at that time. But we have no justification for keeping it now. Voter sentiment about foreign influence and mixed allegiances presents the biggest barrier to ratification, and I'll expand on this point further in the next section. But in short, the current clause does not serve its purpose — ensuring loyalty and allegiance to the United States of America — and there are better ways of demonstrating loyalty and commitment.

Ratifying such an amendment would do much to integrate immigrants into the fabric of American society. By sending the message that immigrants, once naturalized, are truly equal in the eyes of the law by being eligible for the highest office in the land, this amendment would give them a stake in America's future. It could also lead to a representative figure.

Further Elaborations on Foreign Influence

It's completely valid to ask for a sign or indicator of loyalty to the United States. Where you're born is a blunt and ineffective instrument for doing so. If anything, let one's military service record be the testament of loyalty and dedication, not one's place of birth.

The original intention and the situations surrounding the initial amendment are not valid now. Locking a restriction into the U.S. Constitution was a necessary precaution to very real concerns. Keeping it locked in shows a mistrust in the American people to select a good Presidential candidate.

Accentuating the Positive

For those who are overly pessimistic about the current situation, let me just give an overview and explain why I think the current system is actually quite good. By accentuating the positives, my intention is to frame the debate in more forward-looking viewpoint that's conducive to progress, and not wallow in despair over the hopelessness or xenophobia inherent in the situation.

The current system does allow for children of immigrants to become President. If you're born in the United States, you are automatically a natural-born citizen, even if you are a Gonzales, Zhang, or Singh. That part is taken for granted, and in arguing for something better I certainly don't want to take the current system for granted. Rather, I want to make it a point to celebrate the tremendous opportunities already available.

Closing Remarks

There's some merit in the argument that political energy should be diverted to more pressing concerns like the economy, health care, transportation. There are more important things to address in the meantime.

Some would argue, "If it ain't broke, don't fix it." But I'm saying that it's broken. Leaving rotten parts in the system leads to decay. But I acknowledge that, however simple it is, it's still a small issue compared with the bigger problems facing us today.

Since this is such a no-brainer, then, I suggest that it be put forward for passing as a bipartisan measure to encourage working together across the aisle, especially when more divisive issues take center stage. It will be a symbolic gesture to the American people and good practice for our elected representatives. I would welcome a well-argued case for keeping things the way they are.

A lot of times, the status quo is there for complicated reasons that are unknown to us, and set up in a way that we should not try to tamper with. I do not believe that this is one of those cases. The issue is simple, and all that's holding us back is our own reluctance and inherent mistrust of change, even if it's clearly for the better.

References

Help not wanted (The Economist)

Should the Constitution be amended for Arnold? (USA Today)

A pleasant surprise from Pentel

2008-04-25T12:40:00.000-07:00

I was raised on Quicker Clicker mechanical pencils from Pentel. Back when I started using them as a wee lad, they felt heavier and sturdier since they were crafted out of more metal content. Now, Quicker Clickers are mostly made of plastic. But like all Pentel products, they still carry a lifetime warranty.

For pretty much anything else, I don't care about lifetime warranties. First of all, I'm not going to take the trouble to dig up my receipt for something I bought years ago. Second, I just don't care that much about most of the stuff I buy.

But I take my writing implements very seriously. For mechanical pencils, I prefer the Quicker Clicker over anything else. For erasers, I like the Staedtler Mars Plastic but I'll settle for a ClicEraser from Pentel. For ink pens when writing things in my planner, I will go out of my way to get a Uniball Signo RT Gel, 0.38mm in either black, blue, or red.

I keep several Quicker Clickers around at all times to ensure optimal performance, but recently one of them broke. I've sent broken pencils to Pentel before, and they've sent back new ones — without requiring the hassle of presenting the original receipt. I knew that they followed up on their lifetime guarantees.

But this time around, they not only sent it back new, but also filled it up with lead refills and included a separate 12-pack of Pentel lead refills.

It's a small gesture, but it's much appreciated from this customer. Thanks, Pentel. Your products continue to be expensive up front but they are totally worth the premium price.

Growing responsibility on a solid base

2008-04-14T23:02:00.000-07:00

People who cannot handle small, easy duties should not be entrusted with more power and responsibility. Basically, be faithful with the small things, and you will be entrusted with bigger and better things.

I particularly like this lesson because I love the process of building and focusing on smaller tasks, but at the same time I've always been a big dreamer. As one would expect, though, I also experience that impatience to move on to the next step, even if I'm nowhere near ready.

But that's the way the world works.

I've been thinking too, that the ability to wield power and responsibility is not entirely intrinsic. True, you can be put in charge of small things as a test of your potential, which is a property that never changes, and it does serve as a way to winnow out the good from the bad.

But I realized that the process and experience of handling the little issues and tiny problems is itself critical to bringing out any latent potential or untapped talent in people. So essentially, you've got to be trustworthy in the first place, but handling the little things is crucial practice for the big leagues.

Reading old translated classics

2008-03-03T22:55:00.000-08:00

The past couple of years, I've tried to become a more cultivated person by trying to read the "classics" — old books revered by well-read people far and wide. A lot of literary contributions have been in languages other than English — the French and the Russians in particular have made tremendous contributions to world culture — and in reading some French and Russian classics I've wondered how much I'm missing by not reading them in their original languages.

What I've found has been utter genius as far as pure ideas go, and eloquent choices in figurative language. Both translate well, as one would expect. As far as clever turns of phrase, I know I'm missing out because the wording hasn't seemed terribly impressive in the translations of Les Miserables (by Victor Hugo, originally in French) and War and Peace (by Leo Tolstoy, originally in Russian with a lot of French phrases). Even in translation, though, I found Victor Hugo to be frequently poetic even in prose because of he often pulled in some gorgeous metaphors.

Tolstoy was the same too, only his were more gritty. Sure, he wrote grandly of the ocean of humanity, but he also compared the movements of armies to the inner workings of a clock, made some references to ideas from calculus, and compared the actions of humanity to the steam engine.

Reading long 19th century novels in translation, I admit, was an effective way for me to relax and fall asleep at night. I pressed on because the stories were interesting enough. Still, though, with the attention span of most people these days, I think I know why the new books released tend to be much smaller than many of these "classics" that I read. Either readers are more impatient and pressed for time these days, or authors have become better at getting their points across with fewer words.

In any case, it'll be good to mix things up by reading something shorter and something that was originally written in English, so I've started reading All the King's Men, a much more recent American classic.

Project idea: structured financial data

2008-02-29T23:16:00.000-08:00

As they are, things are pretty good for everyday investors on Main Street. The United States Securities and Exchange Commission makes the filings of public companies readily available on its website. If someone were interested in researching a company thoroughly, pretty much all the information is there: how much debt a company is carrying around, what kind of revenues it's pulling in, what the company's assets are worth, and so on.

So, things are pretty good, but they could be a lot better.

Now, I say "investors" and not "traders" because traders will typically need more than just information from the SEC. It's also a clear distinction that will limit the scope of the project I'm about to propose so that it's very clear what will be needed (and, more importantly, what will not).

The current situation

All this data is in HTML. It's great for human readability. Right now, that's exactly what everyone's doing: reading them one by one and poring over them. This is probably because there are a lot of loose, interesting facts that are specific to each company's way of reporting things. But among all that loose, unstructured data are structured tables containing information like a company's balance sheet, its cash flow, and its earnings.

For a stock market related project my team and I are doing, such data in a structured format would go a long way. Currently, there are data providers who would provide this kind of thing for a monstrously large monthly fee, perhaps because they bundle it up with other things. But this is public data. Nobody should have to pay a fee for it.

I personally tried to write a parser to automatically pull this data. I was reduced to entering some data by hand, but with around 8,000 companies to deal with, that would take me a couple of years, full-time. There is no way that this is feasible. Entering data by hand was pretty easy, but really tedious. What a human being can deduce by a quick glance, an automatically programmed parser would have trouble with.

Such a task would be, in my opinion, well-suited for being built and maintained by an open community. It's a big job — not particularly hard, just a lot of work. It doesn't make sense for any one company to feed these in; the ones that have this data are charging high rates for it. Everyone should be able to benefit from it, so contributors should include people from various organizations.

A proposed solution

The best solution I can think of would be to take the lead on the project initially. There would be a project website running on a simple web application. Since the number of companies is known, there would be a frequently updated status report saying what percentage of the task is done. For example, the front page could say "3000 out of 7500 companies with complete data" or "400 companies with incomplete data." To ensure the project's open nature, there would be a daily or weekly dump of the database. That way, no one party will control the data completely. Additionally, the community would have a reasonably complete backup.

It is essentially a massive data entry project. Not so fun, but for the participants there is potential fame and fortune as well as a strong mission. Once the data is available, programmers and financial analysts can graph data that would be buried in tables and easily compare data for multiple companies in a timespan on the order of a few seconds, rather than the current process, which could take minutes, hours, or even days.

The fields that would be stored would be standard items on a company's balance sheet, earnings statement, and cash flow statement. Eventually, if further patterns can be found, more fields would be added. There is a much more ambitious initiative which attempts to address the problem of all the fields, which I will discuss below.

XBRL

Right now, there's a major push underway to address this exact problem: XBRL, which stands for eXtensible Business Reporting Language. I looked into it several times, and have found that it is characterized by slow progress and a quiet reluctance from a lot of the parties involved. Companies in the XBRL test program are few and far between. A lot of big names in financial research are entrenched, and they depend on keeping this structured data as revenue sources. It should be noted that if XBRL somehow succeeds, this open community project I'm proposing would not be needed at all.

The eventual XBRL specification that would be used for public companies' SEC filings would have structured fields to address all the data reported, but this could be a potentially huge problem because of the learning curve for new software tools that report preparers will have to get used to.

If there are any interested readers out there, please leave a comment and let me know what you think. This idea has been in my head for a couple of weeks now and I'm certain I left some gaping holes in my discussion of what I have in mind for this project.

The gospel for stock market junkies

2008-02-21T22:05:00.000-08:00

One of the recurring struggles among people who are part of the evangelical Christian tradition is finding the balance between faith and good works — knowing where each of them fits in and what they're for. Since I've been trying to read three chapters of the Bible every morning to get my bearings before jumping into what are typically full days, I read over something that reminded me of this.

Now, the "full days" have been dedicated to learning about stock market investing and exploring various angles for the individual investor to take when selecting stocks.

The price of a company's stock takes thousands of factors into account. One way to split up these factors is to consider 1) the book value or intrinsic worth of the company, and 2) the other things like projected growth, macro-economic concerns, takeover speculation, and a host of other factors that are not directly tied in to the numbers that the company is posting.

The stocks of most healthy companies will trade at prices above their book value. But when they're hammered by a bad economy or growth slows down, they won't go down to zero if they're solid companies, because people called value investors will swoop in and "support" the stock at a price at or somewhere near its book value. It's like a safety net, and these value investors keep stock prices at certain levels so they don't go down too far.

In good times and when a company's prospects look good, its stock will trade above the price set just by the raw numbers, whether the numbers are already in the bag or just strongly projected. At every point though, it's supported by its raw value, and every other gain in price is tacked on as extra.

I see us like stocks, and our spiritual well-being like the price of a stock. What God does for us is He gives us that solid book value. To get our balance sheets and cash flow in decent working order, so to speak, we must have faith and accept the Lord's gift of forgiveness. Then we're fixed, and our intrinsic worth is affirmed; there's no taking that away. We may be battered by the storms of life but we don't sink down past a certain level, because God sees value in us.

That faith can be enough to get by, but just as investors want to see a stock take off, I think it's in our interest also to grow, and not just rest on our laurels like some stocks languish at seemingly constant lows.

Faith gives us something like the value investor-supported base and ensures we don't sink past a certain low, but doing good deeds gives us growth and dynamism. On top of that, our good deeds often feed back to strengthen our faith, just like a company making constant process improvements and aggressively pursuing business will see the results feed back into the bottom line and strengthen the base of its value.

Freedom is shabby and inconvenient

2008-02-19T21:51:00.000-08:00

As much as we value liberty and tout the virtues of living in a free society, I humbly submit that eternal vigilance is not the only price we pay for our freedom.

Anywhere there's freedom, you will find disorder, inconvenience, and shabbiness. We allow our freedoms to be restricted because we want some sort of orderliness, convenience, or good aesthetics. These are good things, but conceding to rigidity often means losing substance or meaning.

Take homeowners' associations, for example. With pretty much any HOA, you cannot get away with painting your house bright pink, letting the weeds grow to four feet in your front yard, and then parking your car among the weeds. It may be how you prefer to live your life and run your household, but by joining the HOA you are giving up freedoms that you would otherwise have, in exchange for the assurance that your neighbor across the street will not paint his house a loud pastel purple, because pink is okay but purple is not. You may be fine with how shabby your house looks, but you'd rather your neighbor not make his house an eyesore for you. So what you end up with is people being restricted; everyone's content, but chances are that nobody's house really looks any different from anyone else's.

This year, I'm living in Irvine, California. Having lived in more freewheeling places in California, like Davis, or San Bernardino, or Monterey Park, I feel the contrast here. Everything is much more convenient and well-planned, but it's hard to find shops that aren't part of a large national chain. In place of character and charm, I've got well-run homogeneity — which I actually like, but which plenty of others detest. (Even San Bernardino, with neither charm, character, nor well-run homogeneity, had good Mexican food.)

It's kind of like this with people, too. People may dress sharp, but I've found that many people who have taken the time to develop character on a deeper level, or hone their talents, are not the most fashionable or up to date when it comes to the latest trends. Of course, if you were really smart and considerate, you wouldn't be a complete slob and smell utterly foul due to all the time you spent "finding yourself" with little thought of more trivial things. All I'm saying is you just have to look good enough.

Or take a look at any Linux distribution, and compare it to Apple's Mac OS X or even to Microsoft Windows. Running Linux is the ultimate celebration of freedom, if you are using a computer. But you have to go through a lot of trouble to get wireless networking going, or even printing. The font display on any Linux distribution is mediocre at best, but you have access to change all of it, if you are so inclined. The trouble is, it's not fun at all to make these changes, and a lot of these difficulties arise due to inconsistencies and disagreements among all the "free" people about how to do things. With a proprietary, tightly controlled platform like Apple's or Microsoft's, there is much less freedom to tinker, and you have to trust that they make things "just work" but that's precisely why one would give up one's freedom. Most people just want to make things work.

Sometimes, we give up absolute freedom because it's more convenient to do so. Those who restrict us should give us something good in return, but I hope we don't lock ourselves in so tightly that there's no going back. Liberty has a price, and on top of that, it's not always pretty.

Lavish in praise

2007-11-14T00:29:00.000-08:00

I make it a point to reinforce others by highlighting the good things about them. In doing so, I hope to make their days a little brighter and bring out the best in them. But, as usual, there exists a gap between idealism and practical reality. I am a man of few words, which usually means I can stay out of trouble, but I've found that more words are usually better when giving quality encouragement.

For example, I could say, "I like your painting."

What actually goes through my head when I'm thinking that up is, "I like your use of high contrast. I know you went out of your comfort zone here since low contrast colors are your usual fare, and this turned out well. I noticed the attention to detail you put in the left section where the fence is. How did you manage to pull off those shadows?"

But I somehow can't put it into words quickly enough.

Or, I could tell someone, "You don't have to worry. You have what it takes to succeed." If my buddy is down in the dumps and grasping desperately at any reinforcement possible, this will probably make him feel better.

But it's so much more satisfying to hear, "Look, you've got what it takes to succeed. I've seen your analytical and problem-solving skill when I was taking classes with you. I noticed that you broke things down into manageable pieces instead of tackling big impossible problems all at once. You may think your attention to detail is nothing special, but once you get out there you're going to find that it's really a rare quality. And I know you're very ambitious and very driven. You're just feeling a little apprehensive because this is a new situation. If you really want me to name off one thing you need to improve on, it's relaxing under new situations."

Expounding upon the details does two things: it validates your sincerity, and deepens the sense of encouragement you impart upon the other person.

If you absolutely can't think of more to say, you're probably not being sincere. I questioned my own sincerity many times, but at this point I'm sure that the problem is usually that I'm a little rusty getting words out since I'm writing code all day. For me, having to talk about something means I have to know it pretty deeply and pretty well.

I have a theory about why I get a deeper sense of encouragement when people go into details about my good qualities. It gives me many hooks to latch onto, little things that I can evaluate as true or false. Am I really good at this? Wow, she thinks I'm good at handling people? I know I wasn't born like that — in fact I am pretty socially awkward — but it's good that I come off as a natural. The more hooks there are that sound true enough, the more thorough and more solid is my self-esteem after a friend's pick-me-up.

You mean you can't read my mind?

2007-10-14T01:13:00.000-07:00

Once again, I've had it hammered into my head that communication is important. The problem always boils down to me thinking that people can read my mind. In my traditional step-wise approach to problem solving, I hunted for a first step to take. For this first step, I chose to answer the question, "Who needs to know what I'm thinking and doing?"

At this point in my life, it seems that I've got to communicate with a lot of people. It's far from an isolated time.

At the very minimum, I've got to communicate with my girlfriend, my family, my housemates, my co-workers, and my customers. That's a lot of communicating to do — and this is just the bare minimum to get by. It's hard to even address this bare minimum.

Now for the second step: given this narrowed down set of people to communicate with, what potential problems are there? The answer to this is different for each party concerned.

For example, with my girlfriend, I've got to pick the right words and be sensitive to what she's feeling. As a guy and as a software engineer who gets very absorbed in my work, this is difficult. So I turn to literary giants such as Tolstoy and Hugo to teach me how to express myself.

With my co-workers and customers, I've found that the biggest hindrance to my communicating effectively is that things get too busy and I just want to cross items off the to-do list. Often, that's not enough in a business environment because people have to be aware of what you're doing. Often times, communicating means slowing down to express myself. This is at the expense of absolute personal productivity, but in a team environment this is necessary overhead.

I have yet to figure out what keeps me from calling my mother more often. I've just moved into a new house so I'll have to figure out how to communicate with the housemates. It's funny because people can live together while not communicating at all.

Here's to good communication and not expecting people to read my mind.

My change-the-world money

2007-06-19T21:55:00.000-07:00

Money makes the world go 'round. No matter how many of these heart-breaking letters Doctors without Borders, the Grameen Bank, or Amnesty International send me, I won't have anywhere near enough in the foreseeable future to send much to all of them. I can only pick one at a time.

I once read that good entrepreneurs are inherently insecure people — that they're insecure about their material wealth and status, and so they work hard to acquire more, and are never satisfied. In this sense, I worry about my own effectiveness as an entrepreneur, because if that greed is what has to serve as the driving force or the fire or the motivation — whatever you wish to call it — then I am severely lacking it.

It's commonly accepted that the most powerful force that can drive a man is self-interest. Now, the only reason I see for amassing billions in personal wealth is to make the world a better place. I'm a little embarrassed to admit such a touchy-feely motivation — I like to think of myself as a hard-driving capitalist. The people I see most effective in the world are those with loads of cash, and the touchy-feely people I know seem to be digging dams with spoons.

I could make a difference on a small scale, and it's true that everything has to start on a small scale. Even in my line of work, I work iteratively and start small on pretty much everything I do. Maybe one day I'll be okay with just chipping away at enormous problems in the world rather than blowing them to smithereens.

China Import and Export Fair in Guangzhou

2007-04-24T23:00:00.000-07:00

Last week, I went to Guangzhou, China for the 101st China Import and Export Fair (中国进出口商品交易会).

Here's the view from our 18th floor hotel room. We got a better deal on our hotel than most of the other visitors because my father's friend is from Guangzhou. This friend's sister-in-law is a hotel manager.

Everything in China just has to be huge.

The big heavy machinery was necessarily outside. Here's a forklift company with shiny forklifts.

I have no idea what this power company does, but they had a nice booth.

Because of the immense size of the fair and all the heavy catalogs I was carrying around, I had to step outside to rest my feet and my shoulders for a bit. This is the place right by the street where people were allowed to step outside for a quick smoke.

The following is a large expanse that they were just preparing for the second phase of the trade fair.

Inside the complex, there were even more displays, but the booths were far smaller.

Here's my father at the Health and Beauty Products portion of the fair, with a representative of the Chinese company that makes the pictured device.

Finally, proof that I was actually at the fair. I think I had to run out of the way very shortly after this picture was taken, so that a bus full of people could get through.

This was a very eye-opening trip for me and I've written down a lot of ideas from the trip. I took enough notes to be satisfied, but I could have taken more pictures, and I regret not taking one of the Pearl River at night with the boats and buildings all lit up and flashing.

Ethanol can be made from corn or sugarcane

2007-04-10T22:31:00.000-07:00

For as long as I can remember, I've been excited about environmentally-friendly solutions.

For energy, my dream for clean energy centered around solar and wind power. For barren soil, it was composting. For everyday fuel, I thought of renewable sources such as ethanol or hydrogen. Now I'm glad it's becoming more widely adopted; I thought I'd never see the day when such things would actually become economically feasible.

There's an ethanol craze sweeping America at the moment, and I hadn't bothered to read much about it until this week's Economist leader, "Castro was right," pointed out that there are two main ways of producing ethanol on an industrial scale: corn and sugar.

Ethanol advocates often point to Brazil as a shining example of a large country that uses ethanol on a large scale. Knowing that Brazil is doing fine with massive ethanol deployment made me more excited about having it in the United States, until I found out that their ethanol is made from sugarcane, not corn. The advantage to having it made from sugarcane is that the energy requirements are lower, and it's much less subject to the anti-ethanol accusation that it costs more energy to make the fuel than we can get by using it.

Now, the idea of making ethanol from corn is more likely to fly in the United States. I like to think practically, and I know that the farm lobby is a formidable force in American politics. Corn-based ethanol has broad support, and it's likely to enjoy subsidies.

According to the USDA, sugarcane is already grown in Florida, Hawaii, Louisiana, and Texas. The Economist article named developing countries with tropical climates (India, the Philippines, Cuba) as possible sources of sugarcane-based ethanol. If that were feasible, that would be a great way to get business going again in the Philippines, where my father shut down his factory due to intense competition from China and Vietnam. The energy business can be very lucrative.

JavaScript syntax highlighters

2007-03-29T10:17:00.000-07:00

I'm really into writing documentation, and very often, this documentation must include code samples.

I wanted a JavaScript syntax highlighter that:

involved very little hassle in setting up,
worked like a real parser using parse trees to ensure accurate highlighting,
and provided decent coverage of the most common programming languages in use today.

Obviously, by opting to go with a JavaScript syntax highlighter, I chose to forgo the option of doing my syntax highlighting on the server side. Doing it on the server side still interests me, but for immediate and practical reasons, the quickest way to get syntax highlighting in my documentation would be to do it on the client side.

I went searching around for JavaScript syntax highlighters, and these are the top three that met my criteria.

Javascript code prettifier. This was the first decent one that I found. It's pretty basic but I got up and running very quickly. The documentation is sparse and the tone of the README quite terse, but it meant there was less for me to sift through. The neat thing about it is that it has automatic language detection.
SHJS. This is more full featured than the Google tool, and supports a wider range of languages. It appears to be more mature than the other two. Unlike the Google solution, it doesn't offer automatic detection of programming languages.
JUSH. I'd like to check it out in more detail later on, but the website says it only supports HTML, CSS, JavaScript, PHP and SQL. I needed highlighting of Java and Ruby, not to mention C and C++.

For now, I'm going with the Google code prettifier, just because it was the first one I found, integrated, and liked. One that I did not like was dp.syntaxHighlighter, mostly because it required that my code be in a TEXTAREA instead of a PRE (pre-formatted field).

Where the rubber meets the road

2007-03-28T21:55:00.000-07:00

Back when I was writing software that ran on HP LaserJet printers, I hated having to wait 45 minutes for one-line changes to compile (in a ClearCase build system) and then burn to flash memory. Sometimes, a problem with the printer's behavior wouldn't even be a problem with my software. It would sometimes be caused by an obscure bug in the hardware.

Because of time to market pressures, the team would have to work with prototype hardware, the design of which wasn't even finalized yet. The guys who wrote the low-level code had to write some registers and hack their way around hardware defects.

Somehow, it seemed wrong to me that I should constantly be worrying about the integrity of the underlying hardware or the operating system on which my software ran. I should have been able to take it for granted.

I longed for the day when I could instead work on web applications, where I could safely assume that the hardware was fine and that there were no major defects with the platform on which I was building. The platform on which my software ran would come already debugged, I figured.

Since diving more deeply into web application programming, the experience has been as good as I had imagined, for the most part. There's always a lot of documentation out there, and it's usually just a matter of searching through Google results to see which one relates best to the problem I have at hand. I rarely get stuck for long.

If you ask most web developers, the worst thing and most frustrating thing about writing web applications would probably be cross-browser compatibility. Different browsers handle CSS and the JavaScript DOM a little differently. It's the bane of many a web developer's existence.

When I'm working on getting a web application working with different browsers, the thought crosses my mind that I'm wasting a lot of time by having to check for compatibility with browsers that should just work. It seems like an enormous and unnecessary waste of time.

After thinking about it, though, every software industry has its area of "inefficiency" for developers, and that's usually where the rubber meets the road. It's where the software, however elegant it is, has to face the real world in all its ugliness and imperfection.

In web applications, cross-browser compatibility in web applications is needed because that's where the application faces the real world, where people use all sorts of different browsers.

With Unix software, there are portability issues between different kinds of Unix. There's the worry of having to work with different thread models and libraries. There's the process of having to build with different compilers, not just gcc. There's GNU make and BSD make. The #ifdefs and #defines aren't any prettier than JavaScript workarounds.

Network software has to know how to deal with extensions to a standard that may not be universal. The RFCs are great: they've helped to avert major compatibility issues in the past, but technology companies are inevitably going to shoehorn in their own little extensions to network protocols. Clients need to know what servers are capable of, and servers likewise.

The point is, when I think about it, having to check for cross-browser compatibility doesn't seem all that bad. Yes, it does mean that hours and hours will be gone from the day trying to hunt down the problem. Yes, it could have been better if Microsoft had tried to follow standards in their implementation. Yes, many other things could be done in the future to better the current state of things: there are libraries and frameworks out there that abstract away much of the pain, just like people have figured out how to create portable Unix programs (grab a copy of the autotools and run a configure script). But at the end of the day, the reality is that people out there are using different browsers to view the same site, and it's my responsibility to make sure my application works. It often seems like a huge waste of time, but it's that final push that really makes the difference.

Physical media and bit rot

2007-03-17T22:59:00.000-07:00

Last night, I played some of my old classical music CDs — good music to program to. The songs skipped every other second because of the scratches on the discs. It appeared that my careless handling of CDs over the years eventually caught up with me.

Tonight, I ripped some of these to Ogg Vorbis (an open and royalty-free file format similar to MP3) to see if I could listen to my songs without the skipping. Sure enough, they were all smooth!

Now, none of this is stuff that I'm terribly surprised about, but it just served as a reminder to me that relying only on physical media to store data is a little dangerous, especially when the medium goes bad or acts flaky. The lifetime of information on the network is short, but as long as it's out there, stored and accessed frequently, it's constantly being duplicated and refreshed.

Information that's alive and circulating is healthy information.

HBR: "Leading Clever People"

2007-03-15T13:31:00.000-07:00

This month's Harvard Business Review contains an apt and timely article, "Leading Clever People." Rob Goffee and Gareth Jones define clever people as "the handful of employees whose ideas, knowledge, and skills give them the potential to produce disproportionate value from the resources their organizations make available to them." Still, this doesn't mean that they're better off working on their own. One of the people they quoted, the head of development for a global accounting firm, stated that clever people "can be sources of great ideas, but unless they have systems and discipline they may deliver very little."

One good point they made about managing clever folks is the importance of demonstrating that you're an expert in your own right. This is to establish credibility and respect. At the same time, one mustn't be so above-and-beyond or in-your-face so as to discourage the real talent.

Read "Leading Clever People" at HBR. You may have to view an ad.

Parsing lists of e-mail addresses

2007-02-22T17:32:00.000-08:00

I came across a situation where I had to parse a list of e-mail addresses. E-mail clients these days take e-mail addresses in two forms: one showing the name of the individual as well as their e-mail address, and one with only the e-mail address.

When multiple e-mail addresses are listed, they are separated by commas, whether they're of the full form or of the simple form.

When I had to extract the list of e-mail addresses initially, I assumed only that I could separate them using commas. This would capture a list such as the following.

"Joshua Go" <joshua.go@playpure.com>, joshuago@gmail.com

It would capture two e-mail addresses: "Joshua Go" <joshua.go@playpure.com> and joshuago@gmail.com.

A problem arose when I came across one form of the full e-mail address that threw off my simple parsing technique: the occurence of e-mail addresses such as "Go, Joshua" <go.joshua@yahoo.com>.

Since I am no master of regular expressions, and working with regular expressions in Java has somewhat been painful for me, I decided to review my EBNF parsing.

The following is the EBNF syntax, from what I know.

EmailAddressList    = GeneralEmailAddress [ ',' EmailAddressList ] ;
GeneralEmailAddress = [ RecipientName ] '<' EmailAddressOnly '>' 
                      | EmailAddressOnly ;
EmailAddressOnly    = Username '@' Domain ;

My co-worker, Wilson, pointed out that I defined neither RecipientName, Username, nor Domain. For that, I cite the practical demands of industry as my explanation for not adhering to strict academic formality. I also omit it for clarity. Basically, assume that they'll just be alphanumeric (letters and numbers).

Perhaps in a later post, I'll put up the source code to the parser. As it stands, I've yet to move it over from being a test program to being integrated with the rest of our product.

Strained relationships and great undertakings

2007-02-09T13:59:00.000-08:00

Last night, I picked up a book and read the preface at the beginning of the book, and the "Special Thanks" section caught my attention. The author thanked his wife and daughter for "putting up" with the authoring process. This isn't the first time I've seen that kind of thing written in a preface.

Does writing a book necessarily have to put a strain on the author's family?

Arrays in Visual Basic and classic ASP

2007-02-08T10:39:00.000-08:00

For the programmer who is used to C-like syntax, working with arrays in Visual Basic or classic ASP can be aggravating.

In this post, I will briefly go over declaring single- and multi-dimensional arrays, then iterating through them — the basic operations that make arrays useful.

One-dimensional arrays

Let's declare an array with six elements.

Dim OneDimArray(5)

Yes, that says "5", but it has six elements. When we're going through the elements of this array, we'll start counting from zero and end at five.

Iterating through one-dimensional arrays

For i = 0 to UBound(OneDimArray)
    Response.Write(i)
Next

There will be six elements iterated through.

General notes about arrays in Visual Basic

So far, we're left with the impression that Visual Basic is a strange language. When we declare arrays in VB, the real size is the declared array size plus 1.

If you're used to programming in a C-like programming language such as C++ or Java, it's the declared array size, period — although you still start counting at zero. The following would give you a five-element array of integers in C++ and Java.

int one_dim_array[5];

You would access the elements of this C++/Java array with one_dim_array[i] where i = 0,1,...,4. Accessing it with index 5 would take you outside the bounds of the array.

In Visual Basic, however, you get a six element array when you declare the following.

Dim OneDimArray(5)

You can access OneDimArray(i) with i = 0,1,...,5.

Multi-dimensional arrays

The following is a declaration of a two-dimensional array.

Dim TwoDimArray(4,2)

This declaration would give us an array with five rows and three columns.

Here's how a three-dimensional array is declared.

Dim ThreeDimArray(5,6,7)

This gives us a 6 by 7 by 8 array. That's (5+1) by (6+1) by (7+1) because of Visual Basic's array syntax.

Finally, we'll generalize into the case of an n-dimensional array.

Dim EnDimArray(x_1, x_2, ..., x_n)

Iterating through multi-dimensional arrays

Here's how you'd iterate through a two-dimensional array.

For i = 0 to UBound(TwoDimArray)
    For j = 0 to UBound(TwoDimArray, 2)
        Response.Write(i & "," & j)
    Next
Next

The major difference between iterating through this and iterating through a one-dimensional array is that we called UBound() with two arguments instead of one. This is so that UBound() knows which dimension to look up the upper bound for. In this case, our inner loop stops at the upper bound of the second dimension — that's why we specified the "2" there.

For three dimensions, the logic is similar.

For i = 0 to UBound(ThreeDimArray)
    For j = 0 to UBound(ThreeDimArray, 2)
        For k = 0 to UBound(ThreeDimArray, 3)
            Response.Write(i & "," & j & "," & k)
        Next
    Next
Next

In the innermost loop, we specified that we needed to look up the upper bound for the third dimension.

Finally, the general form for iterating through an n-dimensional array's n dimensions.

For i_1 = 0 to UBound(EnDimArray)
    For i_2 = 0 to UBound(EnDimArray, 2)
        ...
        For i_n = 0 to UBound(EnDimArray, n)
            Response.Write(i_1 & "," & ... & "," & i_n)
        Next
        ...
    Next
Next

Why we need UBound()

The UBound() function can be called with either one argument (just the array name), or with two arguments: the array name, and a number representing which dimension we want to count the upper bound on.

Without UBound(), we can't know the upper bound for the particular dimension we're looping through at the moment.

Accented characters with a US keyboard in X11

2007-02-07T17:12:00.000-08:00

I've always been too busy to figure out how to map the useless Windows flag keys on my keyboard to do something useful in Linux/X11.

On traditional Unix systems, there's a Compose key. According to Wikipedia, "On some computer systems, a compose key is a key which is designated to signal the software to interpret the next keystrokes as a combination in order to produce a character not found on the keyboard."

To see what the Windows flag and menu keys are mapped to, I ran the following.

xmodmap -pk | grep 11{5,6,7}

This resulted in the following output:


    115         0xff20 (Super_L)
    116         0xff20 (Super_R)
    117         0xffcc (Menu)

This output told me that the flag keys (left and right) were free to map to Multi_key. I figured I would leave the menu alone.

Next, I had to perform the remapping. I created a file, .Xmodmap, which is sometimes already there in a user's home directory. For me, it wasn't, so I went ahead and created .Xmodmap with these contents:

keycode 115=Multi_key
keycode 116=Multi_key

I then refreshed my keyboard mapping by running:

xmodmap .Xmodmap

To see if the re-mapping worked, I held the flag key down while pressing the n key, and typed in a tilde (~). The resulting character was the ñ character. Presto!

Goals are good for people who tinker

2007-02-02T15:09:00.000-08:00

I like to tinker so that I know how every little bit works. I also like the feeling of knowing I've achieved something. I'm sure it's already apparent to the reader that these tendencies sometimes come into conflict with each other.

My tinkering leads to more extensive knowledge. Undoubtedly, it has helped me many times in the past, and I continue to reap the benefits of my past fiddling.

Still, I hold myself to strict standards of productivity, and I become frustrated when I can't do enough in one day. Setting clear and reasonable goals for myself allows me to satisfy both my hankering to tamper and my drive to have something to show.

Separate user for each webapp's MySQL database

2007-01-30T13:48:00.000-08:00

The following commands will create a separate user for each MySQL database.

> GRANT ALL PRIVILEGES ON PureExample.* TO 'pureuser'@'localhost' 
      IDENTIFIED BY 'purepassword' WITH GRANT OPTION;
> FLUSH PRIVILEGES;

This is good practice when writing web applications, so that each web application has its own database user.

PDF bookmarks with LaTeX

2007-01-28T16:27:00.000-08:00

After the mail fiasco was taken care of, I decided to pursue one of the things I've been wondering about: how to make PDF files have bookmarks (those links on the side pane that let you jump to well-defined sections of the document). For my setup, all I had to do was add this in my document:

\usepackage[pdftex,bookmarks=true]{hyperref}

Since I use Ubuntu, I had to install the tetex-extras package, which contains hyperref.sty for running pdflatex.

Thunderbird's "too many connections" message and Courier IMAP

2007-01-25T15:26:00.000-08:00

When we use Mozilla Thunderbird at my workplace for our IMAP mail, Thunderbird often tells us that we have too many connections. It makes going through my folders impossible. This article explains that it's because Courier IMAP (which is used by Plesk) limits the number of connections from a single IP address to something like 4. Since we're all behind a firewall, we're all considered to be one user, as far as the server is concerned. I just had to change the values of MAXDAEMONS and MAXPERIP in /etc/courier-imap/imapd to allow more connections. Mozilla Thunderbird caches something like 5 connections by default, so even one user will put the IP past the limit.

While I was figuring my way around this problem, I set up another server to copy my mail over. It's an Ubuntu machine and I just used the distribution's Courier IMAP, but since I've been using UW-IMAP all these years, the configuration process for getting a system user up and running was completely foreign to me.

It turns out all I had to do was run this as that user in the user's home directory:

maildirmake -S Maildir

So now I have a nice little IMAP setup, pretty much out of the box. Some assembly required, apparently. To really understand it, I should do some deeper reading of my source for this command.

One problem with my setup right now with this single command is that it's not yet integrated into the MTA on the system, so that mail can't be delivered into that mailbox, only copied via IMAP in a mail client such as Thunderbird.

Iteration produced the Constitution

2007-01-24T22:40:00.000-08:00

The other day, I was in a meeting with a potential customer. I explained that when I'm faced with the daunting task of having to build something complicated, I take what's called an iterative approach.

Taking an iterative approach starts with an early phase of exploring the problem by creating a quick, throw-away prototype that works. It gives a sense of what's possible and what's not, and quickly pokes holes in any weak spots in the overall idea.

Sometimes, I think I'm crazy for taking this approach, but it works wonders for me. It helps that people like the folks at 37signals and Fred Brooks have written explanations of the methodology more thoroughly and eloquently than I could possibly write, but I was doubly reassured during my reading lately when I read about how the United States Constitution came about.

In particular, the odd setup of our legislative branch is due to an iterative approach the Founding Fathers took at the Constitutional Convention in 1787. Large states favored a scheme now known as the Virginia Plan, which proposed a two-house legislature where both houses had representatives that were apportioned according to population. The smaller states favored the New Jersey Plan, which proposed a single chamber with one representative from each state.

Now, we look back on the process with our mighty historical hindsight and we look at the Virginia Plan coming from the large states. We might be tempted to think, "Oh, they're just pandering to their own interests. What a crock. That's a terrible scheme to have. How dare they have the audacity to propose something so blatantly self-serving?" The same could be said of the small states.

But what's important is that they tried. They came up with something, put it forth, and asked, "How about this? What do you guys think?" Both the large states and the small states knew that their proposals would have holes poked through them, but they gave it a shot so they could make some progress.

And when this happens, you get something like the Connecticut Compromise, which looked at what everyone thought of the schemes proposed up until then, and said, "Hey, if we go with the Virginia Plan's two houses, we've got some room to work with. What if we make representation in the lower house proportional to population, but give each state equal representation in the upper house, like the New Jersey Plan proposed?"

If even the Founding Fathers had to resort to iteration, I would think it mighty cavalier of me to think I could possibly plan out a complicated system in one fell swoop.

A cheap and nutritious lunch

2007-01-24T22:27:00.000-08:00

I remember when I first tried ugali during my trip to Kenya last May. It didn't taste like it had much nutritional value, but it made me full very quickly.

It's been a slow month at the company, so we've been saving some money on food. I have stopped going out to lunch, and instead I go home to my apartment and cook myself some pasta. I treat myself to parmesan cheese on top, but no meat.

The trouble with this is, it has been tough to get full, even after a lot of pasta.

Today I was impatient and ate two bowls of cereal while my pasta was cooking. I ended up having room for only one bowl of pasta, and I was stuffed. Normally, I'd eat four bowls of pasta and still be a little hungry. I guess I accidentally stumbled upon a solution to staying full.

I'm writing this down for my own reference: for cheap eats, have two bowls of cereal and a bowl of pasta.

Helping the rich world keep its livelihood

2007-01-23T16:31:00.000-08:00

This week's Economist cover story, "Rich man, poor man", about globalisation's winners and losers, offers three categories of solutions to help out those who lose out.

Education in rich countries must equip people with general skills that make them more mobile in the workforce.
Detach health care and pensions from employment, so people who move jobs don't have to worry about much else.
Beef up assistance for those who lose their jobs, in the form of generous training and policies to help them find new work.

These all sound good, but as the editorial acknowledges, these will take years to implement, and it won't be easy. I've also got my own reasons to be skeptical about the effectiveness of these general measures.

People who are part of the educational system in rich countries will see these "general skills" as irrelevant because they want something that's very close to practical. It's difficult to learn something if you're not really interested.
I never really understood the wisdom of tightly coupling health care and pensions to people's jobs, either. Benefits, when tied to jobs, are peripheral concerns, but I suppose it's a temporary hack in the system that just happens to meet the needs of the moment. The thing is, there is no other viable alternative at the moment to this dirty hack.
Greasing up the labour market to make it more mobile is a task I'm very passionate about, but I'm not sure where government or industry can step in effectively. For private concerns, it may not really be profitable. It's an area where I see a lot of room for improvement, but most of the ideas I've got to remedy the problem are more humanitarian than profitable.

It also occurred to me that it may actually be in the rich world's interest to not have job security--or rather, security at the same job. Personally, I have trouble working on the same kind of thing for more than two years, at best, and I am one of the least fickle people I know. It's always best to keep one's options open.

Business growth and preservation of quality

2006-11-24T21:04:00.000-08:00

Today, I realized one of the key challenges facing small business: growing while preserving the quality of service.

For a business selling goods, there's mass production: leave it to the machines. But in the service industries which still require high levels of skill, this is a little trickier. It's something we've still got to figure out.

I thought of this today as I went to see Douglas, a mechanic friend of my dad. He's a good mechanic, and he set up shop for himself in Mountain View a couple of years ago. My family goes to him every time we want to fix our cars — not only because he's my dad's friend, but because he doesn't charge through the roof, because he won't charge for trivial things, and because he does a thorough job. His customers have been known to bring him pizza and other treats out of gratitude.

Every time I go to his shop, all the bays are full, with several cars queued up waiting for each bay. I asked him if he ever thought about putting up an online tool to show his schedule so his customers could know which times were less busy. In response, he told me that he recognizes the growth potential, but wanted to keep his business small so that his customers could get the best possible service.

If he were to grow, he'd have to hire more people, and it would become much harder to consistently ensure the high quality of service he's offering now.

A business can start off with a handful of great people. It can provide great service to its customers while it maintains its small size. But in order to grow, it has to somehow systematize its greatness and turn it into a process. Now I ask myself: how in the world do you systematize going above and beyond? How do you mass produce a mindset geared towards going the extra mile?

It's possible. That's how great companies come about. In their respective fields, they know how to foster greatness: being honest, thorough, and taking pride in their work. Growth is wonderful. But it's not always so easy--in some fields, it may not even be possible: how do you create a huge company defined by great mechanics or great designers? Let's step back and give credit to those who can say no to growth, to those who can stand their ground to provide something great even though it means staying small.

The beautiful side of arguing

2006-11-20T22:58:00.000-08:00

There are many reasons why I prefer not to argue with people. The two biggest reasons for me are that I don't want to waste my time, and I prefer not to rock the boat by creating personal unpleasantness. After thinking about it, though, I've decided that it's good to argue.

As someone who has worked well on my own for a long time, I'm finding that working closely with a team and putting myself in a leadership role requires that I look beyond my personal productivity and look at how to build up the team. By avoiding arguments, I save myself time from explaining myself, but alienate the rest of the team by not involving them in my thought process. In economic terms, they are forced to operate on imperfect information--and the fault of this lies squarely with me because I have an aversion to heated arguments. Because I don't take the time to argue, I also force myself to operate on imperfect information. Yes, the argument can be unpleasant, but when people listen, it all boils down to an exchange of information, and when it is resolved, the team comes away with a more unified sense of direction. Arguing not only serves to inform, but puts personal biases and misconceptions through a trial by fire. If a point cannot be backed up with good reasons, it's not a very good point.

Even keeping this in mind, however, there's the personal unpleasantness associated with heated debate. But really, there's no need to take disagreement personally. If you argue with yourself, with the entity inside your own head, how much more should you argue with those who are outside so you can come to a consensus? Of course people are going to differ in how they think. They'll differ, hash out the reasons, and come to a conclusion. The many sides of my mind will always be fighting with each other, and it's perfectly fine. It's healthy! It's the same with those around you. So speak up, get ready for a fight, and enjoy it.

What's the Web good for?

2006-08-02T00:48:00.000-07:00

The web is good for quick, throw-away information (time-sensitive things like news) and for reference. It's fast. It allows jumping. It can be updated in real-time.

The web is not good for books or anything requiring deep thought. It's too distracting, and the display technology we have today makes it too straining on the eyes to read for too long.

Learning to use new tools

2006-08-01T09:19:00.000-07:00

So these days we've got Ubuntu, Ruby on Rails, and Aptana.

I'm used to running Fedora, writing webapps with PHP, and doing my development in a text editor such as vim or emacs. It seems like there's something new coming out everyday, and it's a choice between checking out these new offerings or getting something done.

I figure that if the tools I'm using are good enough to get things done, why switch? I'd only consider using a new tool if it allows me to do something that I can't do with my current toolset.

Reporting aggregate quantities

2006-06-05T20:42:00.000-07:00

The trouble with aggregate measurements and policies is that they fail to take local variation and local circumstances into account. This is impossible even with the best-intentioned top-down policies. And this is why we've got local government. Aggregates simplify the complicated picture and most of the composite information is lost when presenting the aggregate.

Public institutions

2006-03-22T20:08:00.000-08:00

Today I went to the post office to mail three envelopes to my brother. I found out that it would cost me over nine dollars to mail them separately, and the lady helping me advised me to just get it mailed in one package since they were all going to the same address. The total came to $4.05. What savings! If the post office were privately run, they would have an incentive to maximize profits, but since it's run by the government, I'm looked after as a consumer.

Macro rides on micro

2006-03-14T10:39:00.000-08:00

Macro-level views are useless without micro-level material to work with. Suggestions about how to run a company are useless if there is no company to work with. For all the attention that macro-level views get in the business journals, what I personally need to pay attention to now is producing the micro-level stuff that gives the macro-level theorizing a reason to exist.

Paring down for sanity

2006-03-09T15:08:00.000-08:00

The past couple of days, I've been working on putting together a software release and the manual to accompany it. I found the task pretty overwhelming, especially since I had to edit the document while putting the software together. One thing that I realized towards the end was that I could cut down the apparent scope of the job by ripping out pages that did not need to be edited anymore. I wouldn't have to worry about touching those pages, and could focus on the remaining pages. This helped me feel like I was making progress, and it actually helped me move forward by keeping my focus on the next page that I wanted to finish (and rip out).

Network of enterprises

2006-03-03T15:25:00.000-08:00

This afternoon, I attended a lecture by Dr. Dean Simonton, professor of psychology at UC Davis. He talked about scientific genius and creativity. One of the common work habits of people who made major contributions to knowledge in their fields was to have a network of enterprises--that is, to have multiple projects going on at the same time. Instead of focusing on just one project until it's done, having a network of enterprises means that these projects feed off each other and contribute to each other, essentially making the ultimate outcome more than the sum of the parts.

Psychological benefits of working in pairs

2006-02-23T17:40:00.000-08:00

When assigning people in a company to do knowledge- or research-intensive work, I can see the benefits of working in pairs. When I'm working as part of a pair on a difficult problem, I don't feel like I'm alone, incompetent, or somehow not measuring up whenever I run into unavoidable (but formidable) obstacles. I don't feel so bad because my partner is going through the same thing. When we report on our progress, it's less psychologically taxing because the burden of responsibility is halved. It might be an ideal way to do things in a lot of cases, because it doesn't completely alleviate the workers of responsibility, but distributes the load on each person to make it more bearable.

Extended racial profiling

2006-02-09T17:39:00.000-08:00

As an Asian-American, it might be alarming to read Bush's details on a foiled terror plot in Los Angeles. The article states that Southeast Asians were recruited to carry out the planned attack. I don't know about anyone else, but I'm figuring that Asians will be profiled from now on. I'm okay with that and I understand the concern for security, but I'm guessing that the whiny Asian-American activist types will not be happy to hear this. They'll make a stink about it.

Eventually, the smart thing to do would be just to profile young people everywhere. Once Middle Easterners and Asians are profiled, al-Qaeda may turn to other ethnic groups to carry out their plans.

Delusional comfort and anarchist terrorism

2006-02-07T17:39:00.000-08:00

I was reading Eric Hobsbawm's Industry and Empire this weekend. One interesting argument in the book is that Britain suffered decline because it got complacent about its dominant position. If it was being out-produced or out-innovated, it could retreat back into the comfort of its world empire. Having a power base is useful to buffer against shock or sudden change, but the factors causing the shock or change shouldn't be ignored. Even though it's reassuring to have something to fall back on, this comfort is a temporary solution and something must be done about the root cause of the problem.

Another book I read this weekend was Murdering McKinley: The Making of Theodore Roosevelt's America. Its thesis is that the man who shot McKinley was only one part of the "murder." The McKinley legacy was also wiped out by his successor, Theodore Roosevelt. (The author of the book, Eric Rauchway, is a professor at UC Davis. I took his course on the Gilded Age and Progressive era.) From reading the book, I noted that America and the world have dealt with terrorism before, during the anarchist movement around a hundred years ago. The anarchists managed to cap several European heads of state. I wonder if there are lessons to be learned from that anarchist movement that we haven't bothered exploring.

Theft and economic development

2006-02-02T17:38:00.000-08:00

I was reading Eric Hobsbawm's Industry and Empire last night, and he wrote about how factories in Europe and the United States were often set up by Englishmen, or--more interestingly--through illegally copied designs. It was interesting to me because something that was illegal grew to something legitimate, even to the point that this initial "theft" went on to transform national economies.

It reminded me of how pirating in Taiwan and South Korea used to be a big problem. Despite that short-term loss in revenues to software companies, their tools proliferated and they gained market share. People learned to use that stolen software, developed skill in using it, and even went on to create their own. As a software industry pops up, people learn to appreciate the effort that goes into writing software. With this, a respect for intellectual property is instilled.

This just reminds me how messy the world is. Things end up working out in the end, though. It just seems funny when the lucrative sectors of national economies are founded on theft.

Pulling others to the leading edge

2006-01-20T17:38:00.000-08:00

Those of us working with advanced technology, who push the frontiers of knowledge forward, take a lot of what we know for granted. We take our mathematical ability, our economic sensibility, or our personal responsibility as givens. It's always mind-boggling for me whenever I'm reminded that there are a lot of people worse off than I am.

Sometimes it seems like the best gains that I could possibly make by working in advanced technology to better the world are marginal, at best. The higher yield activity would be to reach people, care for them, educate them, and convince them to take better care of themselves.

If only Americans as a whole could be convinced to exercise consistently and to eat right, the impact on their health would be more substantial than if I were to discover the next wonder drug.

Rather than spending absurd amounts of time developing myself to understand ever more esoteric realms of knowledge, I could instead equip people to make a good living for themselves by working with kids who are behind in school.

We're in a service-oriented economy now. We've moved on from making physical goods. Just like manufacturing moved from cottage industries to mass production, we're stuck in an intellectual cottage industry right now and need to move on to mass production. In a knowledge economy, people are the greatest asset. It makes sense to invest in people.

I hesitate to use the phrase, but “social work” just may leave the world better off, on average, than advanced research projects. I'm not saying we should stop discovering and innovating, but rather that we should stop and look back once in a while to see if we can pull anyone else up with us.

Dividends aren't such a bad thing

2006-01-18T17:34:00.000-08:00

I've come to think again about what I said about government paying out dividends: that it's a bad idea because the money can be invested into big infrastructure projects. It's not so one-sided to me anymore because I've thought of three advantages to paying dividends rather than keeping the money, even if it can be used for big infrastructure projects.

First, dividends modularize the allocation of money and make investing easier to manage. It splits up a large sum into more manageable chunks. It's hard to be careful when sitting on a billion-dollar pile of cash, but if you give it to people in smaller amounts, they'll be more frugal with how they spend the money. Amassing money is good and necessary for big projects (like infrastructure), but if we want to use money efficiently, a finer-grained approach makes more sense. This doesn't mean we abandon big projects completely; we'll always need roads.

Second, dividends promote good will among the recipients. It keeps people happy, so it's easy to see how tempting it is for politicians to pay dividends to their constituents. Who doesn't want extra spending money? It may be wasteful when people go on spending sprees, but in certain situations, good will can be even more valuable than efficiency. When people are hostile and angry, it doesn't matter how efficient a government can be, when that government is on the verge of being overthrown.

Now, paying dividends to consumers may seem wasteful when we look only at efficiency, but when we look at a third possible benefit of paying dividends, the efficiency sacrifice may not seem so bad. Paying dividends puts extra cash in the hands of consumers--just like a tax refund. If there are enough consumers and enough cash being handed out, this payout may stimulate demand. This might be an option to consider when demand is at a low level and economic activity needs jumpstarting.

Of course, these possible benefits do not apply in all cases. We shouldn't pay dividends when our dirt roads are washed out every winter when it rains; we use the money to build roads. We shouldn't pay dividends with the intention of calming the public when they're content already. It doesn't make sense to try to stimulate consumer demand when spending is out of control and everyone is maxing out their credit cards. When we make decisions about what to do, we should, as always, take into account the specific needs of the situation at hand.

Leadership and the extent of communication

2006-01-05T17:33:00.000-08:00

There seems to be some magic in communicating well as a leader. You can do everything right in a leadership position, but some of your tougher moves tend to come off looking bad to those you are leading. This article at Fast Company tells the story of a pilot who communicated well with his disgruntled passengers.

Yesterday, my company had a meeting when the Vice President came to our branch office and told us how the company was doing. There was one disgruntled person who boldly spoke up about how disgruntled people were that we had our company health plans changed in the middle of the year on short notice. The leadership at this company meant well, but people felt exploited and betrayed because they didn't know what was going on.

The lesson here is to involve people at the lower level when decisions are made higher up. Show them the intermediate steps in your thought process, especially when you make difficult moves.

Evangelism and the success of technology

2006-01-04T17:32:00.001-08:00

It seems like the key to success lies in publicizing something. There's a lot of good technology out there, and for it to be considered good, it's got to get everyone casting in their lot with it. That's what separates sites like Wikipedia from some Joe Schmoe who sets up his own Wiki site. That's what sets apart MySpace and Xanga from someone who downloads the LiveJournal software to run on his own server.

So, building a good product seems to be the easy part. These good products themselves are often commoditized, and so the hard part--that which is most crucial to the success of the product--is telling people about it and convincing them to keep on using it.

Bottlenecks

2006-01-04T17:32:00.000-08:00

I'm working on a project with IBM Rational ClearCase, and the builds take forever. It seems somehow wrong that the bottleneck is the computer's performance when I know it should not take that long. It's one thing if I'm the bottleneck as an engineer when I have to think about the problem, but with something like editing, compiling, and linking source code, I should not have to wait long to see if my changes were successful.

In our quest to achieve efficiency, maybe a key indicator is to see where our IT bottlenecks are. In my case, the bottleneck is in the network filesystems. I think the compiler is also licensed only for a few machines, hence the centralized build system. If an unlimited-use compiler were available, the builds could be performed on each engineer's machine, now that workstations have caught up in computing power to yesterday's servers.

Budget surpluses and infrastructure

2005-12-01T17:31:00.000-08:00

Infrastructure is one of the wisest investments a government can make to improve the lot of its people. It's the gift that keeps on giving. Building roads, railways, schools, and hospitals with budget surpluses builds a foundation for future economic growth. Economic growth comes from increases in efficiency, and infrastructure promotes efficiency by saving people time and money.

In Canada, the province of Alberta is undergoing a huge oil boom right now, resulting in massive surpluses. They can't agree on what to do with the money, and they've considered paying dividends, which is a very short-sighted solution. Infrastructure investments will provide jobs immediately, while continually yielding benefits for years to come. A check from a dividend payment can be blown on something from eBay.

One shining example of money put to good use is Beijing, China. The streets are wide enough to carry all the traffic and keep it moving. That's quite a remarkable feat in any Chinese city. When I visited Beijing last summer, smooth traffic was the last thing I was expecting.

Another success is the BART (Bay Area Rapid Transit) system in the San Francisco Bay Area. Its service is frequent, cheap, and reliable. It allows people to move in and out of crowded areas such as Market Street in San Francisco while saving them the hassle of finding parking spaces.

Granted, infrastructure investment is expensive. It's understandable that people will be reluctant to allocate huge sums of money for something whose payoffs are not immediately apparent. Maybe we are doomed to wait until things get really bad before doing something to change the situation. Once we get around to that, let's do something about traffic on Los Angeles freeways.

Offshoring caveats

2005-11-18T17:30:00.000-08:00

Offshoring high-tech workers to developing countries is a hot trend for big business. It saves companies money in labor costs, while giving more bang for the buck. As far as competency goes, foreign tech workers are up to par with Americans. Foreign workers are also willing to work longer hours. These all make offshoring a very attractive alternative for businesses concerned about cutting costs while increasing production.

Before considering offshoring, however, a company should ask itself whether it can deal with the communication hurdles that come as part of the package. These barriers are: having to deal with thick accents, developing a tolerance for poor phone connections, and scheduling around the half-day time difference.

On several occasions, I've had to get on conference call at 9:00pm to India, straining to pick out what they're saying through their thick accents on top of a poor phone connection. Whether the problem is India's phone infrastructure or a weakness in trans-Pacific phone lines, I don't know. All I know is that I had to strain my ears to compensate for the low volume and muffled voices. I spent half my time asking for people to repeat what they said. It almost drove me crazy. (It should also be noted that, from the Indians' point of view, I was the one with the accent.)

For some companies, offshoring will be worth the 90% savings in labor. The company I work for is majority Indian, so most of the company is comfortable with the accent. This company flourishes on the offshoring/outsourcing model. They still have to deal with the time difference and the poor phone connections, though. As a Chinese-American raised in California, I've found working in this environment to be very difficult.

Basically, with offshoring, the added frustration on American workers shouldn't be disregarded. In your company, how important is communication? If you can get by with minimal communication, then offshore and watch your costs shrink. If communication is crucial, please think twice and also consider the sanity of your American employees.

Chinese military buildup

2005-11-17T17:30:00.000-08:00

Americans typically look at China's military buildup and become alarmed. This should not be the case. Looking at the numbers and looking at history will help us understand why it's sensible for China to have a strong military.

Let's take a look at the numbers. With 1.3 billion people and a million-man army, that's less than one-tenth of one percent of the population in the armed forces. China recently stated her intentions for peaceful development. Why do we have trouble believing this? If you want stability and peace, you will need a deterrent against external forces which threaten that stability.

There are also historical reasons behind the Chinese government's desire for a strong military--reasons which Americans may not understand. America has never suffered an invasion on the scale China has suffered time and again. China's long history, punctuated by foreign invasions from the likes of the Mongols, Manchurians, and most recently the Japanese, has taught the Chinese people the lesson that a strong national defense is necessary for the safety of China's people. The Chinese government would be irresponsible if it didn't build up the military.

Chill out, America.

Introduction

2005-11-14T17:29:00.000-08:00

My name is Joshua Go. I graduated Tau Beta Pi from UC Davis in 2005 with a B.S. in Computer Science and Engineering. I also picked up a minor in history. I'm currently paying the bills as a firmware engineer for a small company in Rocklin, California. My biggest claim to fame would probably be writing the bulk of Josh's Linux Guide while I was in middle school. While in high school, I spent a couple of summers working for Penguin Computing and VA Linux Systems (now VA Software).

During my college years, I organized protein and DNA sequences in a campus research lab. I'm still helping with those efforts in my spare time, since the project leader has moved to Nairobi to build up Africa's capacity for research endeavors. These days, though, I am mostly interested in financial markets and global economic affairs. I remain a big dreamer. My current research interests are health care and infrastructure development.