Thursday, September 18, 2008

Getting more out of a sentence

There's a very simple, effective, and systematic way to amp up the amount of insight you get from the things that you read.

It's probably best to explain with an example. The following is from Ed Catmull's article for the Harvard Business Review, "How Pixar Fosters Collective Creativity."

We must constantly challenge all of our assumptions and search for the flaws that could destroy our culture.

How much thought can this spark in your mind?

If we just read this sentence several times, each time with an emphasis on only one word, we get a new angle on the general idea being conveyed. Something different is emphasized.

We must constantly challenge all of our assumptions and search for the flaws that could destroy our culture.

The emphasis is on the we. This tells me that whatever this sentence is talking about involves a team effort. Everyone on the team must be involved. It's a "we" rather than a "me." But does that necessarily have to be the case? It starts with somebody, right? Why not with individuals?

It's okay — a good sign, even — to ask questions about the truth or applicability of a passage. If you're a reader with a healthy sense of skepticism, those kinds of questions immediately arise. A lot of the same questions will come up: is it necessarily so? Is it true all the time? The more you ask these kinds of questions, the more you hash things out and figure them out for yourself.

Let's look at another word.

We must constantly challenge all of our assumptions and search for the flaws that could destroy our culture.

This time, the emphasis is on the word "search." This tells me that the flaws are often hidden; otherwise, they wouldn't require searching. Now, I'm an imaginative guy, so then I imagine myself searching frantically and then getting tired or discouraged. When I'm tired or discouraged, I ask questions. Why am I searching, anyway? What exactly am I searching for, again? How do I really know when I've found it?

That's only two words, and I could go on for a while, but I think the general method has been made clear. The biggest plus going for this method is that it is systematic; I can just emphasize each word until I'm plumb out of new thoughts. When I'm done with all the words in the sentence, I know I'm done.

Personally, I've found that this process also helps me internalize ideas. Even if I don't remember the exact wording of a sentence, the dance of thoughts anchored around a single idea tends to leave a lasting impression. I imagine this is because the thoughts that come to mind immediately tend to be those of the most concern to me. When it comes to things that truly concern me, I have an easy time remembering.

Wednesday, September 17, 2008

Ruby on Rails: optimizing a slow-rendering page

Recently at work, I was faced with the problem of fixing a slow-rendering page. The page is part of an internal content management system, where the content includes video, images, and text. We have over a thousand pieces of content being managed with this tool and the slow page was a listing of all the content along with key overview-type information for each content package.

Everyone was fine with this page loading slowly, since we knew that there was a lot of content. The problem arose when the page would only show around 50 items and then stop loading; the server configuration was set to automatically stop requests that took too long to process.

I knew that the best course of action would be to start measuring. I took a look at the logs to see how long the request was taking and the approximate tasks where time was being spent. At JibJab, we take pride in caching the hell out of everything. Only a tiny fraction of time was spent hitting memcached. And it looked as though memcached was doing its job: almost no time was spent hitting the database. But a ridiculous percentage of the time was spent "rendering" the page.

That meant the onus was on my code to speed things up. Since the slowdown had become more and more apparent as we added more content, my guess was that the loop was the main source of the problem. Anything that would be only a tiny bit slow on its own would lead to a delay over a thousand times greater.

The first thing I removed was a call to cycle(), which I was using to zebra-stripe the table display for easier reading. In its place, I put in an index variable that would be modulo-two to determine its index into an array of two choices (even row or odd row). To keep the same feature, I had to dirty up my code if I wanted it to run more quickly. And indeed that shaved off about one-fourth of the rendering time.

But the page was still taking over ten seconds to render, so I continued my hunt. A pattern I saw many times within each iteration of the loop was an old friend, link_to. I am not terribly attached to this particular friend, however, so I got rid of it and replaced it with some good old <a> tags, which bought me a good second or two.

So far, I felt pretty good about what I had been able to do to extract render-time savings. But there was still room to shave the time down, and luckily for me, Inflector.humanize was being called in each iteration like there was no tomorrow. I decided that there indeed would be no tomorrow for these expensive-looking helpers, and after removing all five calls inside the loop, I got total render time down to a very respectable two-second neighborhood.

The lesson learned? Sometimes code has to be a little messier so it can be fast enough to be used. Specifically, watch out for those beautiful-seeming Rails helper functions; they do a lot of work for you, but sometimes you're just better off doing the work yourself.

Tuesday, September 16, 2008

Bilateral versus multilateral trade agreements

News sources everywhere seem to lament the collapse of the Doha round of trade talks because countries would then have to resort to bilateral or regional trade agreements. For any supporter of free trade this would be understandable, but if we acknowledge the short term havoc wreaked by rapid changes to trade flows that do not give workers and business owners time to adjust, a gradual and stepwise approach to the opening up of markets seems to be a more prudent way to go.

One of the major problems cited by those who favor a broad trade policy is the inconvenience of having to keep track of different rules for different countries. I would go about solving this by building a hash table of trade rules. Basically it would mean that given a country and a product as input, we would get the policies regarding that product as output. This would be a large table, but we would ask and receive only what we are interested in.

There's also the advantage of isolated policies having limited effects, rather than opening up to the entire world in one go. Maybe there is something I'm overlooking, but I don't consider the use of bilateral agreements to be an indicator of failure. I see great potential for reaping the benefits of free trade with less of the shock typically expected by any opening up.

The collapse of grand schemes for international trade deals is really a familiar scenario that just happens to be playing out on a macro level: a project's planners have decided that they have bitten off more than they can chew, and decide to reduce the scope of the project. Everyone does this, and while we may feel disappointed at envisioning the results of the grand project and having to instead settle for something less, we eventually resign ourselves to the reality that we are doing all that we can, and it is a whole lot better than nothing at all.

Thursday, September 4, 2008

Colored bash prompt

The following will set a bash prompt to highlight the name of the directory in yellow, and in bold.

export PS1="[\u@\h \[\033[1;33m\]\w\[\033[0m\]]$ "
This will also work:
export PS1='[\u@\h \[\e[1;33m\]\w\[\e[0m\]]$ '
The escapes have to be in single quotes, however.

During a previous attempt, I wasn't terminating everything correctly, and so when my command exceeded the length of the row, it wouldn't wrap around and start a new line. It was not just a Mac OS X Terminal problem; I tried it in rxvt (installed via MacPorts) as well as the xterm that came standard with Leopard's X11.

Wednesday, September 3, 2008

Doctors and programmers

One evening, I was watching Mystery Diagnosis on the Discovery Health Channel with my brother. The show is about people who have strange ailments, but doctors can't figure them out.

It got me to thinking about how people in my profession approach problem solving. The approach that computer programmers take is up to us, and it's pretty open-ended, but it's widely agreed that we use measuring tools — debuggers and profilers — to examine whether each little step is doing what it's supposed to be doing. It's also a widely accepted quip that one should never try to guess where a slowdown or a bug is happening, because one is usually wrong about it.

But that sort of guessing without detailed measuring seemed to be exactly what the doctors were doing. For one guy, they drew blood sample after blood sample, but it turned out to be a problem on the genetic level.

I wonder if there's some level of professional ego that keeps doctors from doing what would otherwise be considered sensible. If it took a lot of lobbying to get them to use checklists, I wouldn't be surprised if they think they've gone through too much training to actually rely on detailed measurements.

To be fair, though, doctors probably don't have much scope to apply deductive reasoning on the entirety of the systems they're working on. There are no "development" or "testing" patients where it's okay to mess up and start over. Everything they work on is in live production mode.