List Processing

Recently I was assigned/volunteered for this task at work. Fairly inglorious, basically it ends up being writing a whole bunch of XML configuration. Nothing more exciting than pounding out endless tags. My editor of choice is emacs, mostly because it runs really fast, and I don't like the 'escape' command/insert metaphor of VI and clones. I should repeat that last bit: I like emacs because it is FAST. That's right, the editor who's acronym expansion used to be "Eight Megabytes And Constantly Swapping" is now one of the fastest editors. If you don't think so, try loading up a 80k line XML file in your favorite IDE.

So emacs XML mode does allow for some validation and other things, but normally it does not insert closing tags and other 'niceties' that editors such as TextMate does. So I found myself repeatedly hammering out the same XML blocks, and I thought, "I know emacs has a template system, I should use that." And indeed it does, in fact emacs as 2 separate but complementary systems. The first is 'skeletons' which provide a method of defining a mini-template. It has a pile of features including the ability to automatically put your point in an 'interesting' spot of the inserted text. So it can position your cursor at the middle of 2 tags or in an attribute or wherever. You can then bind these skeletons to key commands.

Binding to key commands is handy, but combined with the abbrevation mode you end up with a powerhouse editing shortcut. Simply put, abbreviations are small letter sequences that get expanded. Simple text substitution is trivial, turn teh to the. Or, you can call an emacs-lisp function, such as a skeleton function you just previously defined. Meaning with a few keystrokes and no special control-bindings you can pound out code when the base format (XML is so verbose!) does not allow shortcut definitions.

Of course each little skeleton (or what one might call "macros" but we don't in lisp because there are already lisp macros which are way different) is pretty custom. But they are so trivial and easy to write, even for small hour or two projects it makes sense to define them. Same with abbreviations. This has easily saved me two hours already, combined with the 45 minutes it took to learn, is about 1 hour and 15 minutes saved.

I should also mention, I am in no ways a lisp expert. Copy and pasting emacs lisp functions into ~/.emacs and doing a little bit of modifying is the extent of my expertise. It took me 10 minutes to figure out how to use the interactive special form properly. I usually use Google to find solutions to my emacs-lisp problems. But defining these skeleton/abbrevs was so stupidly trivial practically everyone should learn it.

Here is a brief example, lets say you are typing the same piece of HTML over and over, it looks like this:

<div class="boxy">
  <div class="custom">
    text
  </div>
</div>

Lets say the div classes are always the same, but the text inside is what is unique. So you define a skeleton:

(define-skeleton
  boxy-custom-div
                "My Custom/Boxy div nested skeleton"
  > "<div class=\"boxy\">" \n
  > "<div class=\"custom\">" \n
  > _ \n
  -2 "</div>" \n
  -2 "</div>" \n )

Basically anything in a quoted string is echoed verbatim. Escape double quotes with a \ as usual. The proceeding > means 'intend according to the mode'. The \n inserts a newline as expected. The -2 is to properly un-indent those lines by 2 spaces. The bare _ means that is an area of interest. What it means is when the skeleton is applied, the first 'area of interest' is where the cursor will be left afterwards. You can define multiple interesting areas and do more advanced things by having skeletons wrap around existing text. However that does not play well with the abbrev trick I will say next.

Small note: I used emacs to type all those damn > and < entities.

The magic comes in with abbrev mode. First off enable abbrev mode with M-x abbrev-mode in your buffer. Then you need to define an abbreviation, but you can't do it with the command sequence C-x a l which the manual tells you. Instead you have to execute the following lisp code:

(define-abbrev
   html-mode-abbrev-table
   "cdiv" "" 'boxy-custom-div)

The first string "cdiv" is the trigger text. The second string is what to substitute it with (nothing) and the last argument is what function to call after the text was inserted. The net result is with 5 keys "cdiv" then space/enter you get to expand out all that HTML. Clearly the more repetitive text you have the bigger of a win this is. Unfortunately skeletons that are triggered via abbrev-mode do not properly work to wrap existing text. That mode of skeleton is very powerful, it lets you wrap multiple (up to 16) chunks of text in a row at arbitrary boundaries in to different points of a skeleton in order. The slight annoyance is you have to mark with C-space each point of interest. I personally find that awkward.

Of course a neater approach would be to ASK in the mini buffer what you'd like to insert in. And you can do this in skeletons. But there is only so far you can go with that, since you have to ask the fields in order and you can't post-insert text. Lets say you are defining database schemas, you might as well only enter the table name and have your skeleton guess the primary key name. This is starting to get a little complex, and this is where my story changes slightly.

I ran in to this particular problem, wanting to do non-trivial calculation and manipulation in a skeleton. The docs say "maybe you should write an emacs-lisp function instead", which is a fabulous idea. I don't really know enough about lisp or emacs lisp so it seems like a good time to learn. Luckily I work with the semi-famous Steve Yegge. As you may know or have guess, Steve is a lisp... well fan. And I raised this to him, that skeletons and abbrev-mode ruled and I needed to learn more emacs-lisp. To which he said "read the book on my desk". Technically it's a print-out in a binder of a book, and the book is "On Lisp" which I am now reading.

The interesting thing about Lisp is it was invented in the early days of computer science (ie: 70s and early 80s) when people didn't "know" how to program. Thus I believe people asked questions like "what does it mean to program" and other seemingly useless philosophical type questions. The result of which is Lisp, a meta-programming language that facilitates writing and rewriting itself. Of course now, being smart programmers we invent useful languages like C++ and Java and C# which are safe and don't let you change too much about the language itself. And we are better off for it, right?

Kathy Sierra

Once upon a time I met Kathy Sierra at a weird mini-conference at my former employer. I really enjoyed her talk, and I have followed her blog since then. I am amazed at 2 traits of her blog (ones I wish I had), first is great graphic design. Stock photos and tablet drawings of pie charts never looked so good! Secondly, every post is part of a excellent narrative and is argued well. Representing no doubt years of research.

Of course by now you have probably heard of the whole death threat bit. I really didn't know what to think when I saw it, but I certainly did not want to write a blog post using the world 'but'.

I don't think anyone can really know what Kathy is feeling. Having met her, and knowing she is on the straight and level, I am inclined to believe her. I do think that contributions indicating that people should "toughen up" really don't help much. Especially when coming from a position of privileged and protection.

Those are easy to dismiss - however what is harder to dismiss is the use of the word 'but'. I really don't like this word, and to illustrate why, I will do an example:

"I support your cause, but I am concerned that ..."

Now lets replace the but with it's true meaning:
"I support your cause, but ignore everything before but, since what follows are my true feelings, I am concerned that...

This is my own cognitive search-and-replace, your mileage may vary, results aren't guaranteed, etc, etc.

Greenspan

For no specific reason I was reminded to dig up the h4x0r economist "comics". Any comic where Greenspan speaks leet is pretty much gold in my books.

Two Cool videos

I want the following UI interaction devices on my desk and in my bag. The first is a new desktop UI metaphor, based on physics. The second is a multi-touch display, sort of like Minority Report and Star Trek.

YouTube Link

Google Video Link

Lunch Hosting In Kirkland

Once nice thing about working here, is it is very easy to convince people to come visit you for lunch. The famous Google lunch, free, and filled with smart people who love to chat. Who could want more? Today I had my ex-coworker Alexy come by. He had some photography tidbits, a cool Leica point and shoot, and plenty of "financial engineering" to talk about. Cool guy, he is currently ramping up to a PhD - good luck there.

Also he owns women.ru of all sites. Crazy guy.

Joost

Thanks to the sweet hook up, I have a Joost account now.

Review forthcoming.

Design

On the subject of design, many people have many things to say about it. Of course, with such a highly debated topic, how can one think they are possibly smart or qualified enough to say more on this subject?

The hubris of developers pretty much demand that we should talk about crap we don't know enough about. Of course there has been much writing about how a "child like approach" can bring a fresh look and new innovative ideas. To that end, I think I have noticed some things about great design that I think is truly new (to me). I'm not sure I can condense my list in to an actual list, so you get prose instead.

Great design generally comes out of inspired creativity. You rarely can force such creativity, which is why those ad companies are paid the big bucks. You rarely get this kind of creativity out of groups, and never out of committees. If you have a small design team that operates in a high bandwidth communication situation (ie: in person) and they "think like one" you may end up with amazing things. Design to completion though is fraught with compromises. Some compromises are ok, others are critical junctures where your design just got shot to hell. I don't know how to differentiate between these two types of compromises.

Every design has essential elements - any changes that detract from these elements end up taking away from the overall picture. Since the essential elements, or first principles, can be unknown to everyone, great designers also end up being great dictators. Since you can't effectively argue about first principles you don't know or can't communicate, you are left with force of personality (see: Steve Jobs).

There are however first principles of great design. Things all great designs have. All designs come from simple ideas that can end up having complex interactions. A great example would be Hypercard. Years later people still rely on hypercard stacks for various things. The iPod is a classic design study, so all I'll say about that is the central idea is "I should be able to hold all my music in one hand". The simple idea must not be a mission statement, it must reflect a functional piece of the design. A counter example would be "the user should be able to manipulate all manners of information" - this is more of a use case or end goal, rather than a design principle.

Since design principles reflect the working or physical aesthetic of the product (see: chair design), they should ideally reflect a subtle or even sublime balance between functionality and simplicity. Simple first principles that allow rich functionality is hard to achieve however. However, simplicity sometimes just ends up in overly simple system, where the basic rules fail to interact in interesting or meaningful ways. One possible test is the orthogonal test - each first principle should be completely free from the others. By combining multiple simple rules you end up with a rich multidimensional space in which aspects can be combined.

The basic bench-test for designs is often the gut or intuition test. This may return conflicting or shallow results depending on the strength of the intuition involved however. This article has suggestively been about general design, but my overall concern is primarily with software design. The goodness of a software design seems to be escaping many people, unlike, say, Architectural design. There is an increasing focus on this subject over the last 10 years, with many new players such as 43 folders. However, this is leading edge stuff, not being taught in school or being spread widely enough in the engineering space.

Perhaps one of the biggest obstacles is the intuitive portion of design. Although intuition is just short for combining many disrelated factors in a semi-unconscious manner, the left-brained focus of many software engineers can leaves them opposed to this approach. Perhaps some software engineers can benefit from right-brained activities? Drawing, musical instruments and other artistic endeavors can trigger right brained thinking. I am blessed (cursed) with an extremely right brained thinking style, which oddly enough gives me good grades in math, but bad grades in organization (you should see my apartment!).