Oh, The Fail I’ve Known

❉
Adam Keys
http://therealadam.com
 Why Am I Here?
❉

So here we are, at RailsConf 2008. We’re both kinda wondering why I’m here. After all, this
year is all about “advanced” Rails. There’s a myriad of talks on the arcane insides of
ActiveRecord and all sorts of great ways to do things.
 That Guy

❉
And here we are. I’m the guy who made a big splash by slapping his hand on his forehead
and impersonating Star Trek characters. Which is a really advanced move, but not of the
programming variety.
 Why Are You Here?
❉

And I hate to tell you, but _you’re_ the folks in here feeding my ego. That brings me to why,
exactly, you are here.
 Learn From My Mistakes
❉

You’re here so you can learn from all my failings. The fact I’m best known for hijinx implies
that I haven’t really done anything interesting in the realm of building Ruby software. Let me
tell you why that is.
 Fail. I knows It.

❉
I’ve been too busy failing!

Let’s dissect the trifecta of fail I’ve found myself in over the past couple years.
 Technological Fail
❉

Technical mistakes are what we tend to think about the most as developers. What if there’s a
bug here I’m missing? Oh my god, what if this doesn’t scale and Arrington decides to use me
as a hobby horse for his NASCAR’d up website? Will I understand this code when I come back
tomorrow? Is this library even maintained? Even though these problems _seemed_ really
important, they end up easiest to solve. It might take time, but the path to solving them is
one you can outline and plan on.
 Unknown Unknowns

❉
Errors of naivete are also something worth worrying about. To paraphrase the words of the
inventor of the double-wide podium, its the unknown-unknowns that will get you. You can
hack along going down the wrong path only to realize you’d have finished weeks ago if you’d
taken a little shortcut way back at the beginning. Or you might realize someone has already
solved this problem or established a language for describing your domain. These are learning
failures; cases where if only we could absorb infinite knowledge, we’d be set. Even though we
can’t learn as fast as Neo in the Matrix, learning is pretty fun so this sort of fail isn’t too hard
to fix.
 Hell Is Other People
❉

Funny because its true!

The last sort of trouble I ran into was actually pretty hard to fix. Turns out, I don’t get along
with _everyone_, as much as I’d like to. Some folks either rub me horribly wrong or they rub
everyone horribly wrong. There are others who are just tedious to deal with. This is a fact of
life, I think, and everyone has to deal with it. Unfortunately, there isn’t much in the way of
programmer’s documentation on this problem-space, so you have to venture into the creepy
realm of business and self-help books. Having ventured into this space, I’m here to guide
you towards the good stuff so you can avoid the cheese.
 Technological fail

❉
Let’s start on the easy side and look into what we normally think of as reasons for failure in
software projects. These are the most fun to talk about, because we can easily do something
about it. As futile as it is, I think we all enjoy armchair scaling discussions and 20/20
retrospectives.

The important part here is to figure out how to avoid these types of failure in the future. So,
let’s get to know technical failure so we can do it less frequently.
 Picking A Winner
❉

Flamewars. Who doesn’t not love them? In technical circles, they typically come down to folks
disagreeing on what the best technical choice is.

Making technical choices is hard. You’ve got to walk a fine line between learning from past
mistakes, living in the now, and not screwing yourself in the future.

Luckily, most of you have a pretty good bounding box established on how to figure out
what’s technically good and what is technically not so great. If you don’t already have a
heuristic, then it goes like this. If it feels like Ruby, Rails, Merb, etc. then its probably good. If
it feels like whatever you switched to Ruby _from_, it bears a suspicious eye.
 Technology Predictor
Success Matrix
❉

Tim Bray wrote an interesting series of articles on his weblog a while back. He called it the
Technology Predictor Success Matrix. Its a boring name for a useful way to look at a
technology to decide if its going to succeed, and more importantly, with whom it will succeed.
 The Strangler Application

❉
So that’s all and well for making good choices. What do you do if you’ve made a bad choice?
Let me tell you about my friend, the strangler.

Martin Fowler described something he calls a Strangler Application. Its a way to code a new
system while maintaining an old system. You code up your new system to incrementally
replace functionality in the old system. So for a time, you’ve got some requests flowing
through an old system and some through the new system. As time progresses, you should
end up with all your functionality in the new system. At this point you can turn off the old
system and, *voilá*, you’re out of hot water.
 Solve The Right Problem
❉

Its easy to get distracted when you’re programming. You’ll find yourself hacking along and
then you come upon something your framework handles poorly. You can look upon that point
from two weeks in the future and think to yourself, “perhaps writing my own framework was
not the most expedient way to solve this problem”.

Keeping your focus on the ultimate goal is really important. I think you can have a mildly
successful project with some serious diversions. But if you can stay laser focused, you can
have a wildly successful project.

This is the sort of technical failure I fear the most. Like most people, I have been known to
have Architecture Astronaut tendencies. Its hard to resist the temptation to build up
frameworks for something that I am certain will come to happen in the future.
 3 Strikes And You Refactor
❉

Pro-tip: the best frameworks have always come out of real use. The heuristic I apply these
days is three strikes and you refactor common cases into something framework-ish (even if
that framework only has one entry point.)

Inappropriate framework creation is one way you can solve the wrong problem. More visibly,
you could actually solve the wrong problem. Build something no one, especially the customer,
wants. Then you’re totally screwed.
 Feedback Loops
❉

What you need here is a tight feedback loop. Again, its tempting to collect requirements from
the customer, disappear for two months and then *voilá*, you appear with a fully developed
product. Its cool if you can pull it off, but you probably can’t.

You have to keep the people who matter to your software involved. Keep showing it to them,
get their feedback early. Put intentional rough edges around unfinished work so they don’t
get obsessed with stuff you’re not ready to talk about yet.

Feedback loops are a powerful tool. You can use them all over the place, but that’s another
talk. Just make sure you have a feedback loop connected to you and the customer so you’re
heading down the right path most of the time.
 Obsession

❉
Here’s some genuine, hard-won advice. Tread carefully where the mandate of an organization
is based around a technology. No technology is panacea, no matter _what_ you read on some
dude’s blog linked off Reddit.

I’ve found myself in too many conversations where there was an obvious answer to a problem,
but it didn’t use the “blessed” technology. And so away we toiled, flagellating in the
wilderness.
 Tread Carefully
❉

OK, so this is a Rails conference. I’d be remiss if I didn’t tell you about some fail-ish Rails and
how you could’ve spotted it from a mile away.

Fixtures. For a long time, I didn’t really see why people complained about them with such
fervor. But then I really started to use them and I found their true pain. So, how could I have
seen this coming? To bolster my point, I probably couldn’t have. I just had to get my hands
dirty, oh so dirty, to discover when they can be a pain in the ass. Luckily this story has a
happy ending: fixtures are considerably less sucky in Rails 2 and there are lots of great
alternatives if you feel they are still a little too suckish.

SOAP. Its full of fail. Ergo, ActionWebService, as hard as it tries, is something that’s going to
pale in comparison to its peers in terms of programmer joy. You see, suck is transitive. Its
very, very hard to write an API on top of something that sucks and end up with something
fun. ActiveRecord is probably the best counterpoint. But most APIs aren’t as nice as
ActiveRecord.
 On Learning

❉
What I’ve really done here is lured you all into a session about learning. Ya see, there’s lots to
learn in this world. That’s why mankind created the story, then the picture, then writing, etc.
This culminated in, of course, the webinar. Who _hasn’t_ learned a lot from a webinar?

By my estimation, you can learn some things just by pulling up your big-boy-or-girl shorts
and getting down to it. And this will get you pretty far. But at some point, you have to fall on
your face.
 Two Kinds Of Learning
❉

Allow me to make a binary distinction. There are two kinds of learning in this world: things
you can learn from others and things you just have to learn for yourself. In our brief time
together, I’d like to focus on the latter. In the meantime, we’ll discard the fact that my binary
value system doesn’t allow for the possibility you’ll learn anything from me in this time.

So first I’m going to sort of lay down the theory of learning from others. It will sort of come
off as self-helpish, as you can apply it to software or life in general. I’ll keep the big-tooth
grinning part short and then I’ll go into “learning hacks” that are specific to software
development.

Put on your thinking caps!

 You Only Learn By Falling
Down
❉

I’ve been embellishing this quote for years now. I got it from a pretty unlikely candidate. The
fellow who told me this played with me on the university rollerhockey team. He was way
better than me and he was a member of a fraternity known more for their acumen at drinking
than their studies. Nonetheless, I’ve learned about life from these simple six words.

But he’s right. You can get to the point where you’re OK at hockey without falling down much.
You won’t become a Gretzky, or even a Crosby/Yzerman/insert your favorite hockey player
here unless you take a few spills. Hockey’s got the right idea here.

You can’t play hockey without strapping ten pounds of padding onto your body. You are set
up for failure, but you’re also set up to bounce right back up and get back in the fray. That
last bit is pretty important – you can push yourself just a little bit, dive for that loose puck in
front of the net. Sometimes you’ll score, but mostly you’ll fall on your face and then get
roughed up by a defenseman.
 Don’t Fear the reaper

❉
Adam, you’re thinking. “How do you apply this to software?” Well first off, don’t fear
mistakes. No one gets anything right the first time. Iterate on things you’re not sure about.
When you come upon unknown territory, set yourself up for rapidly trying different
approaches until you get it right.

Setting up “padding” is the important part. Like I said, you play hockey with tons of
protection. Make it easy to move patches into your production systems (Capistrano), reduce
the friction in trying something crazy in your code (Git), arrange your system so it tells you
when it hurts (unit tests, exception notifiers, etc.) With these sorts of padding, you’re much
less likely to hurt yourself trying something awesome.
 I Need To Make Bigger
Mistakes
❉

Allow me another sporting metaphor. A while back, I was playing golf. I hadn’t played in quite
some time and my game was _way_ off. My confidence was destroyed. Golf is like
programming in that there’s a positive feedback loop based on your current confidence level.
In my state of self-doubt, I was playing shorter-distance clubs that are easier to hit. And still,
I was doing rather poorly. Further, in playing short clubs, I was playing poorly _and_ slowly.

So while I was holding up the group behind me, I thought to myself. “I need to make bigger
mistakes.” I knew I was going to be in trouble no matter what clubs I hit. But if I’m going to
only hit one out of every five shots well, I might as well hit that one shot with a club where the
result will prove more impressive.

If you’re currently down on your golf game, I highly recommend making bigger mistakes.
Works for programming too!
 Getting Into The Groove
❉

I find that the internet has destroyed my attention span. Its disturbingly frequent that I find
myself rapidly context switching between surfing, attending to IRC/IM/Twitter and trying to
code. My theory is that this is a failure to get my feedback loop started. Once you get into a
coding mood, its easy to keep going; getting there is the hard part.

One way I’ve found to jump-start my coding day is to pull off a few really easy bugs and fix
them. It feels great to get a few checkins under your belt real quick and as a bonus, you get
to cross out a few items on the todo list.

Another starter is to just set yourself a goal and see how fast you can accomplish it. Make it
something achievable, not a boil-the-ocean sort of thing. But still make it challenging. The
right kind of time pressure can do wonders for your productivity!

The last trick I’ll touch on is picking some game to play as you code along. At one point I was
on a run where I was trying to code as expressively as possible and not use comments. In an
old C# job, I’d try and write code that was as idiomatically Ruby-esque as possible. Having an
ulterior motive makes it much easier to get through some of the more tedious programming
exercises.
 Learning
❉

I started off saying that you learn the really, really important bits by doing. I’m going to
contradict myself and tell you about all sorts of interesting things you need to go out and
learn.

Besides writing code, learning is what we do as software developers. We pick up new

languages, APIs and customer domains all the time. We figure out what is stable or buggy,
what is performant or slow, what is bullshit and what is great. Yet we are never trained in
_how_ to learn about these things. I’m going to try and change that.

The following ideas aren’t so much knowledge unto themselves. Rather, they’re sort of a
language describing the sorts of knowledge that is really helpful to inform the directions you
take when you are learning by doing (and failing).
 Layer Up, Layer Down
❉

There’s a metric shit-ton of things that we, as developers, need to know. Relational theory,
metaprogramming, cascading style sheets, document object models, object-oriented
programming and the list goes on and on. Its tempting to focus on just one or two of these.

But you can’t really do that. I’ve realized all kinds of WIN by making a point of deeply knowing
my problem domain (web applications) but also having deep knowledge of the layers above it
(HTML, CSS, JS, visual design, etc.) _and_ the layers below it (virtual machines, relational
databases, web servers, etc.)

Its unlikely I will ever need to write a web server, for example. But knowing enough about that
domain that I could talk to a web server implementor has proven quite useful. Knowing the
size and shape of the web server box prevents me from pursuing false paths. That lets me go
off and fail in much more novel and interesting ways.
 Conversational Knowledge
❉

When it comes to learning about something tangential to my ostensible goal, my rule of

thumb is this: know enough to have a useful conversation with someone who lives in that
world. For example, I’m happy with my knowledge of kernels and compilers and I could
probably hold my own in a deep conversation with someone who works on Linux or GCC. On
the other hand, my knowledge of database internals still needs love, so I would defer to those
who work on MySQL day in and day out.

Knowing these sorts of things sets you up for an important sort of success: knowing the
bounds of a problem space. If you know what the compiler or browser is doing to your code,
you can make better decisions about how to use them. Better decisions lead to less fail!
 The grass is greener on the
other side

❉
So they say, the grass always looks greener on the other side. Some new language, library,
practice or methodology seems to solve all the grief you’re suffering from now. Its must be all
good over there!

Well, you’ll never know until you go try. As _The Pragmatic Programmer_ suggested, go out
and learn new ways of thinking as much as possible. You can’t lose. Either you end up with a
new tool in your belt or your knowledge of the world grows and you’re better prepared to not
fail.

On the other hand, you might find that the grass is not so green other there. They say you
can’t prove a negative, but knowing how to find negatives is a really great skill to have. It
keeps you from going down false paths. ‘Nuff said.
 Get Outside Your Bubble
❉

A great way to get outside your technological comfort zone is to go to conferences outside
your little technology bubble. It will probably earn you some grief amongst your friends, but
its very educational to drop in on your local Java, .NET, Python or PHP group. I’ve learned a lot
this way.

For a while, I found that I could convince myself every three or four months that Java must
not be _that_ bad. So I’d take a shallow little dive into it and try to build something small. In
hindsight, trying to build something _small_ was my mistake. Regardless, the result was
always that I ran back to my joyous world of not programming in Java.

The take-away is to always look for greener pastures, and double-check yourself that
pastures you once thought were brown aren’t actually green.
 Developing Intuition
❉

Earlier, I said that we as software developers build up an intuition about what is good and
what is bad. Its a sort of mental defense mechanism that protects us from specious practices
or overblown solutions.

But how do we build up this intuition? How can we recognize bullshit faster? Personally, I feel
like my bozo filter is very well attuned. I think I got here in two ways.

First off, reading comments on weblogs, Digg, Reddit, etc. can prove useful. People often
complain about the low quality of discourse on these sites. Figuring out how to filter the
wheat from the chaff in comment threads is a good way to level set your bozo filter. The same
bozo filter works whether you’re considering the validity of a person’s claims or a tutorial for
a new language.
 Passive Mentor
❉

More importantly, I think I more or less inherited a lot of my intuition from those wiser than
myself. I’m lucky to have chosen a few good individuals to “passively mentor” myself to. By
listening to and observing these people, I’ve vastly expanded my knowledge. More
importantly, I’ve adopted some of their sense of what is useful versus what is useless.
 People Hacks
❉

Working with other people is a fact of life. The days of the cowboy coder, toiling alone in his
garage, are over. This is largely a good thing. One need only look at the success of GitHub to
see how much more fun coding is when you've got other people watching and helping.

The flip-side of this is that to avoid lots of annoying sorts of fail, you need people skills. Not
everyone is as cool as your friends. So people downright suck. Nonetheless, you'll have to
work with them to some extent. Let's figure that bit out.
 $ man that_guy
❉

That guy. We all know him. As People Hacks go, its your worst-case situation. Some people,
you will just not find a way to get along with. Now, I'd really like to give you some awesome
hack about how to fix the situations. But, I tried for a couple years, and I failed.

Along the ways, I did learn a few things. First off, check your ego at the door. Your customer
or users probably don’t care which one of you is right.

I’ve found that those who annoy me the most are those who are quite similar to me in some
way. At this point I have two tasks: figure out how to change the annoying quality in myself
and how to deal with it in the other person.

Finding a buddy is really helpful in level setting your own expectations as to what a good
resolution to your people problems are. It doesn’t necessarily solve the problem, but it can
help your morale if you find that everyone has troubles dealing with your problem person.

If all else fails, the best I can tell you is, figure out how to amicably end the work relationship
with them. It will probably tax your patience temporarily. Just keep reminding yourself its
almost over. Marking days off a calendar helps.
 Discard Your Ego
❉

On the other hand, we can all prove difficult to work with at times. I know I've got my
buttons, and if you push them, I get unpleasantly crabby. But I still love you baby. Its not you,
its _me_.

I’ve found that most of the time when I’m a jerk, it’s because of my ego. I think its pretty
universal. To compensate for this, I try to practice kindness at all times. I know, this sounds
horribly cheesy. How will I ever become an überhacker like Bram Cohen or Linus Torvalds if I
don’t lash out at someone every once in a while?

I say, the world will prove no worse if we can put our egos behind us and just treat each other
nice and fairly. Doing so only makes it easier for us to build big, awesome things. Isn’t that
the goal?

If you suspect you might qualify as a royal jerk (like I did), then I highly recommend the
completely trite _How To Win Friends And Influence People_. If you'd told me two years ago
that I would read, let alone recommend this book, I'd have laughed at you. But, its really good
at figuring out how to turn that smile upside down, per se, and get stuff done despite
yourself or other people.
 In Praise Of Fail
❉

I’d like to wrap up by praising failure. It gets a bad rap. But, consider how lame a world
without fail would look. We wouldn’t have this presentation; seriously, he’s the guy who did a
Star Trek skit. We wouldn’t have _LOST_; too expensive and weird. We wouldn’t have Rails;
Ruby is too slow.

The important thing I need to impress upon you is to learn from your failures and those of
others. That way, you can go fail in even more extravagant and interesting ways. After you’ve
done that a few dozen times, you’re ready for *spectacular win*.

In the interest of not ending on a internet meme (its bad luck), I’ll leave you with a Go
proverb. “Hurry up and lose your first fifty games.”
 EOF
❉
http://therealadam.com