Beruflich Dokumente
Kultur Dokumente
Contributors
Curator's Note
Brian Shul
I
Carlos Bueno
Jamie Zawinski
would like to give huge thanks to the contributors,
Eric Davis
who so generously provided Hacker Monthly the per-
Carter Cleveland
missions to reprint their articles, the advertisers, who
Bradford Cross
believes in us despite it's just the first issue, Paul Graham,
Hamilton Ulmer
who thankfully did not oppose this idea and gave me the
Adam Kempa
'go-ahead', and most of all, the members of Hacker News,
Gary Haran
who provided both support and valuable feedback to mate-
Walt Kania
rialize the idea.
Evan Miller
Creating Hacker Monthly has been both interesting and
Tawheed Kader
educational. Prior to this, I do not have any experience work-
Paul Graham
ing with magazine or print. Throughout this one month,
Jason Cohen
I've learned everything I could about print magazine and
Steve Blank
spent countless hours working my way through Adobe
Dave Rodenbaugh
InDesign. I've also exchanged near hundreds of emails
William A. Wood
asking for reprint permissions and looking for prospective
advertisers. The only downside though, is I don't have much
time left to code, which I missed quite a bit.
Printer I still remember the day sitting at Starbucks and imagin-
MagCloud
ing what the magazine version of Hacker News would be
like. I can finally stop imagining now. — Lim Cheng Soon
Advertising
ads@hackermonthly.com
Rate Card
hackermonthly.com/ratecard
HACKER MONTHLY is the print magazine version of Hacker
News (news.ycombinator.com) — a social news website
Contact wildly popular among hackers and startup founders with
curator@hackermonthly.com
its content can be "anything that gratifies one's intellectual
curiosity"(as quoted from the site's submission guidelines).
Every month, we select the top voted articles from Hacker
Published by News and put them in the magazine format. For more, visit
Netizens Media
www.hackermonthly.com.
46, Taylor Road,
11600 Penang,
Malaysia.
Contents
FEATURES
PROGRAMMING CAREER
14 2 Steps to Becoming a Great Developer 20 What Value do We Create Here?
By ERIC DAVIS By CARTER CLEVELAND
16 iPhone Developer: "This is why I sell 22 7 Tips for Successful Self-Learning
beer" By BRAFORD CROSS and HAMILTON ULMER
By JAMIE JAWINSKI
SPECIAL
18 Top Three Motivators for Developers
By DAVE RODENBAUGH 25 Adam?...is there a reason your laptop is
in the fridge?
STARTUP By ADAM KEMPA
32 How I Took My Web-App to Market in 3 26 The Scariest Pricing Idea Ever
Days By WALT KANIA
By TAWHEED KADER
29 5 Actions that Made Me Happier
34 Organic Startup Ideas By GARY HARAN
By PAUL GRAHAM
30 How Not to Run an A/B Test
36 Not Disruptive, and Proud of It By EVAN MILLER
By JASON COHEN
39 Best Writing Advice for Engineers
38 Turning on Your Reality Distortion Field By WILLIAM A. WOOD
By STEVE BLANK
Photo credit: Bisecting the Moonrise by Steve Jurvetson (www.flickr.com/photos/jurvetson/2693120417/) in front cover, Dark Clouds With a Touch Sky by Dean Souglass (www.flickr.com/
photos/deansouglass/494131526/) in back cover, SR-71 @ Air And Space Museum by Sh4rp_i (www.flickr.com/photos/85638163@N00/426687730/).
Licensed under Creative Commons Attribution 2.0 Generic licence (creativecommons.org/licenses/by/2.0/deed.en).
3
FEATURES
6 FEATURES
Nicaragua , Iran , Libya , and the Falk- is gone. We’ve become so used to the steadily increasing in hundredths, in
land Islands . On a weekly basis, the constant buzzing that the jet sounds a rhythmic consistency similar to the
SR-71 kept watch over every Soviet quiet now in comparison. The Mach long distance runner who has caught
nuclear submarine and mobile mis- correspondingly increases slightly and his second wind and picked up the
sile site, and all of their troop move- the jet is flying in that confidently pace. The jet was made for this kind
ments. It was a key factor in winning smooth and steady style we have so of performance and she wasn’t about
the Cold War. often seen at these speeds. We reach to let an errant inlet door make her
I am proud to say I flew about 500 our target altitude and speed, with five miss the show. With the power of forty
hours in this aircraft. I knew her well. miles to spare. Entering the target area, locomotives, we puncture the quiet
She gave way to no plane, proudly drag- in response to the jet’s newfound vital- African sky and continue farther south
ging her sonic boom through enemy ity, Walt says, ‘That’s amazing’ and with across a bleak landscape.
backyards with great impunity. She my left hand pushing two throttles Walt continues to update me with
defeated every missile, outran every farther forward, I think to myself that numerous reactions he sees on the
MiG, and always brought us home. In there is much they don’t teach in engi- DEF panel. He is receiving missile-
the first 100 years of manned flight, no neering school. tracking signals. With each mile we
aircraft was more remarkable. Out my left window, Libya looks traverse, every two seconds, I become
With the Libyan coast fast approach- like one huge sandbox. A featureless more uncomfortable driving deeper
ing now, Walt asks me for the third brown terrain stretches all the way to into this barren and hostile land. I am
time, if I think the jet will get to the the horizon. There is no sign of any glad the DEF panel is not in the front
speed and altitude we want in time. I activity. Then Walt tells me that he is seat. It would be a big distraction now,
tell him yes. I know he is concerned. getting lots of electronic signals, and seeing the lights flashing. In contrast,
He is dealing with the data; that’s they are not the friendly kind. The jet is my cockpit is ‘quiet’ as the jet purrs
what engineers do, and I am glad he is. performing perfectly now, flying better and relishes her newfound strength,
But I have my hands on the stick and than she has in weeks. She seems to continuing to slowly accelerate.
throttles and can feel the heart of a know where she is. She likes the high The spikes are full aft now, tucked
thoroughbred, running now with the Mach, as we penetrate deeper into twenty-six inches deep into the
power and perfection she was designed Libyan airspace. Leaving the footprint nacelles. With all inlet doors tightly
to possess. I also talk to her. Like the of our sonic boom across Benghazi, I shut, at 3.24 Mach, the J-58s are more
combat veteran she is, the jet senses the sit motionless, with stilled hands on like ramjets now, gulping 100,000
target area and seems to prepare herself. throttles and the pitch control, my eyes cubic feet of air per second. We are
For the first time in two days, the glued to the gauges. a roaring express now, and as we roll
inlet door closes flush and all vibration Only the Mach indicator is moving, through the enemy’s backyard, I hope »
Lockheed SR-71 (Blackbird) front view by Keith (www.flickr.com/photos/pheanixphotos/4109160435/), SR-71 2 by Andrew Fogg (www.flickr.com/photos/ndrwfgg/55930821/). Licensed under
Creative Commons Attribution 2.0 Generic licence (creativecommons.org/licenses/by/2.0/deed.en).
7
» our speed continues to defeat the mis- will defeat any missiles. We are not It is a race this jet will not let us
sile radars below. We are approaching there yet, though, and I’m wondering lose. The Mach eases to 3.5 as we crest
a turn, and this is good. It will only if Walt will call for a defensive turn 80,000 feet. We are a bullet now -
make it more difficult for any launched off our course. except faster. We hit the turn, and I feel
missile to solve the solution for hitting With no words spoken, I sense Walter some relief as our nose swings away
our aircraft. is thinking in concert with me about from a country we have seen quite
I push the speed up at Walt’s request. maintaining our programmed course. enough of. Screaming past Tripoli, our
The jet does not skip a beat, nothing To keep from worrying, I glance outside, phenomenal speed continues to rise,
fluctuates, and the cameras have a rock wondering if I’ll be able to visually pick and the screaming Sled pummels the
steady platform. Walt received mis- up a missile aimed at us. Odd are the enemy one more time, laying down
sile launch signals. Before he can say thoughts that wander through one’s a parting sonic boom. In seconds, we
anything else, my left hand instinc- mind in times like these. I found myself can see nothing but the expansive blue
tively moves the throttles yet farther recalling the words of former SR-71 of the Mediterranean. I realize that I
forward. My eyes are glued to tem- pilots who were fired upon while flying still have my left hand full forward
perature gauges now, as I know the missions over North Vietnam They and we’re continuing to rocket along
jet will willingly go to speeds that can said the few errant missile detonations in maximum afterburner.
harm her. The temps are relatively cool they were able to observe from the The TDI now shows us Mach num-
and from all the warm temps we’ve cockpit looked like implosions rather bers, not only new to our experience
encountered thus far, this surprises me than explosions. This was due to the but flat out scary. Walt says the DEF
but then, it really doesn’t surprise me. great speed at which the jet was hurl- panel is now quiet, and I know it is
Mach 3.31 and Walt are quiet for the ing away from the exploding missile. time to reduce our incredible speed.
moment. I see nothing outside except the end- I pull the throttles to the min ‘burner
I move my gloved finger across the less expanse of a steel blue sky and range and the jet still doesn’t want to
small silver wheel on the autopilot the broad patch of tan earth far below. slow down. Normally the Mach would
panel, which controls the aircraft’s I have only had my eyes out of the be affected immediately, when making
pitch. With the deft feel known to cockpit for seconds, but it seems like such a large throttle movement. But for
Swiss watchmakers, surgeons, and many minutes since I have last checked just a few moments old 960 just sat out
‘dinosaurs’ (old- time pilots who not the gauges inside. Returning my atten- there at the high Mach, she seemed to
only fly an airplane but ‘feel it’), I tion inward, I glance first at the miles love and like the proud Sled she was,
rotate the pitch wheel somewhere counter telling me how many more to only began to slow when we were well
between one-sixteenth and one-eighth go, until we can start our turn Then I out of danger. I loved that jet. n
inch location, a position which yields note the Mach, and passing beyond
the 500-foot-per-minute climb I desire. 3.45, I realize that Walter and I have Brian Shul was an Air Force fighter pilot
The jet raises her nose one-sixth of a attained new personal records. The for 20 years. Shot down in Vietnam, he
degree and knows, I’ll push her higher Mach continues to increase. The ride spent one year in hospitals and was told
as she goes faster. The Mach contin- is incredibly smooth. he’d never fly again. He flew for another
ues to rise, but during this segment There seems to be a confirmed trust 15 years, including the world’s fastest jet,
of our route, I am in no mood to pull now, between me and the jet; she the SR-71. As an avid photographer Brian
throttles back. will not hesitate to deliver whatever accumulated the world’s rarest collection
Walt’s voice pierces the quiet of my speed we need, and I can count on no of SR-71 photographs and used them to
cockpit with the news of more missile problems with the inlets. Walt and I create the two most popular books ever
launch signals. The gravity of Walter’s are ultimately depending on the jet done on that aircraft, Sled Driver, and The
voice tells me that he believes the now - more so than normal - and she Untouchables. Brian today is an avid nature
signals to be a more valid threat than seems to know it. The cooler outside photographer and in high demand nation-
the others. Within seconds he tells me temperatures have awakened the spirit wide as a motivational speaker.
to ‘push it up’ and I firmly press both born into her years ago, when men
throttles against their stops. For the dedicated to excellence took the time
next few seconds, I will let the jet go as and care to build her well. With spikes
fast as she wants. A final turn is coming and doors as tight as they can get, we
up and we both know that if we can hit are racing against the time it could take
that turn at this speed, we most likely a missile to reach our altitude.
Reprinted with permission of the original author. First appeared on the book 'Sled Diver'. For more information, visit www.sleddriver.com.
8 FEATURES
Reach to the Hackers,
Founders and the people
who are changing the web.
10 FEATURES
programming is at splitting large jobs be on our way. Concurrency would and business can’t proceed. But when
into parallel tracks. be another thing we no longer worry enough members are in communica-
How hard could it be? I thought. Very about unless we want to, like memory tion the decision-making process is fast
hard, as it happened. I got up to 4 par- management. Unfortunately there is and unambiguous.
allel processes before my program col- evidence that it won’t be this clean Consistent/Partitionable means that
lapsed under its own weight. The crux and simple. 2 A lot of things we take the system can recover from failures,
of the problem was that the file was for granted may have to change. but requires so much
stored on a hard drive. If you’ve never There are at least two concurrency extra coordination
peeked inside a hard drive, it’s like a problems to solve: how to get many that it collapses under
record player with a metal disc and components inside one computer to heavy use. Imagine
a magnetic head instead of a needle. cooperate without stepping all over having to send and
Just like a record it works best when each other, and how to get many com- receive a status report
you “play” it in sequence, and not so puters to cooperate without drown- for every decision
well if you keep moving the needle ing in coordination overhead. These made at your company. You’ll always
around. And of course it can only play may be special cases of a more general be current, and when you come back
one thing at a time. So I couldn’t just problem and one solution will work for from vacation you will never miss
split the file into 32 chunks and have all. Or perhaps we’ll have one kind of a thing, but making actual progress
each thread read a chunk simultane- programming for the large and another would be very slow.
ously. One thread had to read from for the small, just as the mechanics of Available/Partitionable
the file and then dole out parts of it life are different inside and outside of means that you can always
to the others. It was like trying to get the cell. read and write values, but
31 housepainters to share the same At the far end of the spectrum are the values you read might
bucket. large distributed databases, be out of date.
When I looked at other people’s such as those used by search A classic example is gossip:
entries for hints I was struck by how engines, online retailers, and at any point you might not
almost all of them, good and bad, social networks. These things know the latest on what
looked complicated and steampunky. are enormous networks of Judy said to Bill but even-
Part of that was my unfamiliarity with computers that work together tually word gets around.
the techniques, but another part was to handle thousands of writes When you have new gossip
the lack of good support for parallel- and hundreds of thousands of to share you only have to
ism, which forced people to roll their reads every second. More machines in tell one or two people and trust that in
own abstractions. (Ask four program- the system raise the odds that one of time it will reach everyone who cares.
mers to create a new abstraction and them will fail at any moment. There Spreading gossip among computers is
you’ll get five and a half answers.) The is also the chance that a link between a bit more reliable because they are
pithiest entry was 130 lines of OCaml, groups of machines will fail, cutting the endlessly patient and (usually) don’t
a language with good support for “par- brain in half until it is repaired. There garble messages.4
allel I/O” but which is not widely used is a tricky balance between being able After lots of groping around with
outside of academia. Most of the others to read from such a system consistently billions of dollars of revenue at stake,
were several hundred lines long. Many and quickly and writing to it reliably. people who build these large systems
people like me were not able to com- The situation is summed up by the are coming to the conclusion that it’s
plete the challenge at all. If it’s this CAP Theorem, which states that large most important to always be able to
difficult to parallelize a trivial string- systems have three desirable but con- write to a system quickly and read
counting program, what makes us think flicting properties: Consistency, Avail- from it even in the face of temporary
we’re doing it right in complex ones? ability, and Partition-tolerance. You can failures. Stale data is a consequence
Ideally, concurrency shouldn’t leak only optimize for two at the expense of looser coupling and greater auton-
into the logic of programs we’re trying of the third. omy needed to make that possible. It’s
to write. Some really smart people A Consistent/Available system uncomfortable to accept the idea that
would figure out the right way to do means that reading and writing always as the computing power of an Avail-
it. They would write papers with lots works the way you expect, but requires able/Partitionable system scales up, the
of equations in them and fly around a majority or quorum of nodes to be fog of war descends on consistency, but
to conferences for a few years until running in order to function. Think in practice it’s not the end of the world.
some other smart people figured out of a parliment that must have This was not a whimsical nor
what the hell they were saying. Those more than half of members easy choice. Imagine Ebenezer
people would go develop libraries in present in order to hold a vote. Scrooge is making so much
our favorite programming languages. If too many can’t make it, say money that Bob Cratchit can’t
Then we could just put import concur- because a flood washes out the keep up. Scrooge needs more
rent; at the top of our programs and bridge, a quorum can’t be formed than one employee to receive »
11
» and count it. To find out the grand total import concurrent; always have the last laugh. A concur-
of his money at any point, he has to Shared memory can be pushed fairly rent system is fundamentally limited by
ask each of them for a subtotal. By far, however. Instead of explicit locks, how often processes have to coordinate
the time Scrooge gets all the answers Clojure and many newer languages use and the time it takes them to do so. As
and adds them up, his employees have an interesting technique called software of this writing computer memory can
counted more money, and his total is transactional memory. STM simulates a be accessed in about 100 nanoseconds.
already out of date. So he tells them to sort of post-hoc, fine-grained, implicit Local network’s latency is measured in
stop counting while he gathers subto- locking. Under this scheme semi-inde- microseconds to milliseconds. Schemes
tals. But this wastes valuable working pendent workers, called threads, read that work well at local memory speeds
time. And what if Scrooge adds another and write to a shared memory space don’t fly over a channel one thousand
counting-house down the street? He’ll as though they were alone. The system times slower. Throughput is a problem
have to pay a street boy, little Sammy keeps a log of what they have read and too: memory can have one hundred
Locke, to a) run to the other house and written. When a thread is finished the times the throughput of network, and
tell them to stop counting, b) gather system verifies that no data it read was is shared among at most a few dozen
their subtotals, c) deliver them to changed by any other. If so the changes threads. A large distributed database
Scrooge, then d) run back to the other are committed. If there is a conflict can have tens of thousands of inde-
house to tell them to resume count- the transaction is aborted, changes pendent threads contending for the
ing. What’s worse, his customers can’t are rolled back and the thread’s job is same bandwidth.
pay him while this is happening. As his retried. While threads operate on non- If we can’t carry the shared-mem-
operation gets bigger Scrooge is faced overlapping parts of memory, or even ory model outside of the computer, is
with a growing tradeoff between stale non-overlapping parts of the same data there some other model we can bring
information and halting everything structures, they can do whatever they inside? Are threads, ie semi-indepen-
“Into ask
essence, transactional memory allows threads
for forgiveness instead of permission.”
to wait on Locke. If there’s anything want without the overhead of lock- dent workers that play inside a shared
Scrooge likes less than old numbers, it’s ing. In essence, transactional memory memory space, absolutely necessary? In
paying people to do nothing. allows threads to ask for forgiveness his “standard lecture” on threads Xavier
Scrooge’s dilemma is forced upon instead of permission. Leroy details three reasons people use
him by basic physics. You can’t avoid As you might have guessed from them:
it by using electrons instead of street those jolly hints about conflict and
• Shared-memory parallelism using
urchins. It’s impossible for an event rollback, STM has its own special
locks or transactions. This is explic-
happening in one place (eg data chang- problems, like how to perform those
itly disowned in both Erlang and
ing inside one computer or process) to commit/abort/retry cycles efficiently
Leroy’s OCaml in favor of message-
affect any other place (eg other com- on thousands of threads. It’s fun to
passing. His argument is that it’s too
puters or processes) until the informa- imagine pathological conflict scenarios
complex, especially in garbage-col-
tion has had time to travel between in which long chains of transactions
lected languages, and doesn’t scale.
them. Where those delays are small unravel like a cheap sweater.5 STM
relative to performance requirements, is also not able to handle actions that • Overlapping I/O and computation, ie
Scrooge can get away with various aren’t undoable. You can’t retry most while thread A is waiting for data to
forms of locking and enjoy the illusion kinds of I/O for the same reason you be sent or received, threads B-Z can
of a shared, consistent memory space. can’t rewind a live concert. This is han- continue their work. Overlapping
But as programs spread out over more dled by queueing up any non-reversible (aka non-blocking I/O) is needed
and more independent workers, the actions, performing them outside of to solve problems like WideFinder
complexity needed to maintain that the transaction, caching the result in a efficiently. This is often thwarted by
illusion begins to overwhelm every- buffer, and replaying as necessary. Read low-level facilities inside the operat-
thing else.3 that sentence again. ing system that were written without
Undeniably awesome and clever as regard to parallelism. Leroy thinks
STM threads are, I’m not convinced this should be fixed at the OS level
that shared memory makes sense out- instead of making every program
side of the “cell membrane” of a single solve it again and again.
computer. Throughput and latency
12 FEATURES
• Coroutines, which allow different computing is designing good proto- sense of “old news”, such as when you
functions to call each other repeat- cols. I know of no language that allows find out your cousin had gotten mar-
edly without generating an infinitely protocols as first-class entities that can ried last year. Her wedding and your
long stack of references back to the be passed around and manipulated like unawareness are both “true” relative to
first call. This looks suspiciously like functions and objects are. I’m not even your reference frames until you receive
message-passing. sure what that would look like but it news to the contrary.
might be interesting to try out. 4. The categories are not rigidly exclu-
Message-passing, which first
There is a lot of sound and fury sive. The parliment problem is miti-
appeared in Smalltalk, is the core
around parallelism and concurrency. gated in real parliments with quorum
abstraction of Joe Armstrong’s pro-
I don’t know what the answer will be. rules: say if a majority of members
gramming language Erlang. Erlang pro-
I personally suspect that a relaxed, are in one place, or some minimum
grams do things that make program-
shared-memory model will work well number is present in chambers, they
mers take notice, like run some of the
enough within the confines of one can act as though they were the full
busiest telephone switches for years
computer, in the way that Newtonian body. The status report problem is
without fail 6. It approaches concur-
physics works well enough at certain usually handled by having heirar-
rency with three iron rules: no shared
scales. A more austere model will be chies of supervisors and employees
memory even between processes on
needed for a small network of comput- aka “reports”. The gossip consistency
the same computer, a standard format
ers, and so on as you grow. Or perhaps problem can be helped by tagging data
for messages passed between pro-
there’s something out there that will with timestamps or version numbers
cesses, and a guarantee that messages
make all this lockwork moot. n so you can reconcile conflicting values.
are read in the order in which they
5. There is a recent paper about an
were received. The first rule is meant
Notes interesting variation on this theme
to avoid the heartaches described
0. Parallelism is the act of taking a called HyTM, which appears to do a
above and embraces local knowledge
large job, splitting it up into smaller copy-on-write instead of performing
over global state. The second and third
ones, and doing them at once. People writes to shared memory.
keep programmers from endlessly
often use “parallel” and “concurrent” 6. A lot of writeups repeat a “nine
reinventing schemes for passing mes-
interchangably, but there is a subtle nines”, ie 99.9999999% reliability
sages between processes. Every Erlang
difference. Concurrency is necessary claim for Erlang-based Ericsson tele-
process has sovereign control over its
for parallelism but not the other way phone switches owned by British Tele-
own memory space and can only affect
around. If I alternate between cooking coms. This works out to 31 milliseconds
others by sending well-formed mes-
eggs and pancakes I’m doing both con- of downtime per year, which hovers
sages. It’s an elegant model and hap-
currently. If I’m cooking eggs while you near the edge of measurability, not to
pens to be a cleaned-up version of the
are cooking pancakes, we are cooking say plausibility. I was present at a talk
way the Internet itself is constructed.
concurrently and in parallel. Techni- Armstrong gave in early 2010 during
Message-passing is already one of the
cally if I’m cooking eggs and you are which he was asked about this. There
axioms of concurrent distributed com-
mowing the lawn we are also working was a little foot shuffling as he qualified
putation, and may well be universal.
in parallel, but since no coordination it: it was actually 6 or so seconds of
There are probably more axioms
is needed in that case there’s nothing downtime in one device during a code
to discover. Languages become more
to talk about. update. Since BT had X devices over
powerful as abstractions are made
1. “The slovenliness of our language Y years, they calculated it as 31ms of
explicit and standardized. Message-
makes it easier for us to have foolish average downtime per device per year.
passing says nothing about optimizing
thoughts. The point is that the process Or something like that. Either way it’s
for locality, ie making sure that pro-
is reversible.” -- George Orwell, Politics an impressive feat.
cesses talk with other processes that are
and the English Language
located nearby instead of at random. It
“That language is an instrument Carlos Bueno is an engineer at Facebook. He
might be cool to have a standard way to
of human reason, and not merely a writes occasionally about general program-
measure the locality of a function call.
medium for the expression of thought, ming topics, performance, security, and
Languages become even more power-
is a truth generally admitted.” - George internationalization. His long-term project
ful when abstractions are made first-
Boole, The Laws of Thought is to “save the web”: to build a network of
class entities. For example, languages
2. Neither was the switch to memory independent, redundant, Internet archives.
that can pass functions as arguments
management, come to think of it.
to other functions can generate new
3. This is not about speed-of-light
types of higher-order functions with-
effects or anything like that. I’m only
out the programmer having to code
talking about reference frames in the
them by hand. A big part of distributed
13
PROGRAMMING
iPhone Developer: “T
By JAMIE ZAWINSKI
14 PROGRAMMING
This is why I sell beer”
in GL and throw away all this bullshit Quartz code.” (Let’s unicycle is a slight variant of a city bus. If you can handle
keep in mind here the insanely complicated thing I’m doing one, the other should be pretty much the same, right?
in this program: I have a bitmap. I want to put it on the Again, what the hell — I can almost understand want-
screen, fast, using two whole colors. And the colors change ing to get rid of display lists for efficiency reasons in an
some times. This should be fucking trivial, right? Oh, ho embedded API (I don’t like it, because my screen savers
ho ho.) tend to use display lists a lot, but I can sort-of understand
So I rewrote it in OpenGL, just dumping my bitmap it), but given that you could totally implement glBegin() and
into a luminance texture, and this is where some of you glVertex() in terms of glDrawArrays() why the hell did they take
are laughing at me already, because I didn’t know that the them out! Gaah!
iPhone actually runs OpenGLES! Which has, of course, even Anyway, where was I?
less to do with OpenGL than iPhones have to do with Macs. Oh, yeah. So Dali Clock works on the iPhone and iPad
I expected the usual crazy ifdef-dance around creating the now, I think. I can’t actually run it on my phone, because
OpenGL context and requesting color buffers and whatnot, I haven’t gotten over my righteous indignation at the idea
since OpenGL never specified any of that crap that I’m supposed to tithe $100 to Captain Steve before
in a cross-platform way to begin with, I’m allowed to test out the program I wrote on the phone
but what I didn’t expect — and that I bought. I imagine I could manage it if I jailbroke my
I’m still kind of slack-jawed at phone first, but the last time I did that it destabilized it a
this — is that OpenGLES lot and I had to re-install.
removed glBegin() and So if one of you who has supplicated at the App Store
glVertex(). troth would like to build it from source and let me know
No, really, it really if it runs on your actual device, that’d be cool.
did. Oh, PS, I just noticed that since I rewrote it in OpenGL,
That’s like, the it’s now too slow to get a decent frame rate when running
defining character- full screen on an 860MHz PPC G4. I mean, that machine
istic of OpenGL. is only 53× faster than a 16MHz Palm Pilot, and only 107×
So OpenGLES is faster than an 8MHz Mac128k.
just a slight variant This is why I sell beer. n
of OpenGL, in the
way that Jamie Zawinski was one of the founders of Netscape and
Mozilla.org, was the primary developer of Lucid Emacs,
and wrote most of your screen savers. Today he is
the proprietor of DNA Lounge, an all ages dance
club and live music venue in San Francisco.
15
2 Steps to Becoming a Great
Developer
By ERIC DAVIS
16 PROGRAMMING
Getting Started If you are working solo:
Now here’s the call to action, because you will never become
• download some popular projects and read through a
a great developer without taking action.
single class every week
Write At Least One Line Of Code In A New Code
➊ Base Every Day For A Week. Switch Code Bases
After Each Week.
• get some API documentation that shows the method’s
source code inline and read the source each time you
look up a method, or
This can be a new feature, a bugfix, a refactoring, or just
• find a mentor and work with them on some real code
monkeying around with an idea. It doesn’t matter, the act
of thinking through the code and writing is what you are So whatever you do, take action today. Unless you’re
after. Don’t know one a good code base to start on? Do a afraid of becoming a great developer...But there is plenty
refactoring on Redmine and tell me about it in the com- of room at the top. n
ments below.
Notes
Find A Way To Learn From A Great Developer
➋ Every Week.
If you are working with a great developer:
1. Notice that the smart developers are always producing
new code.... they are following step #1.
17
Top Three Motivators for
Developers
By DAVE RODENBAUGH
18 PROGRAMMING
“Turns out, we’re kidding ourselves if we think that
[money] is our real motive as developers.”
companies now regularly practice Purpose
some, if not all of those things we did Nothing is more tedious, horrific, or
back in the late 90s? (Albeit with uninspiring to developers to work on
more restraint, thankfully) projects that lack any real meaning in
the world. Or lack any real direction.
With that in mind, let’s take Daniel’s
Or lack any substantial need from the
idea of the results-oriented work envi-
company. In fact, you can probably
ronment (ROWE) forward and create
point to the brightest points of your
something new for the 21st century.
career all stemming from those projects
It focuses on three important ideas,
that had the deepest meaning to you
which developers already love and That’s
personally. Maybe the darkest points
embrace: Autonomy, Mastery, Purpose. my challenge
are those soul-sucking projects that you
for you in 2010.
waded through because you were glad
Autonomy “Make software suck
to have a job but desperately waited for
What developer out there doesn’t like less in the 21st century”. Good luck. n
things to improve so you could find a
to be given the freedom to do their
better job elsewhere. Preferably where
own thing, on their terms, with their Notes
soul-vacuums didn’t exist.
preferred hours, using their tools, 1. http://www.computerworld.com/s/
Google gets it: They already advo-
environment, IDE, language, operat- article/347538/The_Big_Squeeze
cate the 20% time concept and (near-)
ing system and favorite t-shirt? Find 2. Anything that isn’t a cognitive task,
complete workplace freedom. Atlas-
me a single developer anywhere that simple or complex, according to the
sian gets it: They have the Fedex chal-
doesn’t crave this kind of freedom and research I quote below.
lenge where everyone in the company
I’ll pay you $10. Seriously. Drop me 3. Sorry, outsourcers…dangling the
gets 24 hours to work on something
a contact above. I’m good for it. Of reward under your workers noses
they are interested in, with the caveat
course, you’ll search for the rest of your doesn’t help even when your home
you have to deliver it at the end of 24
life and won’t be able to do it. country is considerably poorer on
hours and you must present it to the
average than Western economies. Yet
company. Think those don’t create
Mastery another surprising finding of their
passion for the company? How about
Every developer on the planet wants to research.
the Nine Things Developers Want
get better at what they do. We crave 4. http://www.youtube.com/watch?v=
More than Money? These points all
new knowledge like some people quaff rrkrvAUbU9Y
touch on the same three basic con-
coffee after a hangover. Fortunately,
cepts: autonomy, mastery, and purpose.
the side effects of getting better at Dave Rodenbaugh is an independent soft-
Does your company “get it”? If the
development are far more benign than ware contractor with nearly 2 decades of
answer is NO, what can you do right
caffeine binging. enterprise project experience in a variety
now to change your workplace to “get
of companies and industries. Although
it”? And if that is too Sisyphean a task
he loves Java, he sometimes drinks a good
for you, how about starting your own
black tea when the mood strikes. He’s still
company instead, that does “get it”?
waiting for his first business trip to the
Caribbean.
19
CAREER
What Value
O I thought I had
ne summer
the ultimate dream job.
During the day I created
software that accessed
some of the world’s largest financial databases
and provided traders with real-time data and
analysis for trade ideas. At night I worked with
the CTO on a side project that analyzed huge
amounts of transaction data to identify arbitrage
opportunities. We figured that if we could start find-
ing enough of these opportunities, we could present
them as trade ideas to the bosses. So we wrote scripts,
and at night, after everyone else left the office, we installed
them on their computers and ran the scripts in parallel to
try and crunch through the massive amount of data we
had access to. This was fun. Really fun. And even better,
the CTO was an awesome guy who taught me a lot about
programming.
They also paid well. Really well. Even more than my friends
received working 100 hour weeks at I-Banking jobs. In retro-
spect, no college student should ever have been paid that much
(on the bright side, the savings were enough for Art.sy’s initial
funding).
But that summer it meant I could go out to nice dinners with my
girlfriend, and never worry about paying for drinks at expensive clubs.
It meant I could afford fancy clothes, an iPhone, and plane flights to
Asia. Having always worked in labs prior to that job, it redefined how I
thought about money.
20 CAREER
e do We Create Here? By CARTER CLEVELAND
21
7 Tips for
Successful
Self-Learning
By BRADFORD CROSS and HAMILTON ULMER
22 CAREER
“
No matter what, you’re going to have to learn
most everything on your own anyway.”
one element of graduate school that is in a classroom or library to study, but
You don’t read textbooks, you
➍ work through them
Imagine taking a 12-hour flight with
hardest to replicate is the advisor-advi- notice the relative isolation and focus two books, Machiavelli’s “The Prince”
see relationship. They help guide you, those environments afford over read- and Shilov’s “Elementary Functional
smoothing out the uncertainties you ing a book with your laptop on while Analysis.” It would be typical to finish
have about certain topics, and help you writing emails and checking facebook the 100 pages of Machiavelli in two
make your own learning more efficient. or twitter with the TV on. hours or so, and spent the rest of the
As a self-learner, you do not have the Remove the distractions and allocate time working through 10 pages of a
convenience of scheduled class time large blocks of time. You might find Shilov’s “Elementary Functional Analy-
and required problem sets. You must that for more difficult material, you sis,” minus some breaks for napping
be aggressive about finding people to need larger blocks of time to study and eating undesirable airplane food.
help you. because it takes longer to shift into the Reading a technical book is nothing
context of harder problems. like reading a novel. You have to slow
➌ Avoid multitasking
Another reason school is great
for learning is that you
down and work carefully if you want
to understand the material. Have you
ever found yourself 10 pages further
plan your day around in a book and having forgotten what
your classes. There are you’ve just read?
distractions, of course, Successful self-learners don’t read,
but if you’re concerned they toil. If there are proofs, walk
with learning at school, them through, and try proving results
you prioritize on your own. Work through exer-
your classes cises, and make up your own
over other examples. Draw various dia-
things. grams and invent visualisa-
You don’t tions to help you develop an
have to be intuition. If there is a real-
world application for the
work, try it out. If there are
algorithms, implement them
with your favorite program-
ming language. If something
remains unclear, hunt down
someone who’s smarter than
you and get them to explain.
Sometimes you just need to
put the material down, step
away, relax, and think deeply
to develop an intuition. »
Figure 1. The "I'm stuck"
decision tree.
1
23
“
In theory, there is no difference between
theory and practice. But, in practice, there is.”
- Jan L. A. van de Snepscheut
➎ Build Eigencourses
Great self-learners spend a lot
of time to find the best resources for
➐ There is nothing so practical
as a good theory. -Kurt Lewin
Sometimes you are several hops away
Notes
1. Keep in mind that you often just
need to build a general foundation in
learning. You can find all the textbooks, from something you can code up and the field, or mastery of some subset of
papers and other resources you need apply to a problem directly. Not all a field - you don’t have to master the
on the Internet. Many of the course textbooks can be read with applica- entire field.
materials from among the world’s tion in mind, despite that they serve as 2. http://www.jimmyr.com/blog/1_
best universities are available for free the theoretical foundation for applied Top_10_Universities_With_Free_
online2. Check out the great lists of work. This is why you must have a Courses_Online.php
links to video courses on this Data deep sense of patience and commit- 3. http://www.datawrangling.com/
Wrangling post3. ment - which is why a prolonged hidden-video-courses-in-math-science-
You can pick and choose the best curiosity and passion for a topic are and-engineering
“eigencourse” with lecture slides, video so valuable. 4. Figure 1.
lectures, textbooks, and other materials. Understanding analysis (particu-
The best way to find these materials is larly sets, measures, and spaces) will Bradford has been doing applied research
on Google. You will often only need to serve as your foundation for a deep since 2001. His interests are in Maths, Sta-
pay for the book, and sometimes even understanding of probability theory, tistics, Computer Science, Learning Theory,
the book is free at the course website and both will then serve as your foun- Network Theory, Information Retrieval,
in pdf form. dation for understating inference, and Natural Language Processing, and engi-
Take the time to triangulate on the a deep understanding of inference is neering at scale. Most recently, Bradford is
right material. Find the greats in the a mainstay of achieving high quality co-founder and head of research for Flight-
field, see what they use and recom- results on applied problems. Caster, where is responsible for the statistical
mend. Find other students and read Avoid the dualistic mistakes of learning and supporting architecture that
the reviews on Amazon. Google is technical execution without intu- power Flightcaster’s predictive algorithms.
your friend. ition, and intuition without technical
execution.n Hamilton Ulmer is a Master’s student in
What to do when you don’t
➏ understand
Learning is all about abstractions. We
Statistics at Stanford. He has a great deal
of experience as a data engineer, having
helped startups of various sizes and shapes
build up abstractions on top of other get on their feet with processing and visu-
abstractions. If you do not know the alizing their data, as well as helping them
abstractions you are reading about that make data-driven decisions. In August he
are being composed into new higher- will join the Mozilla analytics team.
level abstractions, then you aren’t going
to understand the new abstraction. If
you get stuck, the way to get un-stuck
is to follow the I’m stuck decision tree4.
24 CAREER
Adam? …is there a reason
your laptop is in the fridge? By ADAM KEMPA
25
SPECIAL
H
all time.
ere’s a pricing technique
that sounds, at first, like the
dumbest newbie move of
26 SPECIAL
“Let me just concentrate on
getting this done for you, and
we’ll settle up later. I trust you
to be fair.”
As an added bonus, you will most
likely do the best work of your life, and Example.
deliver obscenely wonderful service to A client of mine was knee-deep in
your clients at the same time. (Mainly redoing all her company’s web site con-
because you’ll be too scared not to.) tent. She was getting raw material from
the various divisions that was ugly,
Making it pay. More. undecipherable and unusable. The
Naturally, the sole reason for using fill- go-live date was looming. She called
in-the-blank invoicing is to net more me in to figure out how to fix it all.
from a project than you could with But she had no idea how many sec-
“traditional” pricing. tions we’d be doing, how many pages,
The idea is to get paid for the value nor how bad the raw material would
the client derives from the work, rather be, so it was impossible to estimate
than for the number of hours it took. any sort of fee.
Or how hard it was. Or how many I said, “Let me just concentrate on
“I have no idea what to bill for this,”
shots you had to take. Or what some- getting this done for you, and we’ll
I’d say. “Just send me whatever seems
body else charged some other client settle up later. I trust you to be fair.”
right to you.” Sometimes they would
somewhere. She agreed.
send a hundred or two more than I
And by value, I don’t mean only hard I did the work as it came in over a
anticipated, sometimes less. But it was
economic value, like sales or savings or couple of weeks, revising, re-writing,
always intriguing to see how the client
new business. (Which in most cases is re-building the content. We came up
perceived what I had done. And a little
hard to quantify anyway.) with a neat and tight format, a solid
humbling, too, on occasion.
As I’ve discovered, clients are also voice, sharp messaging. Everybody
But over the past year or so I finally
willing to pay lavishly to get a nose- loved it.
got the guts to try this on large proj-
bleed project done and off the desk, I then told the client to let me know
ects for big clients. (Partly because,
to look like geniuses in front of their what she felt was a reasonable fee for
while developing “Talking Money,” I
bosses, to have presentations that their the project. It was entirely her call.
was thinking/obsessing about pricing
sales people rave about. To finally get Meanwhile, I went back and parsed
issues pretty much all day long. I was
the bosses sold on videos for user train- out the work based purely on hours
itching to see how this worked.)
ing. To untangle a project that some- spent. Had I been pricing conven-
I can tell you this: the ‘pay what
body else screwed up. tionally, it would have come to 3800
you want’ idea can be surprisingly and
That kind of value has no relation to 4200 bucks, depending on how I
dumbfoundingly profitable.
to how long it took you to do the job. counted.
Better still, I can guarantee you that
It’s irrelevant, immaterial. And it is Next day, I get an email from the
it will shake up your thinking about
difficult to guess what that value might client. She says, “I’m thinking $9,500.
fees and pricing. It will un-stick some
be from our side of the glass. So it can How does that sound?”
old notions. And heaven knows we
pay to let the client set that value. I wrote her back and said “Fine.
need that; most of us are way too
Sold.”
myopic, constipated and chickenshit
Now, lest you think I’m just handing
about fees.
you rosy stories, here’s another. »
27
“Don’t try this with one-time clients...Been there,
done that, lost shirt.”
» A designer friend is working on a
web site for a financial firm, two part-
ners. He refers them to me for the
• The client has a big personal stake
in the project. They have skin in
the game. They stand to look grand
J ust so you know I’m not the only
crackpot using this idea, Matt
Homann of LexThink, a consultant
writing. We have a few phone conver- if all goes well, score some points, who works with law firms, offers this
sations. Seems simple enough. Not a be a hero, win some kudos. This ‘you decide’ option to all of his clients.
ton of content, straightforward mission. does not work for low-level back- His experience with the technique mir-
The clients don’t know much about burner projects that no one cares rors mine exactly. There’s more about
marketing or web stuff. about. (Like my Wall Street clients; his approach here too, in The Non-
I say, “Tell you what. I’ll write every- to them, their website was just some Billable Hour. (It’s for lawyers, but the
thing for you, and when you’re happy bullshit thing they needed to have. ideas apply to us, I think.)
with it, send me a check for what you They didn’t perceive it as critical.) Oh, and see the classic Little Rascals
think is reasonable.” episode from 1936, “Pay as You Exit.”
• The project looks hard, impossible,
Ordinarily, I would have quoted As the story goes, the gang was put-
and indecipherable. (My Wall Street
about $2500 for the project, although ting on a show in the barn, but the
clients thought it was a cinch to bang
I don’t say that. neighborhood kids were reluctant to
out a few pages of drivel, and there-
I do some drafts. There are some pay the penny admission, fearing that
fore paid accordingly. My technol-
comments, some revisions. Slam-dunk. the show might be lame.
ogy client tried untangling her web
Site goes live. Time to settle up. And Over Spanky’s objections, Alfalfa
content herself, and got scared. To
I’m thinking the Wall Street guys are decided to let everyone in for free, and
her, it seemed insurmountable.)
seeing a fee with a lot of zeros. allow them to pay on the way out if
They send a check for $1200. And How do clients react? Do clients they liked the show.
say, ‘Thanks for the great work.” like this idea? As it turned out, the gang botched
Ouch and a half. A few will balk. They don’t want the show horribly, but the result was
the responsibility of figuring out a fee. so hilarious that the kids filed out
What works, what doesn’t They don’t want the anguish. That’s laughing.
After a few painful scorchings, and sev- okay. Give them a quote. Leaving Alfalfa with cigar box full
eral delightfully lucrative wins, here is Most will be astonished that you of pennies. n
the bottom line. offer the option. It shows you trust
This technique works only when: them. That you value their judgment. Walt Kania is a freelance writer who runs
That you even thought to ask. Huge The Freelancery site (thefreelancery.com),
• You have a long-term relationship
karma points translate to more dollars. and develops marketing content (waltkania.
with the client. You’ve done work
Sometimes (as one client confessed com) for B2B and technology companies. He
for them before, at your usual rates.
to me) they’ll reflexively crank up the has plied his trade independently his entire
They trust you. They know your
fee when filling in the blank. adult life, due to a congenital inability to
work. And mostly likely they need
Sort of like the way we reflexively tolerate conventional employment for more
to work with you again.
and fearfully crank down the price than three to five days.
• Don’t try this with one-time clients, when the client says ‘How much will
clients who don’t use this work it cost?”
often, or clients who didn’t seek you
out. Been there, done that, lost shirt.
28 SPECIAL
5 Actions that Made Me
Happier
By GARY HARAN
➌ Played Sports
A Harvard University study started in 1937 that
spanned 72 years determined that healthy play could relieve
Gary is a programmer and entrepreneur in Montreal, Canada. He is
a father and entrepreneur currently working at SocialGrapes.com.
daily frustrations making us happier overall. You can follow him on twitter @xutopia.
A few years ago I joined a volleyball team and now I play
a minimum of once a week.
29
How Not to Run an A/B Test By EVAN MILLER
I f you run A/B tests on your website and regularly check Example
ongoing experiments for significant results, you might Suppose you analyze an experiment after 200 and 500
be falling prey to what statisticians call repeated signifi- observations. There are four things that could happen:
cance testing errors. As a result, even
though your dashboard says a result Scenario 1 Scenario 2 Scenario 3 Scenario 4
is statistically significant, there’s a After 200 observations Insignificant Insignificant Significant! Significant!
good chance that it’s actually insig- After 500 observations Insignificant Significant! trial stopped trial stopped
nificant. This note explains why. End of experiment Insignificant Significant! Significant! Significant!
Background Assuming treatments A and B are the same and the
When an A/B testing dashboard says there is a “95% chance
significance level is 5%, then at the end of the experiment,
of beating original” or “90% probability of statistical signifi-
we’ll have a significant result 5% of the time.
cance,” it’s asking the following question: Assuming there is
But suppose we stop the experiment as soon as there is
no underlying difference between A and B, how often will
a significant result. Now look at the four things that could
we see a difference like we do in the data just by chance?
happen:
The answer to that question is called the significance level,
and “statistically significant results” Scenario 1 Scenario 2 Scenario 3 Scenario 4
mean that the significance level is
After 200 observations Insignificant Insignificant Significant! Significant!
low, e.g. 5% or 1%. Dashboards usu-
ally take the complement of this After 500 observations Insignificant Significant! Insignificant Significant!
(e.g. 95% or 99%) and report it as End of experiment Insignificant Significant! Insignificant Significant!
a “chance of beating the original” or
something like that. The first row is the same as before, and the reported
However, the significance calculation makes a critical significance levels after 200 observations are perfectly fine.
assumption that you have probably violated without even But now look at the third row. At the end of the experiment,
realizing it: that the sample size was fixed in advance. If assuming A and B are actually the same, we’ve increased the
instead of deciding ahead of time, “this experiment will ratio of significant relative to insignificant results. Therefore,
collect exactly 1,000 observations,” you say, “we’ll run it the reported significance level – the “percent of the time the
until we see a significant difference,” all the reported signifi- observed difference is due to chance” – will be wrong.
cance levels become meaningless. This result is completely
counterintuitive and all the A/B testing packages out there How big of a problem is this?
ignore it, but I’ll try to explain the source of the problem Suppose your conversion rate is 50% and you want to test
with a simple example. to see if a new logo gives you a conversion rate of more than
50% (or less). You stop the experiment as soon as there is
5% significance, or you call off the experiment after 150
observations. Now suppose your new logo actually does
nothing. What percent of the time will your experiment
wrongly find a significant result? No more than five percent,
right? Maybe six percent, in light of the preceding analysis?
30 SPECIAL
Try 26.1% – more than five times what you probably thought variance is given by:
the significance level was. This is sort of a worst-case sce-
nario, since we’re running a significance test after every
observation, but it’s not unheard-of. At least one A/B test- Committing to a sample size completely mitigates the
ing framework out there actually provides code for auto- problem described here.
matically stopping experiments after there is a significant If you write A/B testing software: Don’t report significance
result. That sounds like a neat trick until you realize it’s a levels until an experiment is over, and stop using signifi-
statistical abomination. cance levels to decide whether an experiment should stop
Repeated significance testing always increases the rate or continue. Instead of reporting significance of ongoing
of false positives, that is, you’ll think many insignificant experiments, report how large of an effect can be detected
results are significant (but not the other way around). The given the current sample size. That can be calculated with:
problem will be present if you ever find yourself “peeking”
at the data and stopping an experiment that seems to be
giving a significant result. The more you peek, the more
Where the two t’s are the t-statistics for a given signifi-
your significance levels will be off. For example, if you peek
cance level α/2 and power (1-β).
at an ongoing experiment ten times, then what you think
Painful as it sounds, you may even consider excluding the
is 1% significance is actually just 5% significance. Here are
“current estimate” of the treatment effect until the experi-
other reported significance values you need to see just to
ment is over. If that information is used to stop experiments,
get an actual significance of 5%:
then your reported significance levels are garbage.
You peeked... To get 5% actual significance you need... If you really want to do this stuff right: Fixing a sample
1 time 2.9% reported significance
size in advance can be frustrating. What if your change is
a runaway hit, shouldn’t you deploy it immediately? This
2 times 2.2% reported significance
problem has haunted the medical world for a long time,
3 times 1.8% reported significance since medical researchers often want to stop clinical trials as
5 times 1.4% reported significance soon as a new treatment looks effective, but they also need
10 times 1.0% reported significance to make valid statistical inferences on their data. Here are
a couple of approaches used in medical experiment design
Decide for yourself how big a problem you have, but if that someone really ought to adapt to the web:
you run your business by constantly checking the results
• Sequential experiment design: Sequential experiment
of ongoing A/B tests and making quick decisions, then this
design lets you set up checkpoints in advance where you
table should give you goosebumps.
will decide whether or not to continue the experiment,
and it gives you the correct significance levels.
What can be done?
If you run experiments: the best way to avoid repeated sig- • Bayesian experiment design: With Bayesian experiment
nificance testing errors is to not test significance repeatedly. design you can stop your experiment at any time and
Decide on a sample size in advance and wait until the make perfectly valid inferences. Given the real-time
experiment is over before you start believing the “chance of nature of web experiments, Bayesian design seems like
beating original” figures that the A/B testing software gives the way forward.
you. “Peeking” at the data is OK as long as you can restrain
yourself from stopping an experiment before it has run its
Conclusion
course. I know this goes against something in human nature,
Although they seem powerful and convenient, dashboard
so perhaps the best advice is: no peeking!
views of ongoing A/B experiments invite misuse. Any time
Since you are going to fix the sample size in advance,
they are used in conjunction with a manual or automatic
what sample size should you use? This formula is a good
“stopping rule”, the resulting significance tests are simply
rule of thumb:
invalid. Until sequential or Bayesian experiment designs are
implemented in software, anyone running web experiments
should only run experiments where the sample size has
Where δ is the minimum effect you wish to detect and
been fixed in advance, and stick to that sample size with
σ2 is the sample variance you expect. Of course you might
near-religious discipline. n
not know the variance, but if it’s just a binomial propor-
tion you’re calculating (e.g. a percent conversion rate) the
Evan Miller is a graduate student in Economics at the University
of Chicago, and the author of the Chicago Boss web framework.
31
STARTUP
32 STARTUP
Forget Servers. Forget Deployment. Build Apps.
heroku.com
33
Organic Startup Ideas By PAUL GRAHAM
Tfor you?
he best way to come up with startup
ideas is to ask yourself the question:
what do you wish someone would make
34 STARTUP
“Just fix things that seem broken, regardless of
whether it seems like the problem is important
enough to build a company on.”
of a startup idea. And in fact, it wasn’t initially a stuff, they’re in a position to discover valuable
startup idea. When Mark spoke at a YC dinner types of fixable brokenness first.
this winter he said he wasn’t trying to start a There’s nothing more valuable than an unmet
company when he wrote the first version of Face- need that is just becoming fixable. If you find
book. It was just a project. So was the Apple I something broken that you can fix for a lot of
when Woz first started working on it. He didn’t people, you’ve found a gold mine. As with an
think he was starting a company. If these guys actual gold mine, you still have to work hard to
had thought they were starting companies, they get the gold out of it. But at least you know where
might have been tempted to do something more the seam is, and that’s the hard part. n
“serious,” and that would have been a mistake.
Notes
35
Not Disruptive,
and Proud of It By JASON COHEN
36 STARTUP
than the 10 search engines that came their time,” able to create but not able I would have failed.
before. (Or 18.) to hold out long enough for others to I made less money personally at
Scott Berkun gives several other appreciate the innovation? Is it because ITWatchDogs, but the company was
examples in a recent BusinessWeek innovation and business sense are profitable and sold for millions of dol-
article. He highlights the iPod — an decoupled? Is it because “version 1” lars. We took a simple problem (when
awesome device, but not the first of of anything is inferior to “version 3,” server rooms get hot, the gear fails)
its kind. Rather, there were a bunch of and by the time the innovator makes and provided a simple solution (ther-
crappy devices that sold well enough to it to version 2 there are new competi- mometer with a web page that emails/
prove there were a market, but no clear tors — competitors who don’t bear the pages you if it’s too hot). There were
winners. Here an innovation in design expense of having invented version 1, many competitors, both huge (APC
alone was enough to win the market. who have silently observed the failures with $1.5 billion market cap), mid-
Not inventing new markets, not inno- of version 1, and can now jump right sized (NetBotz with millions in rev-
vative features, not even improving to version 3? enue and funding), and small (sub-$1m
on existing features like sound quality “Why” is an interesting question, but operations like us). We had something
or battery life — just a better design, the bottom line is clear: Disruption is unique — an inexpensive product that
unconcerned about “disrupting” any- rarely profitable. still had 80% of the features of the big
thing else. boys — but nothing disruptive.
Simple, modest goals are
Setting your sights on being dis-
ruptive isn’t how quality, sustainable
companies are built. Disruption, like
➎ most likely to succeed, and
most likely to make us happy.
Had we tried to fundamentally
change how IT departments monitor
server rooms, I’m sure we would have
expertise, is a side effect of great suc- It’s not “aiming low” to attempt modest failed.
cess, not a goal unto itself. success. There’s nothing wrong with mod-
It’s not failure if you “just” make a esty. Modest in what you consider “suc-
➍ The disruptors often don’t nice living for yourself. Changing the cess,” and modest in what you’re trying
“
make the money. world is noble, but you’re more likely to achieve every day:
The construction of high-speed Inter-
net fiber backbones and extravagant
data centers fundamentally changed
how business is conducted world-wide
My daughter convinced me that insisting
both between businesses and consum- something be Deeply Meaningful With
ers, but many of the companies who
built that system went bankrupt during Purpose could sometimes suck the joy from it.”
the 2000 tech bubble, and those who - Kathy Sierra
managed to survive have still not recov-
ered the cost of that infrastructure.
to change it if you don’t try to change Of course it’s wonderful that disrup-
They were the disruptors, but they
everything at once. tive products exist, improving life in
didn’t profit from the disruption.
I made millions of dollars at Smart quantum leaps. And it’s not wrong to
Disruptive technology often comes
Bear with a product that took an exist- pursue such things! But neither is it
from research groups commissioned to
ing practice (peer code review) and wrong to have more modest goals, and
produce innovative ideas but unable
solved five specific pain points (annoy- modest goals are much more likely to
to capitalize on them. Xerox PARC
ances and time-wasters). Sure it wasn’t be achieved. n
invented the fax machine, the mouse,
worth a hundred million dollars, and
Ethernet, laser printers, and the con-
it didn’t turn anyone’s world inside Jason is the founder of three companies,
cept of a “windowing” user interface,
out, but it enjoys a nice place in the all profitable and two exits. He blogs on
but made no money on the inventions.
world and it is incredibly fulfilling to startups and marketing at http://blog.
AT&T Bell Labs invented Unix, the C
see people happier to do their jobs ASmartBear.com.
programming language, wireless Ether-
with our product than without it.
net, and the laser, but made no money
Had I tried to fundamentally change
on the inventions.
how everyone writes software, I’m sure
Is it because disruptors are “before
37
Turning On Your
Reality Distortion Field By STEVE BLANK
H
Our product is really complicated ow do you put together a 30-second pitch?
After hearing more details about the features of
Envision how the world will be different
the product (I think he was heading to the level
five years after people started using your product.
of Quantum electrodynamics) I asked if he could
Tell me. Explain to me why it’s a logical conclu-
explain to me why I should care. His response
sion. Quickly show me that it’s possible. And do
was to describe even more features. When I called
this in less than 100 words.
for a time-out the reaction was one I hear a lot.
The CEOs reaction over his half- finished
“Our product is really complicated I need to tell
muffin was, “An elevator pitch is hype. I’m not a
you all about it so you get it.”
sales guy I’m an engineer.”
I told him I disagreed and pointed out that
The reality is that if you are going to be a
anyone can make a complicated idea sound
founding CEO, investors want to understand
complicated. The art is making it sound simple,
that you have a vision big enough to address a
compelling and inevitable.
major opportunity and an investment. Potential
employees need to understand your vision of the
Turning on your Reality Distortion Field future to decide whether against all other choices
The ability to deliver a persuasive elevator pitch
they will join you. Customers need to stop being
and follow it up with a substantive presentation
satisfied with the status quo and queue up for
is the difference between a funded entrepreneur
whatever you are going to deliver. Your elevator
and those having coffee complaining that they’re
pitch is a proxy for all of these things.
out of cash. It’s a litmus test of how you will
While my ex student had been describing the
behave in front of customers, employees and
detailed architecture of middleware of electric
investors.
38 STARTUP
“Envision how the world will be different
five years after people started using your
product.”
vehicles I realized what I wanted to understand Lessons Learned
was how this company was going to change the
• Complex products need a simple summary
world.
All he had to say was, “The electric vehicle • Tell me why I should quit my job to join you
business is like the automobile business in 1898.
• Tell me why I should invest in you rather than
We’re on the cusp of a major transformation. If
the line outside my door
you believe electric vehicles are going to have a
significant share of the truck business in 10 years, • Tell me why I should buy from you rather than
we are going to be on the right side of the fault the existing suppliers
zone. The heart of these vehicles will be a pow-
• Do it in 100 words or less.
ertrain controller and propulsion system. We’ve
designed, built and installed them. Every electric
truck will have to have a product like ours.” Steve Blank is a retired serial entrepreneur and the
75 words. author of Customer Development model for startups.
That would have been enough to have me say, Today he teaches entrepreneurship to both undergradu-
“Tell me more.” n ate and graduate students at U.C. Berkeley, Stanford
University and the Columbia University/Berkeley Joint
Executive MBA program.
39
Hacker Monthly is an independent project by Netizens Media and not affiliated with Y Combinator in any way.