One of the most popular ones answers the question “What is the hardest thing you do as a software engineer?”. I wrote something similar in Brazilian Portuguese in the post “Estimativas são Promessas. Promessas devem ser cumpridas.”.

In a nutshell, you can never give an estimate that is “correct”. If you could, it would not be an “estimate”, it would be a “prediction”.

Let us assume we have neither precognition powers nor magic crystal balls to tell us the future.

To estimate something is to “guess” the value of something. It’s always a guess, the same thing as a valuation. And like any guess, it can never be deemed “correct”: it’s just a likely candidate among an infinite range of possible values.

There is zero connection between a guess and the outcome. Understand this simple truth: saying something can happen does not MAKE it happen.

Estimating tomorrow’s weather as rainy does not MAKE it rain. Estimating the result of the Super Bowl does not MAKE it happen. There is zero correlation between an estimate and the actual outcome.

I stated in those articles that “Estimates are Promises”. The intention was to provoke a reaction, since most people assume that estimates can never be promises, exactly because of what I just explained.

What makes a promise special is that once you promise something, you’re expected to ACT on delivering it.

No one that has no “skin in the game” should give estimates.

If you’re not an active player in the game, you shouldn’t be handing out guesses. The same way no one makes promises on someone else’s behalf.

Can you actually make credible promises and keep them? You can, but first you have to understand a few more truths about reality.

You’ve probably already seen plenty of articles explaining project management methodologies and techniques. Most of them manage to explain the Whats and the Hows, but as usual, they fail on the Whys.

Why do we need those methodologies? Why are they necessary? Why do they work? What hidden mechanisms do they set in motion?

What makes Agile techniques any different from your usual homeopathy or a budget self-help cliché?

It’s always all about Pareto

There is no such thing as a precise project scope. There is a limit to adding detail, and past it you just get diminishing returns.

The most precise level of detail a feature scope can have is the actual code of the feature. And this matters: the lines of code are NOT the outcome. What end users actually experience is the execution of that code at runtime.

The programming itself is the actual blueprint. The diagrams, the use cases, the user stories, and anything else that comes before the programming are just a draft, a mere sketch.

A naive architect or designer might think that detailed diagrams, use case documents, and fancy PowerPoint slides are worth as much as an engineering blueprint. They’re worth less: they’re the equivalent of a sketch on a napkin. Volatile and, in practice, mostly worthless.

“But isn’t the programming the same as the construction phase itself, laying bricks on top of bricks?” NO, that’s not what a programmer does. The bricklaying is the job of the language interpreter or the compiler that spits out the executable binary.

This is the metaphor that drives non-programmers crazy. In engineering, the construction itself is the most expensive and time-consuming part. In programming, the expensive part is the blueprint (writing the code), and the “construction” is trivial: just compiling, which is automatic.

In the same way, a project scope is just a set of sketches. We have to drop the notion that a “complete” scope will ever exist. There’s no way to say “100% of the scope” or “closed scope”, because a software project scope is, by definition, always incomplete.

On top of that, I’ll argue that roughly 80% of this so-called “scope”, what I prefer to call a sketch, is mostly worthless to most end-user activities: the admin section, the institutional pages nobody reads, convoluted sign-up flows, and so on.

This is why every feature list MUST be prioritized. You can usually get by with 20% of the features, which is roughly what people mean when they say “MVP” or “minimum viable product”. Release as early as you can, get user feedback, and refine the rest of the “sketch” you call a backlog.

So instead of aiming for an all-or-nothing proposition, hunting for stupidly complex equations to figure out a “precise” estimate of an incomplete sketch, assume you can actually deliver EARLY the first 20% that really matters and figure out the rest in iterations.

Oh yeah, this is what we call “Agile”, by the way.

Agile is about Risk Management

People assume Agility is about managing the project management instruments themselves: the backlog, the rituals, the metrics.

Having Agile-like instruments doesn’t make you Agile.

Being Agile is keeping Risk under control.

Instead of treating projects as an all-or-nothing endeavor, start thinking about them the way an investor thinks about a portfolio of stocks. You don’t expect the whole portfolio to turn a profit; you assume some stocks will underperform. You just don’t know which ones, so you dilute your risk.

Trying to predict the stock market is an exercise in futility.

Trying to predict the precise implementation of a project, especially the long ones, is also an exercise in futility.

So you have to deal with uncertainty the right way: by becoming Antifragile.

“Some things benefit from shocks; they thrive and grow when exposed to volatility, randomness, disorder, and stressors and love adventure, risk, and uncertainty. Yet, in spite of the ubiquity of the phenomenon, there is no word for the exact opposite of fragile. Let us call it antifragile. Antifragility is beyond resilience or robustness. The resilient resists shocks and stays the same; the antifragile gets better” - Nassim Taleb

Instead of the preposterous exercise in futility of trying to predict uncertainty and random events, you do the reasonable thing: you assume Black Swans exist and you can’t predict them. So you prepare for uncertainty the only reasonable way there is, by not trying to predict them.

Expose the small mistakes early and correct them often. Implementing everything in a black box and doing a Big Bang deployment is the easiest path to failure. Delivering often, exposing the bugs, and fixing them constantly is accepting that mistakes will happen and gaining strength along the way.

A Better Metaphor

Imagine that you, whether the non-programmer client or the programmer who has no clue how to explain the process to that client, have an iron ore furnace to manage.

The thing about those furnaces is that if you heat them too much, they can explode in your face. If you cool them down too much, the ore hardens.

Your job is to add coal to the furnace, and you decide the rate. Too fast and it gets exploding hot. Too slow and you might put out the fire and lose your furnace.

Now try to estimate a constant intake of coal to keep the furnace in good shape.

You can’t.

The easiest way out is to install a thermometer that tracks the furnace’s current temperature.

You stay safe within a certain margin of temperatures, speeding up or slowing down the coal intake by checking the thermometer all the time.

ALL THE FREAKING TIME.

This is what Agile-based “Velocity” (or any of the fancier Monte Carlo simulations) actually is: a thermometer.

If the Velocity is too high, your team is probably working extra hours or delivering lower-quality code. This will backfire: either your team burns out too fast, or the code piles up technical debt too fast and you won’t be able to pay it back. Keep that pace and Velocity drops to a halt (the furnace explodes).

If the Velocity is too low, your team is slacking off, your backlog is a useless piece of shit no one can understand even after 10-hour meetings, or you already let Velocity run too high and now you’re paying back technical debt, or your team is dead from burnout (fire extinguished).

You want to keep Velocity steady, constant. This is what being Agile is all about: keeping an eye on the thermometer and responding.

The Triforce of Projects

Welcome to the Iron Triangle of Project Management.

Repeat after me: if I want to lock down the time, the cost, and the scope, I am a moron.

Repeat again.

You should lock down time and cost. If you’ve read this far, you know you can never lock down “scope”; you can only make it fat, and not necessarily more valuable. This is why I always say the very definition of a Product Manager or Product Owner is to be the bastion of ROI (Return on Investment).

Now, why?

Because the Iron Triangle has the following corollary:

• If you want it fast and good, it can’t be cheap
• If you want it fast and cheap, it can’t be good
• If you want it good and cheap, it can’t be fast

This is a Law, you can’t fiddle with it. Pick your choice.

How do I keep Promises then?

Now, with those 3 truths in hand:

• You can’t have your lunch and eat it too (cheap, fast, and good)
• You’re managing the temperature of the furnace
• You only need 20% of the “sketch” you call “scope”

Yes, any experienced developer can give you a “ballpark” estimate. A ballpark goes like this:

• 1 month (maybe 1 month and a half, but definitely less than 2)
• 3 months (it’s more than 2 months, less than 6 months)
• 6 months (it’s more than 4 months, less than 9 months)
• it’s more than 6 months, probably less than 1 year

Don’t even try to granulate more than that. It’s useless.

Lock the time. And lock the cost (that being the number of developers times the hourly rate times the total amount of estimated hours). That’s it.

Now jot down what the client calls “scope” as user stories in a backlog and have him prioritize it.

Start the iterations. After each one, release to a staging environment. Non-programmers, beware: ALWAYS make sure whichever programmer you hire delivers testable versions of the finished stories to a publicly available URL you can actually visit and test.

If your programmer refuses to do that or gives excuses: FIRE HIM.

If your programmer or company or whatever promises you a “closed scope” price, promises to do everything, and you believe them, you’re too gullible.

Do you think it’s funny to play this stupid game of “I’ll pretend to tell you that I know the truth and you will pretend that you believe what I tell you.”

No serious professional has time for stupid games, and the only honest thing anyone can say about any software project is: “given my experience I believe the ballpark for this kind of project is within X months, given the Y and Z assumptions”.

You don’t have to believe him. You just need to start checking the thermometer. Any non-programmer can assess the quality of the delivery based on the frequent deliveries of the prioritized features.

“But what if after 2 weeks I don’t like the results?” Easy: FIRE HIM.

And sometimes “firing” isn’t even the right word. Sometimes the relationship is difficult and the best thing to do is to part ways.

You have to accept losing 2 weeks, or any short period of time, as part of Risk Management. It’s better to accept losing 2 weeks of your project budget than to blindly trust someone for 6 months and lose the entire project budget and then some.

Pareto again: Agility is about Risk Management. You accept that losing 20% of your budget is okay, and you play with that. And that’s fine, because you only really need a bit more than 20% of the sketchy scope you have.

See what I just did with the Math here?

We stop playing the pretend game and actually manage the project’s risks. You collaborate on the equivalent thermometer, which combines the prioritized backlog (the scale) and the velocity (the temperature), and you keep an eye on the partial deliveries in the staging environment.

Conclusion

So yes, Estimates should be taken as seriously as Promises. You can give reasonable Estimates as long as you can manage the Risks and the Client accepts the rules of the game: there is no “closed, complete scope”, priorities first, and testing and accepting delivered features every week.

The idea behind Promises is that you have to MANAGE in order to meet them. The best way to do that is to frequently stop, re-assess, and keep going. It feels like you’re wasting time, but you’re actually saving yourself from wasting it.

If you don’t have skin in the game, back off.

Velocity is meant to be kept steady, at a predictable rate. Use the variance as a sign of being too fast or too slow, adjust the other variable, measure again, and go. Just like a furnace thermometer.

There are several versions of “thermometers”, from Joel Spolsky’s Evidence Based Scheduling all the way to fancy Monte Carlo simulations and other stochastic processes (they are all thermometers, and NOT estimation tools).

What stops you from doing that is using the wrong metaphors and the wrong references.

Instead of hunting for equivalent metaphors in construction, factories, and other “hard”-ware assemblies, look elsewhere, where you’ll find other “soft”-ware processes.

Musicians have deadlines. Painters have deadlines. Choreographers have deadlines. Sports have deadlines. Research laboratories have deadlines. How do they meet them? By constantly checking the current state, comparing it to the goals, assessing whether what they’re doing is actually working, and changing what doesn’t.

Hollywood has deadlines. They have far worse variables to control than any software project you’ll ever run into, and they still manage to deliver. And profit.

And accept that you can’t control every variable, so stop trying. Think of it as the financial markets. One day you have Ethereum skyrocketing 4,000% and the very next day you have it falling down to ashes in a flash-crash.

Don’t aim to become resistant or resilient. Prepare to become Antifragile.

Start simple with the article that lends its title to this compilation, Processes and Methodologies Won’t Help You, which I published on Abril’s blogs in 2011. It’s the starting point for understanding what processes, methodologies, and procedures are.

Next, go straight to the more recent Agile: the Truth Behind the Method and understand once and for all: no methodology or magic recipe will save incompetent or bad-character members of any team. After trying to improve attitude and training without success, you only rid the team of these elements one way: by replacing the members. Nothing saves a Net Negative Producing Programmer.

Many are left with that feeling of “waste”, like “man, but we’ve invested so much in him, what if we invest a little more?”. That comes from failing to understand that Sunk Cost is better than increasing your Opportunity Cost.

On the same “Agile is Dead” theme, or, more explicitly, “Agile-by-bikeshedding-consultants is Dead”, it’s worth recalling the old theme of The Problem of Credentials and Argumentation through Intimidation. The credential guy has the easiest job in the world: he shows up at your company to implement “practices” he doesn’t really practice. And trying to implement it at one or half a dozen places is ridiculously the same as nothing.

When it “seems” to work, he puts the case on his portfolio list as a “success.” When it goes wrong, he says “you didn’t do it right.” Terrible.

Forget pompous names and different schools of methodologies. Everything starts from exactly the same principle: Scientific Method vs Cargo Cult, where it’s clear no one understands the main thing about the method: it doesn’t pick sides or root for any team.

The phrase “it’s proven to work because I’ve seen it working in 100 places” doesn’t exist.

Anyone who uses a phrase similar to this is clearly an amateur. By Karl Popper’s principle of “Falsifiability”, instead of trying to prove something works, it’s better to try to disprove that it works. Because it doesn’t matter how many tons of evidence we manage to accumulate that something is true, a single piece of evidence that makes it false is enough to destroy any hypothesis. That’s why the scientific body is robust, not because we have mountains of evidence in its favor, but because what we’ve accumulated so far hasn’t been proven false.

But human beings are fallible. Our non-rational brain, a leftover of our long evolution, is still exceptional at seeing patterns where none exist, as I describe in We Are Mathematically Ignorant and Processes, Methodologies and the Human Brain.

The most accepted hypothesis is that we tend to confuse correlation with causation. Because “A” came before “B”, we tend to think that “A” caused “B”, and most of the time this isn’t true. The world is more random than we’d like to accept. Shit just happens.

This leads to the pseudo-intellectuals who want to push their “Southbeach Diet” of the day and become mere trolls or haters, as I describe in Math, Trolls, Haters and Internet Discussions. They create more and more diets, as I talk about in #noEstimates Debunked, Demystifying the Kanban method, and Watch Out for Kanban-butt.

And all of that I linked above is just the introduction to the most important thing of all: be skeptical all the time, nothing is automatically true, no matter who’s saying it. And as Carl Sagan would say: “extraordinary claims DEMAND extraordinary evidence.” In the end, we always come back to the theme of Enemies of Reason.

The problem is: if none of this works, what do you do? It’s much easier to say what’s wrong than to say what to do. The problem was never the recipe. Everything comes down to the decisions you make.

Each decision brings consequences, which lead to new dilemmas, which require more decisions. As Neo would say in The Matrix Reloaded: “The problem is choices.” There simply is no magic recipe, diet, or methodology that will make you make better decisions all the time.

There’s a desire to seek an external tool, an external source of “validation,” one of the worst words invented in recent times. The catch is that decisions carry consequences, and most people simply wish they didn’t have to bear them. That’s the root of the problem. This is the comfort zone: everyone complains and wants things, but no one wants to make decisions, much less face their consequences.

The only way to make better decisions is to study and train, as much as possible, all the time, continuously. Experience isn’t doing the same thing for 10 years straight, but becoming very good at 10 different things over those 10 years. Following the trend won’t help you, just like following magic diets. They leave you a bit thinner at the beginning, and soon the rebound comes and leaves you heavier than you were before. That’s why Processes and Methodologies Won’t Help You.

The trouble is using the phrase as an excuse. It turns into “we don’t need processes, just let people choose,” or “people choose, so we’ll have consensus on everything,” the fallacy of the democratic organization. At bottom, it’s a way to justify not understanding the value of having processes.

Agile, derived from Lean, is entirely based on processes. And the pillar of original Lean (not the derivations like Lean Startup or Kanban) is “Kaizen”, that is, “continuous improvement”.

The idea is that today should be at least a little better than yesterday. What we do is measurable, and because of that we can say that today came out better than yesterday. The principle is simple; the execution, not so much. If you haven’t read them yet, follow the links in this paragraph to understand these concepts.

Now comes the bad part: how do we talk about people? Or, more specifically, about people’s behavior?

Since we’re all “people,” we’re sure we know how people think. And nothing could be further from the truth. We carry preconceived ideas, fallacious, superstitious, and prejudiced, many of them long outdated. From being repeated so often as clichés, we still believe them, the same way plenty of people still believe we only use 10% of our brain or that you can’t have mango with milk.

And we don’t have to guess. Just like with processes, there is a long literature on people: psychology, sociology (which evolved a great deal since the 90s thanks to the study of networks), behavioral economics, among others. It’s impossible for someone from tech to become a specialist in the Humanities, but if the responsibility is to prioritize people, on top of processes, we’d better equip ourselves well enough.

This is a Small Bite because I don’t intend, this time, to go into detail on the subject. I’d rather leave a few books I really like to give some grounding.

Within the tech field itself there are some classics worth reading:

• Peopleware by Tom DeMarco
• The Mythical Man Month by Fred Brooks
• The Psychology of Computer Programming by Gerald Weinberg

From there we can move on to Behavioral Economics and Psychology:

• Thinking, Fast and Slow by Nobel laureate Daniel Kahneman
• Predictably Irrational by Dan Ariely
• The Upside of Irrationality: The Unexpected Benefits of Defying Logic by Dan Ariely
• The (Honest) Truth about Dishonesty by Dan Ariely
• The Lucifer Effect by Philip Zimbardo
• Talent Is Overrated by Geoffrey Colvin
• Descartes’ Error: Emotion, Reason, and the Human Brain by Antonio Damasio
• The Black Swan by Nassim Taleb
• Fooled by Randomness by Nassim Taleb

And, more specifically, about the behavior of various agents in groups (network theory):

• Linked, the New Science of Networks by Albert-László Barabási
• Six Degrees: The Science of a Connected Age by Duncan Watts
• Everything Is Obvious: How Common Sense Fails Us by Duncan Watts
• Sync: How Order Emerges From Chaos In the Universe, Nature, and Daily Life by Steven H. Strogatz
• Chaos: Making a New Science by James Gleick
• Nonlinear Dynamics and Chaos by Steven H. Strogatz
• The (Mis)behavior of Markets by Benoit Mandelbrot

Yes, a lot of these books are pretty superficial (none is technical or specialist), but the intent is just to give an introduction to the various areas. Over time, I dropped some that were too superficial, already in the entertainment and self-help category, like Malcolm Gladwell, Dan Pink, Clayton Christensen, and Clay Shirky. Even so, some of them are worth reading.

The main thing is the conclusion: contrary to what we usually think, we, people, aren’t as rational as we’d like, and our behavior is heavily influenced by external agents and by the environment.

A few articles that can set up the rest of the discussion:

• by Dave Thomas: Agile Is Dead (Long Live Agility)
• by Richard Bishop: Agile Is Dead: The Angry Developer Version
• by Giles Bowkett: Why Scrum Should Basically Just Die In A Fire

At my company, since day one, we don’t use the word “Agile,” “Scrum,” or any other buzzword. We do have “sprints,” but things like grooming and planning poker (just kidding!) are by no means “imposed.” The practices get used, with or without the names, whenever they’re needed. I don’t like saying “Scrum,” “Kanban,” or any other label; mentioning these names gives me a mild sense of embarrassment.

For anyone who actually applies agile practices day to day, none of this is news. Nobody argues about things like Collective Code Ownership (Github!), Continuous Integration (Jenkins, Travis, Semaphore, etc.), Test Driven Development (Rspec, Jasmine, Capybara, etc.), Refactoring, YAGNI, User Stories/Requirements/Use Cases (or whatever you want to call it: basically, scope). Look at how open source projects run: in terms of programming practices, it’s not much more than that.

The problem is the process: project management itself. People talk endlessly about Waterfall vs Agile, but that’s a load of bullshit. What’s worth lamenting is how the world of consultancies “specialized” in “Agile processes” has prostituted Agility.

And it’s true. That’s why I don’t “sell” things like “methodology implementation,” and I honestly don’t recommend anyone hire a third party for it. If you really must, size up how much hands-on time in the actual code and the day-to-day the candidate really has. Most of them rack up more “verbal” hours than hours of action.

In this article I won’t give the solution, that’s for a future one. Here I just want to state the one big truth nobody warns you about, which is why any implementation attempt will fail: implementing any Agile practices, processes, and methods requires a good team, programmers and managers and everyone else.

Does “good team” mean a team of nothing but “seniors”? No. It requires people committed to their craft, whether juniors or seniors. And by the way, the folks many people today call “seniors” or “ninjas” (argh, whether it’s because they give talks or because of their many years of “experience”) are almost all barely committed and merely arrogant. Again, more talk than action.

This point is crucial: plenty of people will try to implement “Agile” hoping to turn a bad team, or a bad company, into something better. That’s not going to happen. At first it’ll look like things “improved” or at least that something “changed.” But of course: you’re implementing a few merely “different” practices, so naturally something will change for a while.

Programming is a profession of practice. You can’t turn a bad programmer into a good one just by bolting on methods, the same way making a rookie soccer player sit through a two-day training will never turn him into a goal-scorer. Only hard, uninterrupted, consistent practice, made of small continuous improvements, will maybe get him there. In my cynicism, I told Bob Martin directly my theory about what motivated Ken Schwaber to create something as silly as Scrum Certification, at Railsconf 2010.

Anyone who was programming in the 80s and 90s didn’t see anything that disruptive when the descriptions of the principles and practices called “Agile” showed up in the late 90s, up to the publication of the manifesto in 2001. What was different is that someone finally condensed everything into a marketable commercial package, mostly Scrum (which on its own includes none of the important programming practices from Extreme Programming). Want to see how the practices have been evolving for far longer? Read The Cathedral and the Bazaar and The Mythical Man Month.

So no: implementing Scrum, or even starting to implement XP, will not save a bad team. One here or there, who already had the disposition, the talent, and the drive, will manage to become agile after practicing, screwing up, and practicing more. The rest, who don’t have the talent or the disposition, or worse, are flat-out bad-character, won’t change.

No methodology on Earth will ever change a person’s personality, at least not in so little time. Think of it this way: his mother couldn’t get the guy to quit being lazy and a stall-tactic artist, you won’t either, and that’s not your job anyway. Your job is to deliver value.

The unspoken truth of the Agile world is that applying agile practices comes naturally among good programmers who already have experience. It won’t be useful on a rookie team, which still needs a lot of practice, with or without agile principles.

I said you just have to look at how the open source world works. Contrary to what a lot of newbie Brazilians imagine, open source has, in principle, nothing to do with inclusion or leftist movements. I’d sum up open source as “the most efficient capitalist way to maintain commodities.” It’s highly exclusive to developers.

With no employment ties, on a purely voluntary basis, the interests revolve around marketing: company exposure, developer exposure, cost reduction, opportunities. You really do need to be “good” to stay at the top of the list. Does everyone’s opinion carry equal weight? Ask Linus Torvalds what he does when a bad idea shows up.

The open source world exposes problems immediately. Just because you think your project is “cool” doesn’t automatically translate into volunteers and exposure. The rate of projects that die is orders of magnitude higher than that of projects gaining traction. Compared to the commercial world, open source kills projects far faster.

Agile was built to expose problems as early as possible. Understand: Agile practices are problem-exposure mechanisms! Agile is a Risk reduction and management mechanism! Catch the obstacles the moment they happen and deal with them immediately to head off unnecessary future risk, the kind that only leads to waste.

Except nobody wants to do this. If the problem is the manager, what consultancy is going to remove him from his post, unless the owner above him is committed to cutting into his own flesh? If the problem is the bad-character programmer everyone likes (the most dangerous type: the one who knows how to work a room and build zones of influence among peers and superiors), how do you fire that guy?

Yes: fixing a team in a short window necessarily means replacing its members. Worst case, replacing all of them. If the method is applied well, it immediately demonstrates and exposes the problems. And since in the vast majority of cases the problem is behavior, not practice, you have to resolve it on the spot. If you, the stakeholder responsible for your company, aren’t firing anyone after you start implementing Agile, you’re doing it wrong.

And here’s the big problem: nobody gets this commitment. There’s no free lunch. There’s no making an omelet without breaking some (or all) of the eggs. Without that understanding and that commitment, there’s no agility. And this isn’t the first time I say it, if you read my 2009 article: Net Negative Producing Programmer. But since most people still seem not to have gotten it, I decided to spell it out more clearly.

This was an introduction. Depending on the comments, maybe I’ll walk down a few more steps to explain exactly the mechanisms of what I laid out very quickly here.

If you’re an experienced programmer, you don’t need me to tell you this. But if the recent news rattled you, maybe what I have to say will give you some perspective.

First, I want to share a few other articles I read over the last few days:

• My Next Chapter: the statement by Mike Perham, one of the best-known Rubyists of this generation and creator of the ubiquitous, excellent Sidekiq. He’s not abandoning the project; he’s building a new product, and that product doesn’t involve Ruby.

• Saying Goodbye To Python: here it was the Pythonista Ian Bicking, announcing he was trading Python for JavaScript, riding the current wave. Worth understanding that today’s JavaScript bubble doesn’t “require” you to “leave” anything for it. You can quietly stay a Pythonista and use JavaScript at the same time, no conflict.

• Why I Left the .Net Framework: a developer named Jonathan Oliver also decided to lay out why he’s leaving his day-to-day technology, weighing the pros and cons of the Microsoft stack. No troll here; it’s actually a pretty interesting read.

• Java Again: anyone from the first generations of the Java community remembers James Duncan Davidson, creator of nothing less than the first Tomcat and Ant. He left Java, spent a while in the Ruby community, and threw himself into photography (go look for photos of the RubyConfs from eight years ago and you’ll see the official ones are all his). Now he’s announced he’s going back to Java.

• Why Go Is Not Good: as the title says, it’s a critique arguing that a freshly launched language should have been thought through better to cover cases other languages already handle.

• Rust vs Go: again with a whiff of sour grapes. Rust really does look like a technically superior language to Go in design and features, and the piece shows the frustration of watching Go stay more popular anyway.

• Why Perl Didn’t Win: mandatory reading on the fortunes of one of the most popular languages from the late 80s to the late 90s, which fell into obscurity and disuse. Even Swift, released by Apple weeks ago, already has more active users than Perl 6. One of the takeaways: anything new has to aim at the problems of the future, not the ones from the past.

Anyone from the first generations of the Ruby and Rails communities will remember the “traumas” we went through:

• Zed is Not Dead: from 2008, when one of our most prolific developers, Zed Shaw, creator of Mongrel and one of the first to take on the challenge of measuring and optimizing Ruby’s performance, announced he was leaving the community and moving to Python.

• Bomb: Merb and Rails Merge!: also from 2008, when the Merb team was on a collision course with Rails Core. The solution, happy for many and unhappy for a stubborn few, was to fold Merb into Rails 3, and it left dead and wounded behind. In retrospect, it’s worth reading “Learning from Rails’ failures” by Matt Aimonetti (who also moved to Go) and who counts among the wounded of that insurgency, and “The Rails/Merb Merge In Retrospect” by Giles Bowkett (still a Rubyist), who breaks down the results with a very critical eye.

• Twitter switches from Ruby to Java: from 2011, when Twitter announced it was replacing the messaging-infrastructure parts written in Ruby with a proprietary solution in Scala. What the original news said and what people chose to remember are two different things, but the fact is it turned into a good Public Relations headache for us. Thankfully that one’s behind us.

And these are just the links I’ve read and can recall off the top of my head. There are dozens more.

Anyone still young or inexperienced in software communities, especially newcomers to Node.js, must be asking themselves: “Now what? If someone important like TJ says he’s leaving, shouldn’t I leave too?” Do your homework and you’ll see this isn’t the first time. In 2012 the creator of Node.js, Ryan Dahl announced he’d keep an eye on his project but would step out of the day-to-day coding. “Man, the world is ending!”

This Isn’t Bad News. This Is an Excellent Opportunity! (Seriously!)

In 2006, right as I was starting my run as an activist in the Ruby and Rails world, I already knew what was coming. We were the newcomers, immature, with unfinished technology and a pile of expectations, so naturally the road would be as hard as it gets. That’s why I wrote the article Evolution through Competition.

A lot of people imagine the best of all worlds is the one where nobody questions what you’re doing, everyone leaves you alone, and they take your promises on faith, promises you fully intend to keep, someday. (yeah, right…)

I disagree. Any promise or expectation worth having has to be questioned, pushed, scrutinized. If it’s so fragile it can’t take criticism, direct and indirect attacks, and sabotage, then the promise was weak to begin with and didn’t deserve to move forward. The sooner it falls, the better, because that’s less time wasted.

That’s why I always welcome the attacks: I expect to be ready to hit back twice as hard or die trying. And I’d expect nothing less from the Rails community. Everyone still believes constraints are bad, but I’ve said before that constraints matter, because they’re the only place innovation comes from.

Anyone who has to reach for the old trick of trashing others, the classic “I don’t feel like getting better, so I’ll drag the other guy down to look bigger” (I wrote about this back in 2007 too, “For me to win, the other has to lose…”), is declaring himself weak all on his own. Stop and think back to articles, talks, and interviews where you heard someone you admire trashing another community. “Ah, Ruby, that girly thing. Because we men use JavaScript.” It makes me laugh and feel a little sorry, because to me it’s clear that person will always be weak.

All the statements above, and others like them (“I’m leaving Node”, “I’m leaving .NET”, “I’m leaving Ruby”, “I’m leaving X”), I honestly believe came with no bad intent or bad faith, at least not consciously. But I have a theory. Everyone who makes statements like that wants validation from their peers, or worse, wants their peers to follow them, which is also validation of their own position. “I don’t like my old toy anymore, but I don’t want anyone else playing with it either.” It’s a pretty childish stance.

Take the Ruby community as an example. Plenty of names sort of “left” their main activities without slamming the door or acting childish about it. Chad Fowler is in Germany running Wunderlist, focused on mobile and back-end development, and we don’t hear him talk Ruby anymore. Ilya Grigorik had his startup bought by Google and eventually dropped off too. Matt Aimonetti, after the Merb vs Rails episode, quietly stepped back and now works with Go.

Ryan Bates, one of the most beloved in the community for his tireless years of work on RailsCasts, stopped publishing months ago and said nothing about it. David Chelimsky, maintainer of RSpec, handed off the baton and stepped away without any fuss. José Valim, one of the key people behind Rails 3, is over in Elixir and Erlang land now and rarely says a word about Ruby.

“O.M.G! You mean it’s all over!? It went up in flames!?”

No. In the meantime, hundreds of new Rubyists showed up and carried the work forward. The most immediate example is Carlos Antonio and Rafael França, who sort of picked up José Valim’s mantle in Rails Core. Evan Phoenix picked up Chad Fowler’s mantle at Ruby Central. Avdi Grimm sort of picked up Ryan Bates’ mantle with his excellent RubyTapas, and on it goes.

A community is made of individuals, and each individual performs best when acting to maximize their own benefit. The side effect is that they end up naturally influencing and pulling others up alongside them. It’s always voluntary exchange for mutual benefit. Nobody acts with zero benefit to themselves; some believe they do, and those are exactly the ones who hit “burnout” fastest, because the question quickly becomes:

"Why do I wear myself out collaborating in this community, stress myself out, and even so not everyone appreciates what I do? I do it all for them!"

That’s the wrong question, and the wrong question leads to the wrong answer. Some start to believe others owe them something, since they poured so much of their free time (especially in the open source world) into “helping others”. And “the others” figure they owe nothing, since nobody asked for that effort in the first place. In practice, “the others” are right.

If you take part in a community, you do it for yourself. Maybe because you understand that helping others is part of your own reward. Maybe because it gives you peace of mind or a clear conscience. Or because you’re a good strategist and know certain contributions will open doors for you down the line. The reason doesn’t matter, as long as there’s a reward you don’t force others to hand over. Demanding respect, demanding manners, demanding compassion, none of it makes sense without a counterpart. Nobody asked you to do what you’re doing; if you are, understand it’s for yourself. That’s the definition of voluntary work.

And to be crystal clear: voluntary work means work without financial pay, and not work without benefit or purely altruistic. If your work is paid, that’s a different conversation, and the best advice there comes from Don Draper:

Back to the topic, I imagine the names I mentioned above, the ones who stepped back partially or quietly, get this. They went looking for personal satisfaction elsewhere. They owe nothing to anyone, and nobody owes anything to them.

The ones who leave with bombastic declarations are, consciously or not, throwing it in everyone’s face: “you didn’t value me, now you’ve lost your chance, and I know you’ll miss me!” It isn’t written that way, but that’s what it adds up to. Otherwise, why throw it into the fan and turn it into a public controversy? Any written argument is only part of the reason, never “the” reason.

You, the Individual

The flip side is whoever follows, admires, and idolizes one of these people who walked away from where you are. Depending on how deep the admiration runs, you feel the floor drop out, stunned, not knowing which way to run. One of two things happens: you either go hunting for another idol and start the classic “he wasn’t even that good”, or you sink into depression and the panic of “I don’t know what happens now”.

My advice: "don't have idols, have references."

Stop thinking you owe others something and, by extension, stop dumping on others the job of helping still more people. That’s what’s going on: you don’t understand your purpose and figure others should do what you yourself won’t. Worse, you let others decide which road you take. If you’re that insecure, stop for a second and think about what you “are” before anything else.

Your only purpose in life is to take care of yourself and produce, without getting in anyone else’s way. When you do that, the natural effect is that spontaneous collaborations spring up, communities evolve, and everyone comes out ahead. A community where everyone makes demands on everyone grows without any purpose.

On technology, opinions, proposals, and promises, remember this too: you want to be under constant “attack” and constraint. Nothing is worse than being ignored. I wrote another article, a translation from someone far wiser than me, Kathy Sierra, called Dilbert and the Zone of Mediocrity, where I quote:

How brave are you? How far will you (or your employer) go to avoid the Zone of Mediocrity? Until or unless you're willing to risk passionate hate, you may never feel the love. Scott Adams agrees. In a recent post on the Dilbert blog, he wrote: "If everyone exposed to a product likes it, the product will not succeed... The reason that a product 'everyone likes' will fail is because no one 'loves' it. The only thing that predicts success is passion, even if only 10% of the consumers have it."

So what if someone left your community? Is it so fragile that one blow, no matter the size of the impact, brings the whole thing down? If you believe your community is only resistant, you’re right: past a certain point, it collapses. But if it’s resilient, malleable and adaptive, it survives. That’s what guarantees the survival of species, the Law of the Most Adaptable (not the “strongest”! Keep the difference between resistant and resilient in mind, you want to be the second one).

Are you adaptable? Or are you just fishing for other people’s validation, letting them decide for you? One thing I’ll say for sure: if I ever give up what I do, nobody will find out from a written statement. I settled that the day I started.

I wrote this article in English right after the last episode of the series aired on TV, in October 2013. A few months have passed, so I imagine everyone has had the chance to watch the whole thing by now. For me, this was one of the best TV dramas of the last few decades. It comes along with The Sopranos.

I won’t review every detail of the series, since there are plenty of other blogs doing that with more competency than I would. I’ll assume you’ve watched the series, and I’m going to spoil parts of the finale, so go watch it before you keep reading.

People love the series, and they root for Heisenberg even knowing he commits plenty of crimes. Personally, I think what grabs people is the story the author is telling, far more than the crimes themselves.

This is the story of a very, very successful startup, literally the classic “garage startup”, except in this case inside a van. Walter and Jesse start with an idea, a few bucks, and a lot of work, cooking meth in the middle of the desert. Even with almost no money and no resources, Walt is adamant about the quality of the product, and that’s a mark of great entrepreneurs. Jesse is the metaphor for the wannabe entrepreneur, a dropout trying to hit the jackpot without experience and without thinking ahead.

For me, the whole idea of Breaking Bad revolves around entrepreneurship. Walt is not only a seasoned chemistry teacher, with real knowledge and talent; he’s also a great businessman, and he proved it many times. One example is understanding that, to score big, growing organically by selling on the streets Jesse’s way just isn’t enough.

Then come Gus Fring and Los Pollos Hermanos, with the explosion in distribution channels. That’s how you scale a business. Gus is Walt’s “angel investor”, and once the money is in, the pressure only goes up. Jesse quickly gets fed up with working so hard in the lab, like every young idealist who doesn’t yet understand what real hard work is. Walt, on the other hand, is focused on scoring big at any cost.

The show also demonstrates how to negotiate with partners you don’t particularly like or trust, in a very demanding game. Once you get the opportunity, you don’t have to do all the dirty work with your own hands: make big dog eat big dog, turn the Cartel against Gus. And if everything goes wrong, you can always blow up the nursing home.

In the process, what we see is Jesse growing up. Walt was a great mentor, and their adventures made him stronger. The finale is also about Jesse understanding what he really is and maybe becoming a better person, in terms of self-awareness. Now he’s ready to commit and sacrifice, instead of being just an idealist.

In the real world, you have to commit and make sacrifices. There will be collateral damage, because you can’t make an omelet without breaking some eggs. Nothing worthwhile is easy, and the process is never going to be pretty.

The finale is great because Walt finally admits it: when you’re building an enterprise, you do it for yourself. Not for your family, not for your friends, not for the good of mankind. It’s for you and you alone. If you’re not doing it for yourself, you’re doing it wrong, and that’s the most basic core concept of entrepreneurship.

Benefiting (or hurting) others is always a side effect of actions you take with the single goal of benefiting yourself. As long as you’re not doing it for yourself, you’ll always use others as the easy way out, the excuse. “I screwed up, but I did it with good intentions.” Good intentions don’t matter, results do. If you do everything for yourself, the responsibility is always your own. No excuses.

Being successful is much more than glory and fame. As Breaking Bad shows, when your business grows, the competition gets harsher, the attacks get more violent, and everybody wants a piece of your pie. Defending your own territory is up to you, and you can’t always do it alone.

Sometimes you need good mercenaries and always a good lawyer, like Mike Ehrmantraut and Saul Goodman, to cover your tracks. It’s the cold-blooded partners who handle the deals you can’t touch all the time.

Marie represents the worst and most degrading thing in the world: mediocre normality. People so entrenched in the status quo, so bored by their own existence, that they’re mortally jealous of anyone who won’t follow their tiny version of “truth”, the path of “being normal”. People full of prejudice and full of themselves, who judge everything and everyone all the time. The best thing to do is stay as far away from that rotten kind as possible.

Skyler starts out the same as Marie, but being so close to Walt, she fights against herself. She suffers to get out of the box, to get used to taking risks, to think differently. She goes through all the common stages of transition: denial, anger, bargaining, depression, and acceptance. In the end, we see that she finally understood.

Walt was not the villain, he was a man doing what he does best. Someone who values his own work so much that he won’t let anything or anyone stop him. That’s “drive”, and it’s the level of drive anyone who wants to be an entrepreneur needs to have. Reaching the goal, whatever it takes.

Walt is the ultimate entrepreneur role model, really a lot like Tony Soprano. The Mafia is the best MBA in the world.

The year was 2001. A group of senior programmers gathered at a ski resort in Snowbird, Utah. The main names of programming in the United States were there: Kent Beck, Alistair Cockburn, Ward Cunningham, Martin Fowler, Jim Highsmith, Ron Jeffries, Brian Marick, Bob Martin, Ken Schwaber, Jeff Sutherland, Andy Hunt, Dave Thomas, Mike Beedle, Arie van Bennekum, James Grenning, Jon Kern, Steve Mellor. A “who’s who” list of the field.

From that historic meeting a manifesto of 4 values and 12 principles emerged, which became known as the “Agile Manifesto.”

Thirteen years later, Dave Thomas, author of “The Pragmatic Programmer” and one of the people most directly involved in writing the manifesto text, published a frustrated article called “Agile is Dead, Long Live Agility”.

The original agilists meant to break the outdated way of thinking about and executing software engineering, the same obsolete model still taught today in most of the country’s universities. It caused a huge stir in the industry.

Along came the derivations, the trainings, the tools, the certifications, the conferences, the consultancies. A huge portfolio of options, for any size and any price. In the middle of this whole circus, what it really means to be an agilist got forgotten.

And, as Dave Thomas summed it up well in his vent, you don’t “do” Agile or “buy” Agile. Agility is a quality, an adjective. To be agile, you have to go back to the principles. Quoting:

• Find out where you are
• Take a small step towards your goal
• Adjust your understanding based on what you learned
• Repeat

How to do this:

“When faced with two or more alternatives that deliver roughly the same value, take the path that makes future change easier.”

And that’s it: four lines and one practice.

Thirteen years after something so simple was first published, the world seems to have gotten way too complicated. Scrum, Kanban, Scrumban, Lean Programming, Lean Six Sigma, Agile PMI. In a moment where chaos reigns, whoever “seems” able to explain the chaos becomes a guru.

And so associations, certifications, trainings, and conferences emerge. That’s how it works for diets, how it works for therapies, how it worked for software engineering. That’s what Dave Thomas laments.

But it’s an inevitable result. People are easy to fool: they always look for someone or something to follow and rarely question. They’ve been dogmatized never to challenge authority and to root for soccer teams without any criteria.

In 2010 I was at a conference in Baltimore and had the chance to talk to Bob Martin, who organized the Snowbird meeting. It was in his Object Mentor office that Ken Schwaber’s first Scrum certification training took place.

I was already immune to the “Agile Hype,” but I wanted to tell Bob Martin what I thought of it. For those who don’t know, the “official” Scrum Master training is a two-day course, and at the end you walk out certified. You don’t even need to show up: just pay for the course.

I told him, cynically, that Ken’s idea probably backfired. “Ken probably thought ‘man, how do I convince these large corporations to adopt agile practices?’ and then he thought ‘of course, large companies love certifications,’ and then he created a Trojan Horse, a joke 2-day course where serious people would know you can’t learn anything serious in 2 days, but that way he could insert the idea into the companies. But everyone took it seriously, and that’s when the shot backfired.” Skip straight to minute 6:24 of the video below if you want to hear that story.

Learning to Learn

Now let’s step out of software a bit and enter the world of “startups.” That world has its own “Agile,” dubbed “Lean Startup.” Eric Ries coined the term in 2008 and made it famous with the book of the same name, from 2011, a direct derivation of Toyota’s Lean principles. It’s a repackaging of old ideas.

This Lean comes from the same ideas of W. Edwards Deming about PDCA, “Plan, Do, Check, Act”. You’ll notice the resemblance to Dave Thomas’s four principles I cited above.

Pompous names, celebrities, gurus, expensive tools, and prominent titles don’t matter. The basic principles of Agile and Lean are the same: deliver value as fast as possible with the least waste possible, learning and using that learning in the process, planning and replanning to reach goals that become clearer along the way. That’s it.

The biggest problem shows up when an amateur programmer asks “what’s the next step?” and the answer is “the goal is to set up a Kanban.” Or when an amateur entrepreneur asks “what’s the next step?” and the answer is “the goal is to build a Business Canvas.”

Kanban and Business Canvas aren’t goals, they’re means. If those means don’t help reach the ends, delivering value to the customer and turning a profit in your business, then they’re disposable. The goal is to have profit. MVP, prototype, Lean UX, and Business Canvas are just possible paths to get there. Which of them serve?

If you want to learn anything, the steps to start are simple:

• Don’t create idols, never “idolize” anyone, at least not seriously.
• Ignore almost everything you see or hear in groups. Take notes, but don’t take everything seriously.
• Ignore advice and mentoring. It’s like a bank manager giving investment tips: if he were any good, he’d be rich instead of selling advice. He’s a salesperson by another name.
• Everything, absolutely everything, starts with a few principles. Nobody becomes a heart surgeon wanting to cut the aorta before learning to wash their hands. Look for the principles and ignore the pompous titles.
• Assume you know little, always. Approach every subject with the eyes of an ignorant person. Whoever thinks they already know everything learns nothing more. But be a smart ignoramus, not a dumb one.

Learn to question. The whole world of the Sciences, from the absolutely small quantum world to the infinite and immense cosmological world, only exists because it accepts one principle: the Scientific Method.

It’s basically PDCA: define the problem, make hypotheses, run controlled experiments, discard the hypotheses proven wrong, repeat. To be more accurate, Deming’s PDCA is the one that draws directly on the Scientific Method.

There is the principle of everything. From Agile to Lean to PDCA, we come back to Karl Popper and his philosophy of “Critical Rationalism”, which teaches that everything should stay open to rational criticism.

That’s why Lean, as an umbrella for selling products and services you don’t need, is dead. Long live the principle of achieving value while avoiding waste through continuous learning, in other words, Efficiency. There are no magic formulas. If magic diets worked, there would be no obesity in the world.

PS: I know, there’s #tongueincheek in the title and in the article, but the intention, as usual, is actually to provoke ;-)

For a change, SoHo is one of those subjects most of us only think about at the surface before deciding “it’s good” or “it’s bad.” It’s bad that Marissa Mayer banned Home Office. It’s good that 37signals is evangelizing Home Office. It’s bad that Microsoft agrees with Home Office, but with a possibly dystopian view. Worth remembering that the industry has been chewing on SoHo practically since the personal computer showed up in the mid-80s. There’s nothing new about it.

A disclaimer: I own a software shop, a company that offers development services. The core of the company is programmers hired not only in São Paulo, where I live and have clients, but also in Porto Alegre, Fortaleza, and Natal. I don’t intend to get into my process in this article.

I thought about dissecting the book Remote, which is the reference of the moment. But there isn’t much to dissect. If you read the table of contents, you basically know everything the book says already, and the rest is filler. Look:

• The Time Is Right for Remote Work: it lists everything we already know as the downside of big offices, the long meetings, the interruptions, the commute, the quality of life. Without reading the chapter, we already know what bugs us. The problem is that this is hard to argue against, because any place has things you like and things you don’t. Listing only what you dislike isn’t the most relevant thing. If you have a girlfriend or you’re married, did you make a list of your partner’s traits stacked up against Miss Universe or Mr. Universe? Is it the set of things that annoy you that matters, or the set of things you love?

• Dealing with Excuses: again, the “arguments,” the “demagoguery,” the “rhetoric” assumed to be the excuses managers and company owners use to block remote work. Half of them don’t even come out of the companies’ mouths; they come from the motivational self-help crowd, along the lines of “working close together boosts information exchange and therefore creativity.” The other half are policies like data security, customer service, loss of control. Those items are secondary. As a company owner, almost all of them are easy to handle and agree with. The industry isn’t that bad, and none of these points slipped by unnoticed over the past few decades. There are still leftovers, no doubt, but compare today with 20 years ago and you’ll see the difference.

• How to Collaborate Remotely: it sums up what we already know, that technology shrank distances. Skype/Hangout, Basecamp/Tracker/Asana, Github/Bitbucket, IRC/Campfire. On top of that, plenty of outsourced services are already remote. An accountant is a good example, and I’ll circle back to it later, so hold that thought.

• Beware the Dragons: the one part that “balances” the book and admits remote work isn’t always a bed of roses. Not feeling lonely, not working more hours than you should, keeping healthy habits. These are tips that apply to remote work and to anybody’s routine. Bottom line: use common sense, but you already knew that.

• Hiring and Keeping the Best: in short, and here I agree, there are excellent professionals in every corner of the world. That’s my reason for looking beyond São Paulo. Again, it’s easy to evangelize an argument without weighing every angle. I agree with the idea and I’ll come back to it later.

• Managing Remote Workers: once more, management common sense. Good managers already know this. We’ve had multinationals running on every continent for decades, and we’re good at it. A lot of people believe companies reject remote work because they don’t know how to manage people at a distance. That doesn’t hold up: look at the massive movement of outsourcing services to countries like India. Like the newspapers, we only look at the bad news, but we know how to do this, and it keeps getting simpler. Outsourcing to individuals looks more complicated than outsourcing to branches in other countries, but overall the process is the same.

• Life as a Remote Worker: more common sense for anybody. Have discipline, build a balanced routine, set aside time for your family. The only genuinely relevant bit is the last one, “make sure you aren’t being ignored.” That’s my central point for the rest of the article, so hold on to it.

To wrap up: of the 37signals books I’ve read, this is easily the weakest. When reading the table of contents is enough to grasp the whole book, the conclusion comes easy. That doesn’t make it wrong or bad, but a blog post could carry the same message: yes, there are plenty of advantages to remote work. And, of course, by pure coincidence they sell tools for exactly what the book describes. It’s a terrific piece of self-marketing, and it’d be dumb not to do it.

The Truth About Remote Work

Now for the other side of the coin. In my recent article Math, Trolls, Haters and Internet Discussions I tried to get across a simple but important idea: an incomplete formula is useless. To recap, a procedure, a methodology, or a process only becomes a “formula” once it defines the “domain” of the problem.

I’ll assume 37signals is arguing that service professionals could work remotely. I’m restricting it to computing because plenty of other fields simply don’t work that way, like airline pilots, firefighters, doctors. The domain is limited to services, never production, and whose output doesn’t need physical media, meaning it travels over the Internet. Musicians, programmers, writers, accountants, lawyers, architects (partly). Simple enough to define. For my purposes, I’ll stick to software development services.

The biggest problem with books and people who evangelize about human behavior is ignoring Sociology, Psychology, and the fields that already studied this thing to death. The book Remote is convenient for 37signals’ domain and works there. The way it’s described, it works only there. That’s the first tell for separating serious research from self-help: “It worked for me, so it can work for you.” That’s how they sell diets.

For several reasons, 37signals did very well as a product company. Among them, the ones Zed Shaw explains in this talk and the capital injection from Jeff Bezos. Good for them, but it means the problem domain is “product company that brings in revenue and profits very nicely.” And that already knocks a huge slice of the market out of the equation. If you don’t know the numbers, we’re talking about a much bigger universe, one that in Brazil came close to R$ 37 billion. Way bigger than the narrow universe of tech startups.

Worth understanding another thing that both Getting Real and Remote try to argue, and could have simplified. Highly centralized structures, with heavy top-down control and little or no regard for the workforce, tend to be bad places to work. The hasty conclusion is that full decentralization is the best form. Worse still is picturing 37signals as a decentralized company just because everyone is remote and there’s no rigid hierarchy. In practice it’s centralized, with intermediate levels of decentralization.

It gives the impression of a decentralized organization because it uses open source processes for execution. But the chain of command is clearly centralized in Jason Fried and David Hansson. There’s nothing wrong with that, and it’s exactly how it should be.

With that in mind, I want to show a chart I pulled from a Nicholas Christakis lecture. He cites a study on Broadway shows. Some are big hits, others are box-office flops, and the researchers wanted to know whether there was any correlation between the type of social network in the show’s organization and the success rates.

Picture on the X axis the density of relationships among the show’s team professionals. 0 is the highly centralized network, where people at the edges don’t even talk to each other; 100 is the highly decentralized network, where anyone talks to anyone. Level 0, intuition already flags as bad, and the correlation confirms it: it shows up in the shows that flopped. The counterintuitive part is that fully decentralized networks flop just as much. The successful shows sit in the intermediate networks, with some decentralization and some centralization.

37signals isn’t fully centralized, since the remote programmers talk over chat, via Github, and so on. And it isn’t fully decentralized, because there’s some level of specialization and, of course, a central strategic command.

I’m testing different scenarios, and the first step, in my case, starts with this other diagram:

But, as I said, I won’t stray from the main subject here. Just a hint.

The Career Problem

I keep repeating that defining the domain matters because, in 37signals’ case, you can raise people’s salaries almost indefinitely. The product already hit a scale where revenue is orders of magnitude larger than costs and expenses. Well, if you can bankroll racing at Le Mans, you can pay programmers 5 or 6 figures.

Keep evolving the product, build new products carefully, keep old customers happy, do good marketing to draw new ones, in short, what a good company should do, and you can keep growing for a good while.

Except a product company like that isn’t common. It’s actually the rarest case in the industry, and the same rules don’t apply to service companies, which are the majority. And the reason is structural, tied to the fundamentals of the industry.

From that comes the reasoning: “That’s why opening a service company isn’t worth it, better to go straight to products.” My answer is simple: “good luck.” The obvious premise is that product companies pan out right away. I already covered this in my previous article Tech Startups, B2C overcrowding. Boring..

“Oh, just follow Lean Startup, Agile processes, Business Canvas,” and the whole little package they sell today, with instructions any kid can follow and batteries included, and boom, automatic success… does anyone really buy that nonsense?

Now back to reality. The biggest problem with hiring home-office programmers is HR and career management, not just control. We’re reaching a maturity level in software development where programming with good practices, good technology, efficiency, predictability, and easy future maintenance stopped being a Premium and became the norm. I won’t offer a client a cheaper programmer because they’re low quality. I want my junior to move up to mid-level fast, the client to get the same technical level no matter who does the work, and the quality to stay above average, because the industry average is still below the poverty line.

Except, if that’s the norm, how does a company give programmers raises? Up to a point, technical ability makes a big difference. But cases of very high, still-experimental technology, where few people know what to do, are rare, in services and in products alike. The rest gets commoditized fast. The open source world all but guarantees that once a hard problem is solved, it gets replicated in no time. If making a responsive, halfway good-looking web front-end was once “complicated,” a Foundation or a Bootstrap already flattened most of that curve.

This is the core of the issue: just programming is too little for 90% of software projects. I discussed in another article, Estimates Are Promises. Promises Must Be Kept., that the hardest thing in a Software Engineer’s career is realizing that much of a software project isn’t solved with software. A “senior” professional is defined less by the prettiest code and more by managing client expectations, whether corporate or consumer, and by taking on responsibility.

Again, I won’t get hung up on exceptions like research centers or established products. Inside a group, properties emerge that only exist inside a group. (See the Christakis talk again, or others like Duncan Watts, to start getting a feel for these properties.) These are activities that usually don’t happen solo: peer-to-peer training, where a junior gets to learn from someone more senior, plus the exercise of communication, negotiation, responsibility, and teamwork.

All of that is possible while remote, just much slower and much less obvious, because each individual really is isolated. A geographical cluster also influences the market and the local community around it, so it isn’t confined to your primary group: secondary groups emerge. And each cluster relates the way Remote itself describes, with online tools and the occasional in-person trip.

In practice, how much more productive and efficient is a good senior developer than a junior? Ten times, tops? Meaning if a junior earns 1000, a senior earns 10,000 at most? And after that? Now, a good professional who trains, mentors, and self-scales from 1 to 3 has already climbed orders of magnitude. They easily reach 20,000 or more that way. Do this across clusters and soon you have a more resilient network.

I’m not saying mentoring only happens in person, but the people who can pull it off remotely are still the exception, not the rule. We hire folks with little experience, or fresh out of college, and mentoring that person at a distance, while not impossible, is tricky. In the end, it always comes down to the person.

So is an individual working remotely wrong? Of course not. That’s another problem with this kind of argument: it makes it look like there are only two paths. As with everything in sociology, there are many scenarios for many configurations. What the evidence flags as the most inefficient are precisely the two extremes, fully centralized and fully decentralized. In the range between them lives a whole series of different opportunities.

Conclusion

So yes, there’s nothing wrong with what’s in Remote. The mistake is treating it as the only solution. I could hire isolated home-office individuals with no problem at all. It’s just that I still can’t picture how to make them grow, expose them to more than coding, and I could hardly delegate responsibility beyond the code. The great advantage of working in the services market is having access to more companies and more situations than any employee of a single company or a single product ever would. Remote and isolated, very few of those opportunities are left.

And again, there are exceptions: big names in the open source world who work as freelancers, alone, doing great, their reputation only climbing. I’ll fire back with a question: is that the rule or the exception? Exceptions exist, and if you can pull it off, being the exception is better. But evangelizing the exception’s path all the time is deeply irresponsible.

Working from home is an experience worth trying. It is by no means automatically better than working inside a group, even counting the “wasted time” on the commute, which, by the way, is a lousy excuse. A lot depends on whether your region has options. If they’re scarce, then remote becomes a good alternative. Just keep in mind there are always pros and cons.

The distinction matters. A junior professional, who still has technical deficiencies and wants to grow, learns techniques, practices, and tools, and gains technical quality, productivity, and efficiency. A bad professional, who may even have some technical capacity, is more interested in little work, little effort, and little responsibility, and turns bad-character in conduct and attitude. That one doesn’t change.

Agile is another formula whose image (result) is the promise of hyper-productivity that can reach 500% of the current. Again, what is the Domain (the premises)?

The most basic premise for Agile is that the team has technical capacity and commitment. No methodology teaches or changes that. Either each individual on the team is a capable person or it simply won’t work. Well-made, well-organized code, following the known good practices, with tests and discipline, is the minimum to start, the rice and beans.

If that doesn’t exist, give up. No methodology, no voodoo, no exorcism, no feng shui will help. Dropping Scrum, XP practices, Kanban, or anything else into a bad team with weak leadership produces some of the following prostitutions of the good practices:

Planning for Show

Good planning is only possible if the Product Owner commits to keeping a prioritized, well-defined backlog. And understand this: the definition of the User Stories should be ready before the Sprint Planning. A vague, poorly defined User Story helps nothing at all.

While the team works on the Sprint, the PO defines the upcoming User Stories, and can pull in one or more members of the team itself or of parallel teams to help. By the Sprint Planning everything should already be well defined, ideally pre-pointed and prioritized. The team just points the Sprint and, according to the current Velocity, sees what fits in the next one. A well-run meeting lasts 1 or 2 hours at most.

Symptoms: if it takes more than 2 hours, has too many people just to make quorum, poorly defined User Stories that spawn endless discussions and no action, you’re doing it wrong.

Pair Programming for Show

Pair Programming is a very good practice, and the team decides how to run it. Except you can end up with a member who almost never programs well alone and, when doing the “pair,” turns into more of a listener than a participant. That’s the famous “dead weight.” Sits there tossing out opinions, or just sits in silence playing Candy Crush and browsing Facebook. No value at all.

Symptoms: this happens more easily on very large teams (above 6), since it becomes easier to “hide” doing pair-fake with different people. In fact, I never recommend more than 4 people on an agile team. Even for a single product, if there are 15 people, for example, better to split into 3 teams of 5. Better still if people rotate across those teams so that “cliques” don’t form where one bad guy covers for another.

Departmental Protectionism

Professional programmers today need to be multi-functional, versatile. Obviously each one is stronger at some type of capability than another. For example, what we’d call a “front-end” should be the master of HTML 5, CSS 3, JavaScript, but should also know the minimum of Rails, Node.js, Python, or whatever the back-end is, enough to organize the templates and integrate with controllers, models, helpers, etc. And the “back-ends” should also minimally know HTML, and even infrastructure.

Pointing User Stories (whether in Story Points, Working Hours, etc.) should always be the same no matter who pulls the User Story. There’s none of: “these 3 stories are mine, these 2 are so-and-so’s, these 4 are what’s-his-name’s.” Stories are prioritized and each person pulls one story at a time as they finish the previous one.

Symptoms: stories pointed differently depending on who’s going to pick them up, or “reserved” stories, or types of stories only one “type” of professional can pick up. There’s none of “this story is infra, only an infra guy should get it, and the back guy shouldn’t even be looking at it.” Sprints are the responsibility of the entire team.

One of the advantages of pair programming and small teams is that each person learns new capabilities from the others. A back-end dev can pull a Story with something of front-end in it, without being the best at front-end, and nothing stops them from asking to pair with a good front-end on the team when they need it. A User Story may take 2 hours if done by a good back-end, but it may take 5 hours if done by a front-end.

The right thing is for the front-end guy to get close to those 2 hours, not to reserve the story. The right thing is for the team’s velocity to climb gradually, and for that the individual velocity of each member also has to climb. And for that they have to keep learning new capabilities. That’s how everyone evolves. Reserving stories is the best example of a punch-the-clock worker wanting to ‘protect’ their job and hide their mediocrity.

Quality for Show

Every member of the team should care about both the technical quality of the code and the functional quality, that is, whether the feature actually behaves correctly with end users. Many times teams allocate people with the sole goal of being a “Q&A” (Quality Assurance), responsible for building test cases, automating integration tests, aligning with the PO and with business analysts, marketing, etc.

I’ll say that personally I don’t like this distinction, but I confess that in some cases it makes a lot of sense. Except it has to be done in a way that doesn’t strip the team of the responsibility to deliver features that actually work. There can’t be a feeling of “who cares if there are bugs, the Q&A will catch them later.” But that’s what often happens.

Symptom: look for User Stories that get delivered, then rejected, then redone, then rejected again, and stay that way for a long time. A User Story that goes 1 month without being accepted is a monstrous absurdity, an abomination. Even worse if there are team members who, when you look at the past backlog, are always involved in stories that always get rejected and always take days to be accepted. It’s the typical case of the professional who keeps their job by solving problems they created themselves. Remove those rotten apples from the team as soon as possible, because they contaminate the whole barrel very fast.

Excuses, Excuses, Excuses

“I couldn’t do it because we were missing a test, but the guy who does tests was overloaded so I got stuck.”

“I couldn’t do it because there was an obstacle in the other team that they didn’t resolve, so I got stuck.”

“I couldn’t do it because it was poorly specified, I sent an email to clear it up, no one answered, so I got stuck.”

A good professional gives solutions, doesn’t point fingers or keep looking for excuses to justify their own incompetence. Does that mean nobody can complain? Of course not. There are always hard problems to solve, and people can and should seek help when needed.

Symptoms: it’s easy to spot the professional procrastinator. The guy is never at his desk, spends more time chatting at the coffee machine than working. Shows up late every day, leaves early every day. When he’s at his desk, you glance over and the poor soul is on YouTube or on Hacker News.

And let it be very clear: none of these activities, on its own, is a problem. Even I take a break and hang out on Facebook. It’s just that whoever behaves this way constantly, routinely, is clowning around. Worse: if by the end of the day his task were finished, with well-made code and no bugs, I wouldn’t complain. Usually it isn’t. The task comes in incomplete, the code poorly made, always needing to be redone, and the guy shows up with excuses like the ones above.

That’s being bad-character, that’s a criminal’s attitude. He’s deliberately stealing from the company: his salary is being paid and the value isn’t being delivered. He’s a thief.

A good professional, who works seriously, doesn’t wait to be pressed to say “ah, I asked, no one answered, so I didn’t do it.” When they hit an obstacle, they chase the problem. Technical question, they ask the colleagues next to them (in person, email, chat, gtalk). Functional problem, they go straight to the PO. Departmental problem, they get up from the chair and walk over to the other team to clear it. If the task depends on a third party who wouldn’t collaborate, they go straight to the boss to solve it.

The basic symptom of the bad professional is the opposite: he doesn’t go after it. He’s happy when he hits the obstacle, because a week from now, only at the next Review meeting, he’ll say “daaaamn, I even tried, but so-and-so didn’t collaborate, so I couldn’t.” Give me a break.

Conclusion

If your team has just 1 of the problems described above, it’s already a catastrophe. If it has more than one, it’s a military-intervention problem, Iraq level. Anything past that, only a nuclear bomb solves it. And worse still, someone who exhibits all the bad behaviors described above is a criminal sociopath. Scolding is useless, trying to fix them is useless: there’s no fixing it… but there’s a solution: swap out the rotten apples as soon as possible.

And again, to be clear, obviously there are all kinds of bad professionals, and I’m not talking only about programmers. There are bad managers, bad analysts, bad coordinators, bad supervisors, bad directors, etc. This applies to everyone.

If you’re a programmer and you see all this, chase after it. If the problem is in upper management, I’m sorry to say there’s nothing to do. Now, if you’re a manager, a supervisor, someone in a position of authority, and you’re taking no action knowing all this: resign, you’re doing it very wrong and being complicit with an environment whose culture is bad character.

Applying ‘some’ Agile practices, interpreted the wrong way, isn’t being Agile, it’s fooling yourself. It’s like a nutritional diet. During the day you follow the diet to the letter, but when no one is looking you eat chocolate and sweets, and the next day you say “damn, I don’t know why this diet isn’t working, I’m following it right, I guess the diet’s no good, let’s try another one.” #lamentable

Let’s make this even: you do #noEstimates if, and only if, you’re willing to do #noSalary. Your employer holds your pay until you deliver, and in that scenario the amount gets depreciated the longer you take. Labor law doesn’t allow that, unfortunately. But it would be an interesting scenario.

There are a lot of people talking about #noEstimates lately. I’ve read most of the arguments in its favor, and anyone can run a quick Google search, so I won’t keep referencing each of them. The gist is this: estimates will never be good enough, and the more specification and planning you do, the less the quality of the estimates seems to improve. In a dynamic market the specs change all the time, and the bigger the estimation effort, the bigger the waste. If estimation is that much of a waste, why not throw it all out at once?

It feels like a noble idea, especially for software developers. Software is malleable, it’s abstract, and it just seems not to fit the traditional notions of project management. And while we’re at it, why not throw out the entire notion of projects too? That’s how another trend came along, #noProjects.

My intention here isn’t to answer every argument, and that isn’t even the point. What I’ll do is explain why the whole idea is absurd right at the root. So let’s start with the basics.

One thing I’ve argued since at least 2008 is looking at project management and markets in general through the models of Complex Adaptive Systems, Chaos Theory, and Evolutionary Biology. I’ve been heavily influenced by the ideas of Nassim Nicholas Taleb and his magnum opus “The Black Swan”. The idea is incredible: markets are driven by chaotic agents that influence a complex system, far from any linear path.

Every company is managed to deal with averages, with limited sigmas as the margin of operational error. But once something big happens, a “Black Swan” like the 2008 economic crisis, most aren’t prepared. No model can predict it, and the whole system locks up and collapses.

If you don’t know the idea, Google it for a second and you’ll see that companies, markets, and human relationships in general are dynamic systems that obey the rules of evolutionary biology. Decentralized systems seem to be the way to go, and those concepts influence many of the Lean movements we see today. So, yes, I’m well aware of these effects.

To sum it up, the ones with the most chance of surviving in a complex system like that are the most adaptable, far more than the ones who stick rigidly to plans. Following long-term plans with rigidity is the easiest way to fall down when the Black Swans show up.

That led me to another concept: Einstein’s General Theory of Relativity. In modern cosmology it superseded Newton’s Theory. When I first learned that, my first thought was: if Newton is “wrong”, why aren’t we using Relativity to calculate everything in our daily lives?

The answer is that Newton is only “wrong” if you demand that it calculate anything. It doesn’t apply to the very very large, to gravitational calculation, to galaxy-level calculations. But at Earth-like sizes, computing the path of an airplane or the trajectory of a bullet, it still holds, because the margins of error are negligible. In day-to-day life you can reduce the problem to Newton and leave General Relativity aside. It’s a crude simplification, of course, but bear with me.

The same applies to companies. We’re all subject to Power Law distributions, to Evolutionary Biology, to the relentless forces of Chaos that make everything behave like Complex Adaptive Systems. But in a constrained environment I’ll argue that you can reduce the calculation back to Bell Curves. That’s the hardest part to “prove”, so I won’t try right now, but the explanation that follows may get you there.

Let’s define what a company is: a set of operations. Operations are repetitive activities, like “pay a supplier”, “send a purchase order”, “process the payroll”, or “transport products”. The set of all those activities defines what a company is.

The core idea of a company is to run those operations as efficiently as possible. You do that by continuously refining the process in small steps, or through a breakthrough that completely changes the way you run a particular operation.

An example: back in the day there were entire groups of people dedicated to filling out paper forms and organizing them so information could flow inside the company. With the rise of digital systems, of ERPs, all that paperwork stopped being necessary. We got rid of an entire profession of typists and added efficiency and precision to the system. Breakthroughs are usually the digital automation of manual labor, or getting rid of a process entirely.

To reach those breakthroughs there are Projects. Projects are temporary endeavors where a group of people concentrate to hit a pre-established goal. They usually have a fixed start and due date, a fixed budget, and a fixed number of people involved.

And here we get to the Estimation part. Every project wants to hit some goal, and in the case of a software project we write code to get there. To do that, we come up with features and break them into Use Cases, User Stories, Requirements, or any artifact that describes what has to be built. Then we estimate how much resource (time, money, people) it takes to implement each piece and integrate them into a “solution” that solves the problem.

What developers complain about is that you can’t estimate those pieces with precision, and that projects will always come out late and over budget. Hence the idea of estimating nothing, just starting to code, delivering value as fast as possible, and calling it done only when it’s done.

Some developers get so fed up with this whole notion that they want to leave their companies and start their own startups, where they’ll get to do whatever they want with no controls at all. Then off they go looking for investors, because they need a lot of money and a lot of time. Of course they do. What they don’t realize is that EVERYBODY needs a lot of money and a lot of time.

And investors know this: the idea is irrelevant, execution is everything. The ones who deserve more money and more time are exactly the ones who can pressure themselves into delivering under budget and ahead of everyone else. Just doing something without worrying about constraints is the realm of the mediocre. And the mediocre deserve nothing.

Back to basics: there’s a thing called Economy precisely because resources aren’t infinite. Everything that has value has a price, whether it’s a physical product or working hours.

Understand this: in the services business, where we developers all live, whether as an employee or a co-founder, value has only two variables, quality and efficiency. We tend to see quality as the only thing that matters. Worse: we tend to see what we think is quality as the only thing that matters.

This brings us back to CONTEXT. Most programmers are bad at estimates, and the root of it is that they’re usually utterly incompetent at understanding context. As someone with a Math background, I read all those articles about processes, methodologies, and things like “#noEstimates” as if they were “formulas”.

Formulas alone don’t mean a thing. Any mathematician knows you have to define a Domain and an Image, the source and the destination of every input and output. If I show you a formula like “f(x) = 1 / x”, you can say it’s invalid, because you can’t divide when x is zero. But if I tell you the Domain is every Natural number except zero, now it’s a perfectly valid formula for the Image of the Rationals.

So when someone says “#noEstimates”, the natural question is: in which Domain and for what Image? That’s the origin of the confusion in most Internet arguments. People argue for or against an idea because each one is in a different Domain. Same goes for Agile in general, Lean Startups, Lean Manufacturing, and so on: they tend to define only practices, procedures, formulas, but they rarely define Domain and Image. That creates confusion and misses the point.

What I’ll define is this: Projects are necessary every time there’s a defined goal to hit. That’s the domain. And I’ll also state that whoever says “#noEstimates” is thinking about the Domain of “Ongoing Operations”, while Projects are a separate Domain. That’s exactly where Lean Manufacturing in general fits too.

I already explained Operations above, and it’s where the small improvement steps (Kaizen) emerge. Sometimes the feedback from an operation gives enough input to justify a Project and take a bigger step, a breakthrough.

Projects, on the other hand, remain temporary endeavors. The whole idea is to set boundaries, like time and cost constraints. And we come back to the original question: is it impossible to predict the effort needed for something as malleable as software development?

First of all, yes, it’s impossible to predict with exact precision. Let’s define it again: it’s impossible to predict a number with zero margin of error. Estimation is prediction with a margin of error. And why do some projects cost more than twice as much and take more than twice as long as the estimate?

Most of the time, in 90% of cases, because the team is incompetent. The problem isn’t the estimate, it’s the execution. Estimation is the establishment of an expectation, and expectations have to be managed. An estimate is only good if the context is taken into account.

Most people don’t like to estimate because of the “what if” scenarios. What if the client changes their mind? What if we hit a hard obstacle? What if a meteor strikes the Earth and all living creatures perish? “What if” can’t be managed; what can be managed is what you know, by creating constraints. A project’s constraints start with its goal. To hit that goal, we also set the rules of engagement, the premises. Without goals and without premises, there’s no game.

None of that guarantees a Prediction. An estimate is only as good as the execution. Now we have to manage it, and everybody has to manage it.

It’s no use setting clear rules and then suddenly seeing a programmer sitting idle. You ask why he’s idle, and the answer is “oh, because I emailed the client about some requirements and he never replied, so I was waiting”. You ask “and did you try calling him?”, and the answer is usually “No, I didn’t”. No amount of process or methodology “fixes” an incompetent employee. Lack of technical skill can be fixed. Bad faith cannot.

The #noEstimates advocates can argue they’re not like that, and I believe them. But 90% of the projects that failed had employees like that. Programmers tend to blame the client, the bosses, the market, but never themselves. And as a programmer I’ll argue that most projects fail because of lazy employees, far more than because of changing requirements or limited time. Want to make projects go right? Start with Human Resources, then go find methodologies and practices.

The trouble with methodologies that don’t state the Domain their formulas work in is that most people don’t see where that Domain begins: at “having competent, committed, and skilled employees”. Most adopt methodologies hoping to turn incompetent employees into competent ones, and that just doesn’t happen.

With that settled, why do we need estimates? Or, more generally, why do we need constraints? Because that’s the essence of the value of any system. Nature puts pressure on every living species: changing climate, limited food, predators. The most adaptable species evolve, and the ones that can’t adapt perish.

When someone says “it’s impossible to do X”, that’s probably the most valuable goal to chase. Because the full sentence is “it’s impossible to do X with what we know today”. It was impossible to go around the globe in 24 hours, now it isn’t. It was impossible to talk to the other side of the world in real time, now it isn’t.

Constraints are the foundation of innovation. If I had to define innovation, I’d say it’s “the process by which you accomplish something that was deemed impossible before”.

If you have infinite resources, or you simply don’t need to worry about constraints, which is what happens in bubbles, you don’t innovate. This is so important that I’ll repeat it:

Innovation is the byproduct of Constraints.

We estimate based on past knowledge. If we can’t outdo our own past selves, how incompetent are we? You can argue that software isn’t predictable, and I agree. But being off by orders of magnitude when recreating similar software just smells of incompetence to me.

Most of the software we produce is the same old thing: content websites, ecommerce, elearning, social networks, social commerce, forums, polls. It’s rarely a brand-new idea or a breakthrough algorithm. Unless you work in research programs, what kind of truly different software have you built lately?

Estimation is rarely the problem. Every estimate starts from a set of premises, and the problem is not managing those premises. If the requirements change, fine, that we can always manage. What we can’t manage is the pile of problems on the last day of the project, after the programmers kept pushing everything under the rug. Again, this is a Human Resources problem.

Meritocracy only exists in a system of scarcity, where one stands out against another in the face of a constraint. In a system of pure abundance, there’s no need to innovate or to have merit. Companies that just got a humongous pile of cash invariably show symptoms of laziness, far more than of innovation.

The confusion comes from some of the “practices” being pushed coming out of this temporary, unreal bubble situation, and they can’t take the pressure of time. Give it enough time and you’ll ask yourself: “how did we, with so much money and so much time, get so little done, and that little startup, with such limited resources, managed to outdo us?”. That’s the answer to why a Yahoo! buys a Tumblr, why a Facebook buys an Instagram, why a Google buys a Waze.

If you advocate #noEstimation, why not go one step further and advocate #noWork? It’s just one more step.

I’m a software engineer and, after twenty years across different companies, markets, and teams, I see a pattern. Practically every engineer I’ve met starts with the premise that “code” is the goal and the solution to every problem.

The truth is that, in the market at large, software problems aren’t solved with software. I’m not talking about the exceptional cases of research and development or academia. This is the first and hardest thing every engineer struggles to understand, and fights against.

Linda Vasquez: I know he made you a promise, but circumstances have changed.

Francis Underwood: The nature of promises, Linda, is that they remain immune to changing circumstances.

– House of Cards

One thing a lot of people cite as the hardest is “estimating,” because you so rarely land on the right estimate. And that’s where the problem lives: the phrase itself is confusing.

Estimates, by definition, will never be “correct.” If they were, we’d call them “predictions,” and they’re two separate words for exactly that reason. In a prediction the variables are known and fixed, the path is predictable and automatic. That’s the lab scenario: “given ideal conditions of temperature and pressure, in a vacuum, with zero friction, a toy car at a constant speed of 10 meters per second, for 10 seconds, travels 100 meters.” And estimates?

Outside the lab, we have estimates, and estimates are PROMISES.

Promises are made to be kept. A promise doesn’t resolve itself. You have to make a deliberate effort to reach it and keep your word. With estimates it’s the same: you estimate and then you work hard to deliver what you promised.

“But the manager/client/investor/boss keeps pressuring me to add more things, make changes all the time, breaking my concentration with irrelevant problems all day long.”

Yes, they do this, and they’ll keep doing it, yesterday, today, and always. The most common reaction is to get defensive. To avoid committing, people decide to give no estimate at all, or throw out a number so out of proportion that any “extra” is sure to fit. That’s a terrible default.

Say you promise your daughter you’ll be at her school presentation. You blow it and show up two hours after the whole thing is over. That failure is yours, and yours alone.

It wasn’t traffic, it wasn’t the surprise meetings, it wasn’t the weather. You made the promise and didn’t prepare for the bumps along the road: leaving earlier, getting ahead of the problems, building in slack for the unexpected. You left everything for the last hour, walked out at the last minute, and, of course, shit happens.

With estimates it’s the same. You said a number with no commitment, no sense of responsibility, and did nothing to get there beyond sitting down and writing code. You never went digging to explain the problems.

Saying it once, your way, doesn’t mean the other end understood. Whoever starts the communication is responsible for finding a way to get the message across; without that, communication didn’t happen. To communicate is to be understood, and being understood is the responsibility of the person talking.

So you failed at communicating, at managing your own time, at helping your peers with the problems. All of it was your responsibility. Writing code was the smallest part.

Having a sense of responsibility is exactly the second hardest thing every engineer faces. If the starting premise is that they only answer for “writing code,” they don’t see the broken communication as their own failure. So they rarely own the outcome: it’s much easier to blame everything else. And that comes straight from the first difficulty, the wrong premise that your only goal in life is to write code and that every problem is solved with code.

This doesn’t apply to software alone. It applies to the musician, the cook, the builder. As a working professional, sure, you’re expected to be great at your craft. But that’s the bare minimum of the foundation, and there’s nothing exceptional about being technically good, however impressive the skill may look.

There’s no point writing the most elegant, perfect code if it solves the wrong problem. It’s like composing the most perfect Beethoven-style classical piece for a job that called for music at a kids’ birthday party. So one of two things: either you don’t take the kids’ party, or you write the best pop tune you can. If you took it and stayed, either you deliver with exceptional quality, within expectations, or you own the fact that you can’t and step aside, without getting in the way of whoever can solve what you couldn’t.

Anyone offering a service needs to understand one thing: the goal is to implement the best possible solution, but for the right set of problems. And the hardest part is finding that right set. Clients come to professionals precisely because they don’t know either. It’s on us to help find those problems, to judge whether keeping the promise is within our reach, and then to keep it, managing every obstacle that shows up along the way.

It sounds simple put like this, but even experienced engineers fail to grasp this truth, and plenty get away with breaking promises using tricks to hide the hole they dug.

So learn the first and hardest truth in the world: software is normally not solved with software, it’s solved with human capabilities. On a Pareto distribution, I’d say 80% of every software problem only gets solved when you invest the remaining 20% in communication, articulation, clear and rational thinking, breaking ambiguities, negotiation, and commitment.

Given all that, it can look easier to never promise anything. Except that never promising means never committing, and never committing means never being a professional. That’s exactly what separates the amateur or hobbyist from the professional: making promises and working to keep them. It also happens that you don’t hold up your side because it depended on the client, the boss, or whoever holding up their side first. A mutual exchange of promises is the whole reason there’s a referee called Justice, and the instrument that records those two promises has a name: a contract.

All that said, can you keep every promise? No, unfortunately you can’t. But it’s our responsibility as professionals to always chase that ideal, instead of inventing ways to dodge it.

I’ve said more about what I consider a professional to be in a 2011 post, "[Off-Topic] Opinions, Truths, Democracy, and Ethics".

There’s a branch of research known as “Behavioral Economics,” a strand different from traditional Economics. Classical Economics starts from the premise that people are rational and decide based on cost-benefit analysis. Behavioral Economics factors in the social, cognitive, and emotional forces that shape our day-to-day decisions.

We like to believe we make rational decisions all the time, but most of the time we’re guessing and deciding on emotion. And that produces some fascinating results. One of the best-known researchers in the field is Prof. Dan Ariely, author of the book “Predictably Irrational.”

In the book he looks at how we see prices, why price isn’t a simple matter of “supply and demand,” why we’re dishonest, and how the placebo effect works. It’s the result of years of empirical experimentation.

Watch this excerpt I subtitled from his TED talk on the subject:

Why does this matter? Because we deal with people every day. We make decisions influenced by others or, worse, that will affect others. And we sleep with a clear conscience, convinced we picked the best rational option with the data we had on hand.

But what if our decisions aren’t that rational after all? What if we’re easily influenced by data and circumstances? Better yet: what if recognizing this fact helps us understand our limitations and actually decide better?

That’s why it’s worth understanding your own psychology. Anyone in a management position needs it more than most, because the job itself is already “managing people.”

We are irrational, predictably irrational. Use it to your advantage.

I got back last weekend from a trip to Baltimore, Maryland, where I spoke at RailsConf. It was an event full of great talks, some aimed not just at Ruby on Rails but at the programming career in general. One of them was by no one less than Robert Martin, of Object Mentor. I translated an article of his in my previous post.

I recommend watching the talk in full.

Since I started this blog, I’ve tried to explain the conditions and difficulties a Project Manager faces. But I’ve been careful not to repeat the term “Agile” at every turn. We’re in the middle of an “agility” fever. Every CIO, CTO, manager, or consultant, to seem modern, says they know “Scrum” and that they’re “agile”.

That has a good side: some companies might finally adopt these ideas the right way. It also has a bad side, and a big one. We’re burning the name “Agile” because of bad consultancies that implement everything wrong at the client. The result is the perception, among the uninformed, that “Agile doesn’t work”.

Well, during RailsConf, I had the chance to talk to and interview Robert Martin, whose recording you can watch below:

From this interview, the part I want to present is the story of how “Agile” came about. Robert Martin, also known as “Uncle Bob”, has been a programmer for decades, certainly more than any of us. In the 90s he was one of those who noticed the rise of “light” or “lean” methodologies. Much of that came from the manufacturing concepts that revolutionized Japanese industry from the 60s onward.

Several big names started proposing smarter ways to develop software. Kent Beck with Extreme Programming (XP); Ken Schwaber and Jeff Sutherland with Scrum; Alistair Cockburn with Crystal.

They all followed a similar line: short iterations, constant feedback, and a lot of communication. Seeing that pattern, Uncle Bob reached out to Martin Fowler, another strong name in the world of software architecture, and together they began organizing a meeting, summoning a group of these professionals.

The legendary meeting took place at the Snowbird ski resort in Utah, between February 11 and 13, 2001. Seventeen professionals spent three days there and walked out with the “Agile Manifesto”. It brings together 4 values and 12 principles that all the participants accepted as the minimum common denominator in the practice of software development. The manifesto says:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

As I said before, there’s a growing fever in the adoption of agile methodologies. Worse than that, there’s a tendency to expropriate the term “Agile” and its derivatives, like “XP” and “Scrum”, creating real frankensteins that blend the old way with the new. It’s the case of trying to join RUP with Scrum, or PMI with Scrum, and so on.

These are opportunistic attempts riding the fever to make easy money. Some companies, for example, insist on a “Certified Scrum Master” in a manager’s resume, among other equally irrelevant things.

And, to make it worse, many try to implement agile methodologies “by the book”, that is, dogmatically. Every dogma, by definition, is dumb.

Back to the 4 values: the priority is working software and value to the client. Implementing a methodology comes after that, if it comes at all. The first value is clear: it values Individuals and Interactions much more than Processes and Tools. What we see today is the opposite, an effort to push processes, methodologies, and tools.

Now is a good time to start demystifying the Agile world. This here is just an introduction, but the advice Uncle Bob gives in the interview holds: follow the originals. If you’re really committed to improving things and delivering projects with quality and value, drop this new generation of consultancies. Start with the first books by Ken Schwaber, Kent Beck, Alistair Cockburn, and Mike Cohn.

Ask yourself all the time: am I in agreement with the 4 values?

If you’re putting more emphasis on the process than on people, you’ve already started wrong. Very wrong.

The human brain is a fantastic machine, and saying that is basically a redundancy of the term. But there are many anecdotes and mysticism when people talk about it.

Everyone has at some point discussed the separation of “Emotion” from “Reasoning.” Since the time of Plato, emotions have been seen as something that gets in the way of logical thinking. Worse: most people believe that, at least in the most important decisions, we’re capable of setting emotions aside. The idea is to think according to classical economics, based on facts, evidence, and cost-benefit.

But in the vast majority of cases, that’s not how we behave. When we go to the supermarket, we rarely stand there doing arithmetic, weighing each product’s ingredients or nutritional value. Usually we look at a product and decide whether to buy it based on how we feel.

Among dozens of brands for the same type of product, we choose based on past experiences, on some memorable event, on memories, and all in fractions of a second. We prefer to buy an “80% light” product over a “20% fat” one, even though both phrases say the same thing.

Antonio Damasio is a well-known neurologist who studies the brain and our behavior. He followed patients who literally lost the ability to have emotions because of brain damage, whether from a tumor or a genetic problem. We might imagine, as the philosopher Plato did, that a person without emotions would make the best decisions all the time, since they wouldn’t be influenced by them.

But without emotions, these people can’t learn. Any trivial decision becomes a cost-benefit analysis. Everyday choices like what clothes to wear, what TV channel to watch, tea or coffee, all the mundane things we decide almost instantly, can take hours before this type of patient reaches a conclusion.

A fact we already know: all emotions, sadness, compassion, joy, depression, pleasure, are chemical reactions in the brain. And we already know how to influence or control many of them.

Drug users know the sensation of pleasure. People who take antidepressants do too. They’re all chemicals that act on the brain and modify our emotions. Schizophrenia and Parkinson’s, on the other hand, are damage to this chemical system.

Another fact is that our brain evolved to learn. We learn through trial and error. We learn by observing and inferring patterns. Emotions strongly influence our decisions, and that’s good.

When we learn some pattern, dopamine is released in the brain and creates a sensation of pleasure. We like to discover the pattern of things and to try to predict future events.

Take an experiment with monkeys: we ring a bell and then give a banana. The monkey learns the pattern “after the bell comes the banana.” You can measure dopamine being released the moment the bell rings, because the monkey starts to anticipate that event.

If we turn on a light, ring the bell, and then give the banana, the monkey learns that the “light” event precedes the bell and then the banana. The pattern can be repeated with several stages, and the monkey learns it. But if we follow the pattern and the banana doesn’t come at the end, the monkey gets sad, because the prediction failed.

We also work this way.

For many simple events, with few variables, this works well. Perhaps that’s why many like routine, where the results are well defined.

Our brain, however, has a flaw: in its eagerness to find patterns, it tries to fit them where they don’t exist. That’s how superstitions are born. Before we understood meteorology, primitive tribes thought “it’s raining” was tied to a dance they had just done, and imagined that repeating the dance would bring the rain back.

I translated on my other blog a Scientific American article showing how we are “mathematically ignorant.” We try to find a pattern for everything. A passage that illustrates our superstitions and explanations about coincidence is this:

It’s always possible to combine random data and find some regularity. A very well-known example is the comparison of coincidences in the lives of Abraham Lincoln and John Kennedy, two presidents with 7 letters in their last names, and elected 100 years apart, 1860 and 1960. Both were assassinated on a Friday in the presence of their wives, Lincoln at the Ford Theater and Kennedy in an automobile made by Ford. Both assassins have 3 names: John Wilkes Booth and Lee Harvey Oswald, with 15 letters in each full name. Oswald shot Kennedy from a warehouse and ran to a theater, and Booth shot Lincoln in a theater and ran to a kind of warehouse. Both successor vice presidents were Southern Democrats and former senators named Johnson (Andrew and Lyndon), with 13 letters in their names and born 100 years apart, 1808 and 1908.

But if we compare other relevant attributes, we fail to find coincidences. Lincoln and Kennedy were born and died on different dates (day and month) and in different states, and none of the dates are 100 years apart. Their ages were different, as were their wives’ names. Of course, if any of these had matched, they would be on the list of “mysterious” coincidences. For any person with a life reasonably full of events, it’s possible to find coincidences between them. Two people meeting at a party often find shocking coincidences between them, but what they are – birthdays, hometowns, etc. – aren’t predicted in advance.

The real world is full of randomness, and it isn’t hard to find patterns in it. Actually it’s very easy. Complex situations have many random aspects, more than we imagine. And our brain hasn’t evolved enough to feel comfortable in a totally random world.

An example is successful people. Imagine a mega-investor like Warren Buffett. Books upon books get printed, with analyses, biographies, and theories, trying to explain the process that takes an ordinary person to earning billions on the stock exchange. Just as speculation: what if Warren Buffett is nothing more, nothing less, than a statistical accident?

Nassim Taleb explains this in a Malcolm Gladwell article:

Suppose that there were ten thousand investment managers out there — which is not an outlandish number — and that every year, entirely by chance, half of them made money and half of them lost money. And suppose that every year the losers were tossed out, and the game replayed with those who remained. At the end of five years, there would be three hundred and thirteen people who had made money in every one of those years, and after ten years there would be nine people who had made money every single year in a row — all out of pure luck. And who’s to say Buffett isn’t one of those nine?

Our brain takes pleasure in looking for patterns in everything, especially in success stories. We want to find the recipe, the magic formula. Sometimes we get that feeling of “what does he have that I don’t?”

Sometimes two people are technically equally capable, but one gets rich and the other doesn’t. The difference can simply be randomness. “But that isn’t fair!” you complain. The world wasn’t made to be fair. Justice is a human concept, and reality has no sense of morality: it simply “is.”

Science, fortunately, has a procedure to avoid this trap of the brain: the scientific method. Everything can start with one of these misfirings, thinking it found a pattern and generating a theory. Now that theory has to be put to the test.

For it to become a good theory, we need not only to try to prove it but, mainly, to try to disprove it. And even if we don’t find a way to disprove the hypothesis, that doesn’t make it a “Law.” Science is very careful with the claims it makes.

Even if a theory works well a thousand times, a single failure is enough for it to be rejected. It’s thanks to this way of thinking that we have solid and reliable foundations in science, free of superstition and mysticism.

Processes, Methodologies, and Superstitions

I gave that whole explanation to get to the main topic of this blog: project and people management. Anyone following my posts noticed that at no point do I preach a methodology, a process, a cookbook recipe. Things like “how to make correct estimates” or “how to predict the success of a project.”

If you’re well informed, you know there’s a whole market of individuals and companies that try exactly this: to sell you processes and methodologies. Whether traditional Software Engineering, PMI, or the so-called “Agile methodologies.”

First I need to leave a warning: all these methodologies were thought up with the best of intentions. The idea was to share the factors that led a certain project to success.

My explanation is more or less this: the leaders of these projects, or rather, their brains, tirelessly start looking for patterns. Maybe it’s the post-it on the wall. Maybe the kind of document used. Maybe the way people sit at the table.

All of this forms a set of “pseudo-truths” that the brain understands as the “lights” and “bells” that lead to the “banana.” These leaders apply the recipe to other projects and, as Nassim Taleb explained, succeed consecutively.

Eureka! It seems we found the formula for turning iron into gold! Quickly these pseudo-truths get formatted as methodologies, canned, packaged, and sold.

But think for a moment: what if all these methodologies are exactly like the rain dance? As I said, it’s very easy to find patterns in randomness.

When we start discussing minutiae, what level of detail to make an estimate at, what metrics to use, what mechanisms for team communication, I can’t help thinking of primitive tribes discussing “What feathers should we wear? What colors on our face? Should we take long or short steps in the dance?”

But the main question never comes up: is the rain dance the thing that actually makes it rain? Or, in our case, is the methodology the thing that actually takes the project to success?

And the scientific method? The problem is that a “project” is a very complex situation. Even the smallest projects are complex, in the sense that they have more variables than it’s possible to measure.

One way to test a theory is to run a double-blind experiment, like drug labs do: one group receives the real medication and another receives a placebo, without knowing. If both have approximately the same result, the drug doesn’t work.

But a methodology is more complicated than a drug. Even if we create two isolated and identical work environments, put two groups of “approximately” similar people in each, start at the same time and with the same requirements, one side with methodology “A” and the other with “B,” we still have no way to compare the results.

That’s because methodologies involve the micro-decisions and capabilities of each person, and “approximately similar” is very different from “equal.” It’s impossible to duplicate the same group of people in each environment.

We would first have to successfully clone the people. Then, with exactly identical groups, we could run a double-blind experiment. But that’s impossible.

The conclusion is that proving a methodology of this type as “correct” is impossible, just as it’s hard to disprove it. So it’s useless to try to claim they work or not. Of course they influence, and separately, each technique can be tested more carefully. But I don’t think it’s possible to scientifically claim that they work.

“But I tested methodology X on my project and it worked!”

Excellent, good for you. But for every project that succeeded with a certain methodology, you can find a case where it went wrong.

And the individuals and consultancies that sell these methodologies are clever: if the project worked, it was thanks to the methodology; if it went wrong, it’s because the people didn’t follow the methodology as they should have. The methodology is never at fault. It’s excellent rhetoric for building a sales argument.

And since people aren’t used to evaluating things rationally, and as I explained, most of the time they don’t reason at all, we’re left with lots of consultants making a lot of money without having to commit to results.

That matters because many people believe that, with the help of the fashionable methodology, simply applying it to a team of nothing but inexperienced people will make a good result appear, as if what mattered were the process and not the people. The opposite happens.

Most of the time these methodologies don’t describe the exact qualifications each member needs to have, because it’s impossible to list them all. At most they offer vague descriptions of “roles,” which generate more discussion than answers. A project with a chance of succeeding starts with at least some very experienced people, capable of guiding the rest of the team.

Anyway, my recommendation: study the so-called “methodologies,” because, as I said, some techniques there have value. But understand that none of them represents the truth.

And there’s no point in adding two half-methodologies together thinking it makes them “more complete.” Projects are executed by people, human behavior is hard to predict, and the real world is full of unpredictable variables, forming a complex system with a lot of randomness involved.

Start by accepting that the world is random and beyond your control. In complex situations it’s better to err sooner than later, because errors bring information that helps refine the next decisions.

Accept that predicting the future in a complex system is impossible, and adapt: some things will go as planned and others will need to change. Maybe some unforeseen feature will need to be built, and the deadline will have to stretch. Maybe some planned feature wasn’t so necessary after all and can be dropped. That’s how you deal with a complex and unpredictable environment: adapting.

Biology explains it: all living beings on the planet are the result of a constant sequence of trial and error. Nature isn’t “perfect,” in the sense that there is no creator, no intelligent design, no grand plan, no destiny. Every living being came from a very bad biological draft that, little by little, via natural selection, discarded what didn’t work and favored what did.

We humans carry biological leftovers, like the appendix. We are a product in constant evolution, and for plenty of things the current stage is “good enough.” In terms of a project, that’s exactly the point where it can be launched: when it’s “good enough,” still with rough edges to fix, but working overall.

Trial and error. Adaptation. Evolution. That’s how you manage anything.

If you want a more detailed view on this topic, I recommend starting with the following books:

• How We Decide by Jonah Lehrer
• Descartes’ Error by Antonio Damasio
• Predictably Irrational by Dan Ariely
• Fooled by Randomness by Nassim Taleb
• The Lucifer Effect by Philip Zimbardo

As incredible as it sounds, there is no good, simple, clear, short definition for what it means to be a “manager.” Manager isn’t a profession. You can’t “learn” to be a manager in a school or a course. The only way to learn is by trying, erring, and improving, continuously and indefinitely.

One of the best definitions I’ve found is from no one less than Professor Henry Mintzberg, in his recent book “Managing,” published in 2009. A mature and light read.

I recommend the read. If you’re a beginner manager, or want to be one, maybe the book gives you some sense of what’s coming. If you’re already a manager, you’ll be surprised. So many theory books paint scenarios you’ve never seen in practice, and they leave you second-guessing whether you’re on the right track. This one does a better job of naming the real problems you deal with day to day.

I want to bring a few excerpts from the first half of the book.

Management Isn’t a Profession

After years of seeking Holy Grails, it is time to recognize that managing is neither a science nor a profession; it is a practice, learned primarily through experience, and rooted in context.

It is certainly not a Science. Science has to do with developing systematic knowledge through research. That’s nowhere near the purpose of management, which has to do with helping get things done in organizations. Management isn’t even an applied science, because that’s still science. Management certainly applies science: managers need to use all the knowledge they can. And they certainly use analysis, rooted in the scientific method (which means here more scientific proof than scientific discovery).

But effective management is more dependent on art and is especially rooted in craft. Art produces “insight” and “vision,” based on intuition (Peter Drucker wrote in 1954 that “the days of the ‘intuitive’ manager are numbered.” Half a century later, we’re still counting.) And craft has to do with learning from experience — working things out as the manager moves forward.

So it’s worth repeating what Mintzberg often says: it’s impossible to form managers in school. That much is a certainty. In school you learn accounting techniques, marketing, repeatable technical knowledge. None of that makes a manager, much less a leader.

When a school or an MBA course slaps “We Form Leaders” on the brochure, it’s lying. Those schools form technicians. Those technicians go work in technical activities until they grow, take on responsibilities, and maybe, one day, actually manage something.

Management Is a Practice

Put together a good portion of craft with the right touch of art along with some use of science and you end up with a job that is, above all, a practice. There is no “one best way” to manage; everything depends on the situation.

It isn’t even a Profession. It has been pointed out that engineering also isn’t a science or an applied science so much as a practice in its own right. But engineering applies a good portion of science, codified and certified as to its efficiency. And therefore it can be called a profession, meaning it can be taught before practice, out of context. In a way, a bridge is a bridge, or at least iron is iron, even if its use has to be adapted to the situation at hand. The same can be said of medicine. But not of management:

Many medical diagnostic, inference, and treatment skills … assume that a disease can be decomposed into separate problems that don’t differ much between patients and can be treated by reasonably standard remedies … In contrast, much of management work involves dealing with problems that are highly interdependent with other parts of the organization, are specific to this particular firm, market and industry and not easily reducible to a standard general syndrome that can be treated by specific techniques.

Managing is a work of negotiation, of trades, of yielding, of conquering. There’s no procedure, no methodology, no rigid body of knowledge that defines this role. You can’t know “what level” a manager is at because there’s no “standard” for comparison. There are good managers and bad managers, just that. “Bad managers,” in reality, couldn’t even be called “managers” because they’re actually not managing.

Real Management

It has been said that an expert is someone who knows more and more about less and less until finally he or she knows everything about nothing. The manager’s problem is the opposite: to know less and less about more and more until finally he or she knows nothing about everything.

Many people think that taking a pile of courses, postgrad, MBAs, and the like will necessarily turn them into good managers. That correlation doesn’t exist. Yes, there are excellent managers with good training, but what really formed them was practice. Correlation isn’t causation. In the general case, this person is just postponing the thing that would actually give them a shot at becoming a good manager: practice.

Managers who don’t do and deal, and so don’t know what is going on, can become incapable of coming up with sensible decisions and robust strategies.

There’s also the idea that managers shouldn’t get their hands dirty, since it’s either they manage or they execute. That doesn’t mean the manager can be ignorant about the area where they work. Quite the opposite: a manager with excellent technical knowledge of their area has a better shot at understanding the context.

Thinking is heavy — thinking too much can exhaust a manager — while acting is light — acting too much can make the manager unable to stop.

Similarly, too much leadership can result in a job without context — without aim, without focus, without action — while too much relationship can produce a job disconnected from its internal roots — public relations instead of tangible connections. The manager who only communicates never finishes anything, while the manager who only “does” ends up doing everything alone. And the manager who only controls risks ends up controlling an empty shell of yes-men and yes-women. We don’t need people-oriented, information-oriented, or action-oriented managers; we need managers who operate on these three planes. Only together, across all three planes, do these roles provide the balance that is essential to the practice of management.

Mintzberg points out something obvious: if you want to be a renowned management author, writing books and giving talks, you have to focus on a single aspect and exalt it as the “missing piece.” But in reality, a manager who focuses on only one aspect of the practice is certainly doing the wrong thing. A good manager balances the different roles as the situation demands, and uses all of them whenever needed.

Anyway, as you can see, the role of the Manager is much broader than it seems. Just studying procedures doesn’t make you a manager, just as studying a color table doesn’t make you a painter, or knowing how to read a score doesn’t make you a composer.

Art, Craft, Technique, and Practice. This is the definition of Management. A manager who doesn’t understand that will always be a terrible manager.

As an addendum, one word of caution. Sometimes we see very active professionals, and not only managers, who seem to do a lot because they talk to a lot of people, act a lot, and are always running hot. That’s usually the symptom of the “hero-professional.” In my definition, it’s precisely the person who is doing the job the wrong way. They look productive, but in practice they’re just being inefficient and, worse, cleaning up the mistakes they generate along the way. It looks like they’re solving plenty when, in fact, they’re just chasing their own tail.

After so much time talking about methodologies, processes, and procedures, there’s one thing that should already be obvious but that most companies ignore: methodologies will never replace good professionals.

If I say this to anyone, everyone will say: “but that’s obvious.” And my urge is to retort: “then why the hell are you trying to implement this methodology?”

First of all, let’s go through some definitions:

• Methodology: a system of methods used in a particular area of study or activity.
• Processes: a series of actions or steps taken to reach a particular goal.
• Procedure: an established or official way of doing something.

Notice that in software development we always talk about processes or methodologies, never procedures. That’s because there’s no established or official way of making software. The problem is that most people confuse processes and methodologies with procedures and try to apply them as if they were the “official” way of making software. And they aren’t.

That’s exactly why disputes like Agile vs PMI make no sense. PMI calls its set of processes PMBOK, where BOK means Body of Knowledge precisely because it isn’t a procedure. It’s just a body of knowledge that may or may not be useful depending on your sector of activity. Agile is no different: a set of methodologies and techniques that may or may not be useful in your company. None of them, and none of the others, are procedures. Understand this first.

Once you understand that, comes the second mistake. People who don’t practice software development notice that the software being produced is of low quality. The indicators are obvious: customer complaints, internal complaints, the sense of excessive delay in completing any software work, and so on.

Seeing this, the first conclusion they reach is: “we’re missing procedures.” And the search for the missing link begins. They find various processes and methodologies and try to institutionalize them. For a short period it seems the result is what they expected, but it doesn’t take long to see that things haven’t changed all that much. Then they blame the methodologies for “not working.”

Software development is an activity of “practice.” And watch out: “practicing” and “being practical” aren’t the same thing. So one more definition:

Practice: 1. the actual application or use of an idea, belief, or method as opposed to theories about its application or use. 2. repeated exercise in or performance of an activity or skill so as to acquire or maintain proficiency in it. (Read a more refined definition on Wikipedia.)

Repeat with me: if you don’t practice software, you don’t understand software. It’s like saying you were good at soccer 10 years ago, but you don’t practice anymore and still want to believe you’re good. If you haven’t practiced for 10 years, you’ve been bad for 10 years, and there’s nothing to discuss.

In software, not practicing continuously, daily, deliberately, for 5 years is the same as a civil engineer saying he hasn’t practiced engineering in 50 years. 5 years in software age is an eternity.

And if you don’t practice software, you have nothing to say about it, by definition. So here’s the tip: processes and methodologies won’t help you, precisely because they aren’t procedures.

If software had a procedure, it wouldn’t be a practice. You’d just mechanically follow the series of steps and, at the end, you’d have quality software. More than that: if it were a procedure, it would already be possible to have software that generates software on its own, with no human help. But that doesn’t exist, and never will.

If a restaurant has bad food, processes won’t help it; changing the cook will. If your orchestra plays badly, processes won’t help it; changing the musicians will. If your soccer team plays badly, processes won’t help it; changing the players will.

Replacing them is perhaps drastic. Before that, I’d say “practicing” and “training” will certainly help. But here’s the thing: if your soccer player believes he only needs to play after he punches in his time card, and can stop when he punches out, he isn’t a good player. So forcing them to train won’t help and, consequently, processes and methodologies definitely won’t help you.

With a good programmer it’s the same. You don’t need to force him to put his code into a versioned repository: he himself will miss it and do something about it. You don’t need to force him to test his code: he himself will miss it and do something about it. You don’t need to force him to refactor: he himself will feel that it’s bad and do something about it.

A good practitioner doesn’t need others to tell them what to do. They know what constitutes good work and will perform accordingly, or they will look for what they’re missing and train until they become proficient at it.

A good practitioner lives by the idea that they have to be better today than they were yesterday, and keep that pace every day. A procedure professional knows that today’s work is the same as yesterday’s.

That’s the difference. If your company manufactures screws, it follows procedures. Today there are people assembling those screws, following procedures; tomorrow you’ll probably replace them with machines that will do the same job better and faster. That’s the destiny of every procedure professional.

If your company is in gastronomy, music, sports, literature, art, or software, you want practitioners. If you don’t have them, remember: processes and methodologies aren’t procedures. Processes and methodologies can’t be installed as procedures. Processes and methodologies will never transform people with a procedure mindset into practitioners. And without practitioners, your result will always be bad.

We’ve been developing software for over 70 years. If the solution were that simple, don’t you think everyone would have done it that way decades ago, with excellent results sustained for decades? Don’t be naïve thinking you’ve found the “missing link”: it doesn’t exist.

The only thing we can do, always, is to try to be better tomorrow than today. But that isn’t done with meetings, committees, procedures, magic recipes, gurus, or talks. It’s done with practice. Want to be better at software? Practice software.

Now, do you have a good team, good professionals who really care about practice? Then yes, refining the processes and applying good techniques and tactics will improve things even more. But the order is this: good professionals first, then good methodologies. Repeating: good methodologies don’t create good professionals, but good professionals will certainly know how to take advantage of good methodologies.

The text below is a translation of the excellent article Standards: excellence vs mediocrity, written by Jason Yip, a Thoughtworks consultant I had the pleasure of meeting in person last year.

Before the translated text, a short introduction. Plenty of concepts we treat as foundational in the Western world carry a radically different meaning in the Eastern world. One of the hardest to carry across is precisely “standard.”

In the Western world, a “standard” is a common denominator: static, rigid, hard to change, the status quo. In the Eastern world, “standard” means “the best.” If tomorrow another “best” shows up, that becomes the new standard. It isn’t something unreachable that we admire from below, knowing we’ll hardly get there, like a “record.”

Imagine a world where the “record” is the “standard.” I wrote about this in another article, Standards, Commodities, and Innovations, and it’s worth the read. Here’s the translation of Jason’s article:

A few years ago, I went on a Lean study tour in Japan. As expected, we did a plant visit at Toyota. Unexpectedly, this visit was hosted by the plant manager who also accompanied us on the standard guided tour. At one point, we were looking at a demonstration area of fundamental skills training – developing the basic skills necessary to be a productive member on the assembly line. Most of this was about hand-eye coordination and motor skills. For example, there’s an exercise where you take a thin rope and wrap it around a set of pegs following a known clockwise or counter-clockwise sequence. I’ve sketched an example of what this might look like:

I was able to do the exercise in 8 seconds. We then watched a video showing the plant champion doing it in 4 seconds. We asked the plant manager what the standard was, expecting the answer to be something like 5 - 6 seconds. His answer? 4 seconds, of course. If someone is able to do 5 - 6 seconds, they’d train them to achieve 4 but slower than that, and they’d probably find the candidate something else to do.

Our basic assumption was that “standard” is supposed to be something easy enough that everyone can do it. Sometimes that “standard” comes with an expectation of training but many times it ends up being about lowest common denominator in order to achieve consistency. Obviously, we can’t achieve consistency if our standards can only be achieved by a subset of people. The consequence of this mindset is that “standards” inevitably pull the organisation toward mediocrity.

The Toyota assumption was that the best should be the standard. There is still an expectation that people can do it (with training) and an expectation of consistency but it’s a standard of excellence rather than lowest common denominator. This creates a much higher expectation of performance and challenge. The consequence of this mindset is that “standards” inevitably pull the organisation toward excellence, especially as people’s capabilities continue to develop over time.

In one case, standards are an anchor; in the other case, standards are an engine.

Is this any different in our world of information technology and software development? I don’t think so. Think about the last standard you encountered (e.g., coding standard, design standard, etc.). Was it designed as an anchor or as an engine? And what would it be like if every standard was designed to be an engine?

Ron Jeffries is one of the 17 original signatories of the Agile Manifesto and one of the 3 founders of Extreme Programming. He publishes articles in the Pragmatic Programmers e-magazine, and the April 2013 issue ran “Estimation: The Best We Can Do.” I asked editor Michael Swaine for permission to translate this article into Portuguese, since it was a subject I meant to write about myself and Ron did a better job of it. I still have plenty to say about estimates, but for now I’ll go straight to the translation:

Two months ago in these pages, Ron Jeffries told us that estimates are evil. Now he’s back to tell us they’re a necessary evil and, done the right way, they aren’t even evil.

Yes, there are many abuses perpetrated by organizations around the notion of estimation. Many developers have suffered grievances from these abuses more than once in their careers.

Unfortunately, this leads to the common naïve view among would-be agile practitioners who are still early in learning: they want to abolish estimates completely. That’s probably not possible, and certainly not ideal. We have the ability to estimate how quickly we can do things, and companies can make better decisions if we share what we know.

Software development isn’t a perfect machine that spits out features quickly. I know it’s comfortable to think we have no responsibility for business concerns like money, time, or dates. But that isn’t true. What we build isn’t solely influenced by some Lord of the Products who decides what we’re going to do and takes on all the risks. Executives are the ones with the final decision, but many of the decisions are better made by the team — and not only in how to do things, but what to do. We have the responsibility to help guide the projects, using our creativity and our special knowledge.

The Agile Manifesto says, “the best architectures, requirements, and designs emerge from self-organizing teams,” and that’s what we meant (my emphasis). Agile developers usually have a good sense of how long things will take, and that’s a valuable component of selecting and refining requirements. Developers need to have the attitude of talking about possible costs of what they’re asked to do.

Concerns About Estimates

There really are several serious problems related to estimating a software job. In a previous article “Estimation is Evil”, I described some of them. Let’s review them:

• Pre-defined work backlogs reduce creativity and inhibit adjusting the project for success.
• Demanding delivery of “everything” on some fixed date is a trick that never works.
• Treating estimates as promises leads to conservative teams and frustrating results.
• Trying to improve the “quality” of the estimates is attention that could give better results if turned toward improving the quality of the product.
• And so on.

In that article, I argued that the result of this focus is “weak Agile.” In essence, teams focused on estimates are usually on the weak side of the balance between cost and value. Better teams focus more on value and less on cost. They still consider cost, because cost is an important component in delivering value. But value is their primary interest.

It’s true that many first-rate teams don’t estimate in the real sense of the term. They work on small stories, small enough that to get a sense of progress you can just count them. They work on the highest-value things first — using whatever notion of value they have. They adjust their vision frequently, and adjust what they’ll work on next according to what is happening. They work to produce a continuous flow of valuable software, and they deliver as frequently as possible, usually daily.

Since estimation is often associated with weak and dysfunctional teams, and since great teams seem to work without estimates, many Agile practitioners strongly resist any estimates of any kind. And it’s a good thing to build a team — and an organization — that is strong enough to work without estimates. But it’s an advanced way of working, never the way to start. It isn’t good to refuse to estimate, or even to feel resentful about it, in an organization where it’s necessary. On the contrary, we need to learn to do it right, in a way that influences the organization positively.

Budgeting New Projects

Now, let’s look at a legitimate need — helping the organization decide whether to proceed or not with a project.

When something goes wrong with our house or car, and we don’t have all the money in the world, we probably get estimates to fix those things. We probably consider alternatives: if the trunk lid is scratched, should we repaint the whole car to match the color perfectly, just spray the lid, try to polish the scratches, or just live with the scratches? We choose the option that seems to be the best use of our money.

Business people budgeting projects have the same problem of allocating funds. They need to consider how to deliver the greatest value possible within the budget. In a small organization trying to create wonderful new products, the concern is even greater. We need to start generating positive cash flow before the money runs out!

In situations like that, we really need to have estimates. We need to know whether it’s a wise decision to go ahead with the new project. Business people already have some estimates in mind: how many people will be interested? What percentage of prospects will become buyers? How much will buyers pay for the product?

But they need to decide: can we start bringing in enough money to stay alive before our money runs out?

The decision-maker knows that all these estimates are estimates, and uses the best judgment to value them. But there’s a key area they don’t know much about: how much of this product can be done, by what date, for how much money? That’s a technical question, and it needs to be answered by technical people.

Yes, of course, we don’t know either. We really don’t know how long it will take. But let’s face the facts — we have a better idea of how long things will take compared to the poor entrepreneur, and they need our help.

I believe we can give a better guess whether it will take a week, a month, a quarter, or a year before we start having a good idea of how quickly we can progress. And this guess, however broad, is a better guess than a business person can give.

We can’t just say “Should we try?” It seems we can say something like:

“Our team costs $10K per week. Let’s start working on the most important, riskiest, most informative parts of your product, and create a sense of how hard it will be and how long things will take. You’ll see us building what you asked for. You’ll see us working. You’ll be able to decide whether you want to keep investing or stop.”

It seems to make sense to say that — unless it’s your $10K coming out of your pocket every week. Then you have more questions. “Will I know after a week? Will I have to wait a whole month to know? Will I know by June? Jeez, guys, that’s about $100K from now until June. That’s a ton of money! Give me a break here! How much is this going to cost?”

We need to take hold of this kind of challenge. We should give the most solid answer we can, even when we don’t have an absolutely concrete answer.

We’re walking on sand. How do we get concrete?

The question is, “How long will it take, and how much money will it cost, to get to a product people can buy?”

We’d like to say, “Well, start spending some money with us, and if you get bored, stop,” but that doesn’t help at all. Can we change this idea a little? Let’s start with a pretty broad estimate just to get a sense of where we are.

We probably already have some notion of whether it will take a week, a month, or six months to do something usable. If so, let’s talk about it. Our investor already has a value in mind that they can spend. Let’s find out how much that is, or get a sense of it in the conversation.

Suppose we imagine that we might get a rudimentary but usable version of the product in six months, and the investor is willing to invest up to $1 million until we achieve positive cash flow. $1 million is roughly a year of our time, and if we take 9 months instead of 6, we’ll need to bring in $1 million in three months to reach positive. Let’s talk about that. How quickly does he expect revenue to grow with the first version? Suppose he thinks revenue will grow quickly, so if we take 9 months, we’re fine. That sounds risky to me, but it seems the project at least looks good enough to explore.

Remember how Agile works. We want business people to be fully engaged in decision-making about spending money based on what they see. So let’s start thinking about selecting features, not just dates.

Estimates That Won’t Make You Regret

“OK, intuitively, we think we can deliver an initial version in six months, and we think that if we take nine months, the project would be viable. It sounds good, but it still sounds risky. We’d hate to see you spend all this and get nowhere, and what we have is still rough guesses.”

“So let’s do this: let’s spend two weeks to produce a more concrete answer. Two weeks will cost you only $20K, which is much less than $1 million. In these two weeks, you’ll actively work with us on the most important aspects of the product. Some will be interface things, some will be driven by business or technical risks we can predict now. At the end of these two weeks, we decide whether we want to continue. Our job will be to show you, concretely, what we can build that convinces us of whether to proceed or not. Your job will be to guide us by describing what you want, and to work with us to identify risks and concerns.”

“If after these two weeks, things look bad, we’ll know it, and we’ll recommend you stop. If things look good by then, we’ll decide together what the next decision point will be. It could be a month from now, or three months from now. Frankly, we’re perfectly happy to do this every two weeks.”

“Yes, that’s right. Every two weeks we’ll talk with you about what to do next. Let’s do it and show you. If it’s good enough, we continue. If not, we stop.”

“That way, your risk will never be greater than another $20K. You can decide whether to use that money to get more information, to redirect your efforts, or to see whether you should spend that money some other way.”

“Can we work this way?”

If he says yes, let’s start. If he says no, we need to find out why, and even whether this project is for us.

This is definitely estimating, and it’s a kind of estimating that we can do in conjunction with our business people rather than in conflict with them.

We estimate, with some certainty, that we can produce useful information in two weeks. If we don’t think we can do that, we’d better suggest four or six or eight weeks.

More than that, we estimate that a viable product is probably possible in six or nine months. If we had no idea, or worse, if we honestly doubted it, then we can’t legitimately invite him to find out — at least not on the terms above. We owe it to him to say “Frankly, this looks like a two- or three-year project to us. We could be wrong: here’s what we need to learn to find out. Here’s what it would cost to find out, and our best guesses now are that the answer will be bad news.”

Sometimes we have a good idea of what we can do. Sometimes we know less. Either way, by estimating what we know, we can design experiments — investments — that improve our ability to decide what to do.

When we’re talking about spending other people’s money, I think we owe them to think and act this way.

This Is Estimation, Not Negotiation!

An estimate doesn’t have to be “This project will be finished on Tuesday, May 14, at 2:35 PM.” An estimate is supposed to be expressed with a range of possibilities. It would be perfectly OK to say something like, “We see no possibility of this being done before April. If everything goes perfectly, we can be ready by mid-May. Based on our experience, we’re thinking June, maybe July.”

Whether we say this or not, chances are we know it, or at least suspect it. And if that’s what we think, the project needs that information. However, if we put it that way, we’ll be pushed back. Someone will challenge us to prove, without a shadow of a doubt, that we can’t deliver by April 30, and if we prove that, they’ll say “How about May 5?” Bah, we hate it when they do that. So how can we deliver information the project needs without playing this game?

Move from Estimated Date to Estimated Velocity

Let’s use the Five-Card method from previous articles, not to estimate a date, but to break the whole project into sub-stories small enough that our team does, say, five of them in two weeks. Of course we can end up having only three done. We may have six, but we doubt it. Then we arrive at an estimate in terms of velocity:

“We broke the project into smaller stories. We believe that with stories of this size, we’ll be able to do between three and five every two weeks. And at the end of a two-week period, everything we do will be visible, tested, integrated, and working. So whether it’s five or six or two per week, we’ll know, and you’ll know.”

“If you can remove or defer some of these stories and still have a viable product, you’ll have the product sooner. If you can simplify, you can have the product sooner. If you add stories, or make things harder, you’ll delay the product to later.”

“It’s up to you. We can break things into this size, we can guess our velocity now as three items every two weeks, and you’ll know every two weeks what’s really happening. You can use that information to decide what to do, what to defer, in order to get the best product by whatever dates you choose.”

Help Management Learn About Velocity

Now management may still want to negotiate about this with us. If they try to add up the numbers to get dates, and then start adjusting the dates, don’t negotiate: turn the conversation back to the rate of progress. “The date is up to you. If you choose to do more, it’ll be later. If you choose to do less, it’ll be sooner. We think we’ll do between three to five items every two weeks. We may be wrong: if we are, you’ll know as soon as we do.”

Someone may try to push velocity. “Can’t you do seven every two weeks? Then you could finish on time.”

Remember how Agile works: developers work at a sustainable optimal pace; developers aren’t responsible for the dates; the business side owns the dates, and owns the responsibility of reaching those dates.

The developers’ job is to deliver at the best possible pace: “We think we can do three to five. If we happen to go faster, you’ll know. If we’re slower, you’ll know too. When we finish depends on that, but depends even more on how little you give us to do. What goes into those dates depends on you. You should plan on three to five items every two weeks, and check what’s actually happening.”

This can press more than once to go faster. But we have the facts at our disposal, as they are. “Based on our experience, we’ll have three to five done. It wouldn’t be wise to assume anything greater than that, because it probably won’t happen. If something happens that makes us go faster, we’ll all know it and you can select more items into the version.”

All of this, of course, has much more to do with communication, not estimation, and emphatically not negotiation. The estimation was trivial: we did it right at the beginning when we separated the cards. Now we’re communicating what we know, what we guess, what we believe.

We give management our best estimates and production rates, and we commit to them, improving as we evolve. We continue to make it clear that the business side controls the dates by choosing how much work they ask for.

Optimizing the Product

Whether you’re in favor of estimates or not, you should know that Agile is about building products incrementally, in good condition, from day one to the last day, and extracting value from it as early as possible. That requires great technical skills, but technical skills aren’t enough. The people paying for our work need to allocate their resources. They need to manage their money, and they need to coordinate our work with activities and stakeholders outside our team.

To do this efficiently, they need information about how long things will take, which translates into how much things will cost, and how long it will take for us to start getting our money back. They need the information we have to do this work well. The fact that our information is vague, flawed, and uncertain isn’t a reason not to expose it. We need to put our information in their hands, and do it in a way that gives them the best opportunity to use it as well as possible.

Refusing to estimate isn’t the right thing to do. Estimating is necessary, and effective communication of the estimate is necessary too.

The best thing to do is to develop small features at the best sustainable pace possible, and use the information generated to help business people decide what to ask for, and what to defer. To do this, we need to estimate. We break stories into small estimable sizes, and we estimate our rate of delivering them. This type of estimation isn’t evil at all: it’s incredibly useful and is what the best agile teams do.

Ron Jeffries has been developing software longer than any living person. He holds advanced degrees in mathematics and computer science, both earned before negative integers had been invented. His teams built operating systems, compilers, relational database systems, and several large applications. Ron’s software products generated more than half a billion dollars in revenue, and he wonders why he didn’t earn any of it.

Send feedback to the author or discuss the article on the magazine forum.

The biggest problem everyone has is believing that “democracy” is the only and best way to govern, or to decide anything, because “the majority,” “the people,” “the masses” are supposedly always right, and whatever was decided collectively must prevail. That’s the greatest mistake of all time.

To recap what I said before: liberty is important, defending liberty is important, and the most important liberty to defend is the individual’s. The majority must never tyrannize the minority. And the smallest minority is always the individual. If an individual is deprived of his liberty, there is no liberty for anyone.

In a state of liberty, everyone has the right to form and express their opinions. The confusion starts when an opinion is uttered by a “majority.” Democratically speaking, that “opinion” seems to gain a lot of weight. And there’s another great mistake.

Let’s define:

• Opinion: a view or judgment formed about something, not necessarily based on fact or knowledge.

• True: according to fact or reality. Accurate, exact. Real.

Opinion is not fact nor truth.

The opinion of a majority will never automatically be a truth. Any opinion of a majority will never be worth more than the proven truth uttered by a single individual, no matter how large that majority is.

This matters. That’s exactly why I said the best way to think about organizations stays far from the “purely” democratic model. In our niche of software development, many assume that agilists and agile methods are “democratic.” If you read the Wikipedia entry on Scrum, you’ll find this definition of a “team”:

The Team is responsible for delivering the product. A Team is typically made up of 5–9 people with cross-functional skills who do the actual work (analyse, design, develop, test, technical communication, document, etc.). It is recommended that the Team be self-organizing and self-led, but often work with some form of project or team management.

Nowhere does it say the Team can be anarchic. But people read the concept of self-organization wrong, as “we do as we please.” And that’s completely wrong.

It’s the same problem as a purely democratic system of government. The majority can decide among themselves the implementation of a law, a principle, or a truth. One inviolable and indisputable principle, for example, is that an individual must be able to keep for himself the products of his own work. The community can decide, democratically, whether the implementation will be a police force whose cost is split equally among everyone.

That’s the point missing when we talk about agile organizations or teams. Teams can decide democratically how to carry out the implementation, but everyone needs to understand that inviolable principles limit which decisions this “democracy” gets to make.

Ignorant voting in large volume, to me, just means a large volume of ignorance. Want to define what’s true? Study, research, demonstrate, prove. It takes a single individual with the correct demonstration to shatter the opinion of any majority. The reverse never happens.

Take a company: its goal is to obtain as much profit as possible. A job has no other goal than to turn a profit. So it does you no good to build the best-made software (in your personal opinion, by the way, which doesn’t make it a fact) if it generates loss instead of profit.

Profit is a premise, but not the only one. Remember the definition of “Team.” A team must be “accountable,” meaning it has to commit to delivering a project the way the client expects, including managing that expectation (something I’ll detail in another article).

A project commissioned to a team belongs to whoever pays for it. A so-called agile team, when it takes on a project, takes on the client’s expectations along with it, from start to finish, far beyond simply coding the requested product.

More than that: the goal of a project is to achieve a result for the client. The client doesn’t care whether the software uses technique X or Y to raise its quality, because quality is a trivial trait already expected by whoever orders it. The client’s value comes from the result the software produces.

Think about an e-commerce project. What matters is the sales it generates, far more than the software itself. If you build the most advanced system in the world, with the best architectures and technologies, but it never sold a single product, it’s worth nothing. Meanwhile an e-commerce built with the worst technologies, full of every documented hack, but that helped generate a million direct sales for the client, is infinitely superior, no argument.

A developer should value their technical work. That’s great and should continue. A systems architect values their analysis and architecture, no problem at all. An agilist values their processes and rituals. Every professional should always try to do their best at what they do.

The thing is, none of these activities is more important than the others, and none of them is an end in itself. A developer worried only and exclusively about their own code, at any cost, is a terrible professional.

And here’s the difference between a professional and an amateur: professional ethics.

The professional carries additional moral responsibilities to those held by the population in general and in society. This is because professionals are capable of making and acting on informed decisions in situations that the general public cannot, because they have not received the relevant training. For example, a lay member of the public could not be held responsible for failing to act to save a car crash victim because they could not give an emergency tracheotomy. This is because they do not have the relevant knowledge. In contrast, a fully trained doctor (with the correct equipment) would be capable of making the correct diagnosis and carrying out the procedure and we would think it wrong if they stood by and failed to help in this situation. You cannot be held accountable for failing to do something that you do not have the ability to do. Ethics are rules and values used in a professional setting.

The tradition of medicine, for example, inherits the Hippocratic Oath, updated over millennia. This is the modern version:

• I swear to fulfill, to the best of my ability and judgment, this covenant:
• I will respect the hard-won scientific gains of those physicians in whose steps I walk, and gladly share such knowledge as is mine with those who are to follow.
• I will apply, for the benefit of the sick, all measures which are required, avoiding those twin traps of overtreatment and therapeutic nihilism.
• I will remember that there is art to medicine as well as science, and that warmth, sympathy, and understanding may outweigh the surgeon’s knife or the chemist’s drug.
• I will not be ashamed to say “I know not,” nor will I fail to call in my colleagues when the skills of another are needed for a patient’s recovery.
• I will respect the privacy of my patients, for their problems are not disclosed to me that the world may know. Most especially must I tread with care in matters of life and death. If it is given me to save a life, all thanks. But it may also be within my power to take a life; this awesome responsibility must be faced with great humbleness and awareness of my own frailty. Above all, I must not play at God.
• I will remember that I do not treat a fever chart, a cancerous growth, but a sick human being, whose illness may affect the person’s family and economic stability. My responsibility includes these related problems, if I am to care adequately for the sick.
• I will prevent disease whenever I can, for prevention is preferable to cure.
• I will remember that I remain a member of society, with special obligations to all my fellow human beings, those sound of mind and body as well as the infirm.
• If I do not violate this oath, may I enjoy life and art, respected while I live and remembered with affection thereafter. May I always act so as to preserve the finest traditions of my calling and may I long experience the joy of healing those who seek my help.

It’s far from a “definitive” version, but I’ll take the chance to adapt, closely following the Hippocratic Oath, what a draft oath for programmers might look like. Just to make clear the idea that every professional carries responsibilities greater than their general work activity:

• I swear to fulfill, to the best of my ability, this covenant:
• I will respect the hard-won scientific gains of those programmers in whose steps I walk, and gladly share such knowledge as is mine with those who are to follow.
• I will apply, for the benefit of the client, all measures which are required, avoiding the two-edged swords of overapplying technologies or prejudice against brands or technologies I don’t know.
• I will remember that there is art to software development as well as science, and that warmth, sympathy, and understanding may outweigh the programmer’s code or the analyst’s diagram.
• I will not be ashamed to say “I know not,” nor will I fail to call in my colleagues when the skills of another are needed for the benefit of the client’s project.
• I will respect the privacy of my clients, for their problems are not disclosed to me that the world may know. Most especially must I treat with care the projects of my company or my clients. If I am given the task of delivering a project, all thanks. But it may also be within my power to recover a failing project; this awesome responsibility must be faced with great humbleness and awareness of my own frailty. Above all, I must not mess around on the job.
• I will remember that I do not deal with a code file or an editing tool, but a project, whose result may affect the client’s economic stability, the client company’s employees, and their families. My responsibility includes these related problems, if I am to develop the project adequately.
• I will prevent bugs and technical problems whenever I can, for prevention is preferable to correction.
• I will remember that I remain a member of society, with special obligations to all kinds of projects, those I can start from scratch my own way, or those I inherit in someone else’s style.
• If I do not violate this oath, may I enjoy life and art, respected while I live and remembered with affection thereafter. May I always act so as to preserve the finest traditions of my calling and may I long experience the joy of developing projects for those who seek my help.

My point with this text is to remind you of the following:

• stop using “democracy” with populist connotations. What needs to be protected and benefited is the client’s result and their projects, way ahead of the “team.” Never forget that.
• certain principles are inviolable and not open to a vote: business objectives on one end, professional ethics on the other.
• everyone has the right to hold and express personal opinions, but an opinion is just that: an opinion. And no vote turns an opinion into a truth. 2 + 2 will be 4 no matter how many people vote that it’s 5.
• show a little respect and dignity for your profession. You work for your own benefit, sure, but without causing harm to others. I’ll repeat this ad nauseam: you only have work because there’s someone willing to pay for it. Be ethical, be professional.

Honestly, it bothers me to imagine a world where an ignorant majority can decide, by vote, that the Earth is flat, that folk charms work, that an employee “deserves” a salary higher than the company can afford, that a “professional’s” fun ranks above their responsibilities. Swear to yourselves, not to me or some third party, that you’ll be professionals. Or give up working in the same field we do, so you stop embarrassing those of us who take this seriously.

Instead, I’m going back to zero and redefining everything. The first thing I need to correct is the use of the word “Democracy.” It doesn’t describe the kind of organization I have in mind.

My line of thought always started from Darwinian Evolutionism. It’s the only way in which Chaos tends toward a certain Order, through self-organization. This process involves several mechanisms, and it’s about them that I’ve already given talks: Scale-Free Networks, Complex Adaptive Systems, and how all of that leads to processes and methodologies, including the much-discussed Scrum.

This line of study, which you can follow in my blogs and talks of the last three years, brushes past biology, sociology, psychology, philosophy, physics, and mathematics. In the end, the most coherent path seemed to lead to what today is called Democratic Organizations, mainly because of famous cases like Semco, of Ricardo Semler, plus democratic aspects observed in companies like Southwest Airlines, Dreamhost, Groupon, and Zappos.

Except I fell into exactly what I always criticize: a handful of positive evidence about a model, however good it may be, never proves the model. Concluding from positive evidence alone is the same as cargo cult. So I need to correct course: the kind of organization I describe and defend has nothing to do with “Democracy.” In my conception, the term “Democratic Organizations” is a mistake.

Democracy vs Republic

Thank goodness our country is called “Federative Republic of Brazil.” These days I don’t know anyone who speaks ill of the word “democracy.” I won’t give a politics lesson, since I’ve studied almost no political science. Without going into all the ramifications, I want to go down first to the minimal definitions of the term and the original ideas.

I don’t defend the owner of the site I’m about to quote, nor all of his ideas, but the text serves my purpose, so I decided to translate it directly. The original is this article by Gary McLeod. If you search Google for “democracy vs republic,” you’ll find hundreds of articles detailing the differences. For now, this one serves. Here’s the translation:

Rule by Law vs Rule by Majority

Just after the completion and signing of the Constitution, in reply to a woman’s inquiry as to the type of government the Founders had created, Benjamin Franklin said, “A Republic, if you can keep it.”

Not only have we failed to keep it, most don’t even know what it is.

A Republic is representative government ruled by law (the Constitution). A democracy is direct government (Akita’s note: yes, there is representative democracy, but as I said I won’t go down all the ramifications) ruled by the majority (mob rule). A Republic recognizes the inalienable rights of individuals while democracies are only concerned with group wants or needs (the public good). Lawmaking is a slow, deliberate process in our Constitutional Republic requiring approval from the three branches of government, the Supreme Court and individual jurors (jury-nullification). Lawmaking in our unlawful democracy occurs rapidly requiring approval from the whim of the majority as determined by polls and/or voter referendums. A good example of democracy in action is a lynch mob.

Democracies always self-destruct when the non-productive majority realizes that it can vote itself handouts from the productive minority by electing the candidate promising the most benefits from the public treasury. To maintain their power, these candidates must adopt an ever-increasing tax and spend policy to satisfy the ever-increasing desires of the majority. As taxes increase, incentive to produce decreases, causing many of the once productive to drop out and join the non-productive. When there are no longer enough producers to fund the legitimate functions of government and the socialist programs, the democracy will collapse, always to be followed by a Dictatorship.

Even though nearly every politician, teacher, journalist and citizen believes that our Founders created a democracy, it is absolutely not true. The Founders knew full well the differences between a Republic and a Democracy and they repeatedly and emphatically said that they had founded a republic.

Article IV Section 4, of the Constitution “guarantees to every state in this union a Republican form of government”…. Conversely, the word Democracy is not mentioned even once in the Constitution. Madison warned us of the dangers of democracies with these words,

“Democracies have ever been spectacles of turbulence and contention; have ever been found incompatible with personal security or the rights of property; and have in general been as short in their lives as they have been violent in their deaths…”,

“We may define a republic to be … a government which derives all its powers directly or indirectly from the great body of the people, and is administered by persons holding their offices during pleasure for a limited period, or during good behavior. It is essential to such a government that it be derived from the great body of the society, not from an inconsiderable proportion or a favored class of it; otherwise a handful of tyrannical nobles, exercising their oppressions by a delegation of their powers, might aspire to the rank of republicans and claim for their government the honorable title of republic.” James Madison, Federalist No. 10, (1787)

“A wise man will not leave the right to the mercy of chance, nor wish it to prevail through the power of the majority. There is but little virtue in the action of masses of men.” Henry David Thoreau (1817-1862)

Our military training manuals used to contain the correct definitions of Democracy and Republic. The following comes from Training Manual No. 2000-25 published by the War Department, November 30, 1928.

DEMOCRACY:

• A government of the masses.
• Authority derived through mass meeting or any other form of “direct” expression.
• Results in mobocracy.
• Attitude toward property is communistic, negating property rights.
• Attitude toward law is that the will of the majority shall regulate, whether it be based upon deliberation or governed by passion, prejudice, and impulse, without restraint or regard to consequences.
• Results in demagogism, license, agitation, discontent, anarchy.

REPUBLIC:

• Authority is derived through the election by the people of public officials best fitted to represent them.
• Attitude toward law is the administration of justice in accord with fixed principles and established evidence, with a strict regard to consequences.
• A greater number of citizens and extent of territory may be brought within its compass.
• Avoids the dangerous extreme of either tyranny or mobocracy.
• Results in statesmanship, liberty, reason, justice, contentment, and progress.
• Is the “standard form” of government throughout the world.

The manuals containing these definitions were ordered destroyed without explanation about the same time that President Franklin D. Roosevelt (F.D.R.) made private ownership of our lawful money (US Minted Gold Coins) illegal. Shortly after the people turned in their $20 gold coins, the price was increased from $20 per ounce to $35 per ounce. Almost overnight, F.D.R., the most popular president this century (elected 4 times), looted almost half of this nation’s wealth, while convincing the people that it was for their own good. Many of F.D.R.’s policies were suggested by his right hand man, Harry Hopkins, who said,

“Tax and Tax, Spend and Spend, Elect and Elect, because the people are too damn dumb to know the difference.”

It Can’t Be Democracy

If you research at least the Wikipedia articles on Democracy and Republic, you’ll see that the subject is much more complex than this.

What I most admire about the Founding Fathers of the United States is the philosophical and political clarity that led them to describe a Republic based on inalienable Individual Rights. Out of that came solid foundations like the Declaration of Independence and the Bill of Rights. I’m not saying they were perfect people, they all had a thousand flaws, but that’s not where we should get stuck (that would be the Straw Man Fallacy).

The so-called Democratic Organizations fail precisely at this point, in the foundation. It simply doesn’t exist. And worse, from a philosophical point of view, it’s incompatible with the Constitution.

There’s no “official” document with these principles, so I’ll borrow the list from WorldBlu, an organization that tries to spread the idea of Democratic Organizations around the world. For the details, it’s worth reading their page. In summary, the principles are these:

1. Purpose and Vision: normal, any company needs this clear, what its core business is, and what it exists for.
2. Transparency: there are controversies, not everything can be open, including some things that are actually legally protected.
3. Dialogue + Listening: I agree, the concept is valid, and against top-down authoritarianism.
4. Equality + Dignity: hard to define, some differences in treatment are more cultural than mandatory.
5. “Accountability”: hard to translate into Portuguese, but it’s something like “Responsibility”; in this concept it’s more about better defining who is responsible for what.
6. Individual + Collective: this is where the whole concept starts to fall apart.
7. Choice: implementable, but within limits.
8. Integrity: mentions “morally” and “ethically” correct, and that’s a problem.
9. Decentralization: instead of centralization, I’m always in favor of this, but within limits.
10. Reflection + Evaluation: a “touch” of Toyota’s Hansei+Kaizen model.

The last two points, Decentralization and Reflection + Self-Evaluation, are the only ones I would call “principles.” If you read the articles I linked in the introduction, you’ll understand why.

My problem with these principles is that they are rhetorical, and so they don’t work as principles. Saying “it has to be transparent” is pure rhetoric. I can claim I never told a lie, and who’s going to disprove me?

The same goes for “dialogue” and “dignity”: they’re good rhetoric, but they describe conduct. They’re process, not foundation. They don’t answer the real principle: “Why should I act this way?”

And this lands on “Integrity,” on doing what is morally and ethically right. But who defines what is morally and ethically right? Without a philosophy behind it, everything stays undefined and open to interpretation, because there’s no objective, unambiguous principle holding the idea up.

Private Property

I finally get to the points “Individual + Collective” and “Choice,” which deal directly with “Democracy.” These two knock down all the others. Democracy and Collective are bad principles. Any system where the majority can crush the minority, and the minority has no objective, defined property rights, doesn’t work by definition.

And “right,” to be clear, isn’t just anything. Things like “right to bonuses” or “right to more rest hours per week” run into the fundamental problem of any organization: “And who pays the bill?” If, for one individual to have a right, another has to work for free, that’s slavery and, by definition, cannot be a right. Read my article on Rights of Man to understand it better.

More than that: “Transparency” as unrestricted access to all company data, and “Choice” over every relevant matter, should never be automatic rights of employees.

Let’s get to the foundation of all this: every company and organization is private property of its owner, partners, or shareholders. It does not belong to the employees, unless they are also shareholders, for example via a stock options program or something of the sort.

Employees are individuals who trade their capability and their work hours, in a voluntary negotiation with the company, for a value called “salary.” It’s a fair, voluntary exchange, for mutual benefit. The employee’s right ends where what is written in the signed work contract ends. There’s nothing to “claim.”

Want more access to confidential information and more say in strategic decisions? Earn it. Make yourself deserve it, don’t expect it to fall into your lap automatically. Show good work, deliver results, and demonstrate the capacity to decide; only then earn the merit of being part of it. From the employees’ point of view, the principle of an organization should always be meritocracy.

But what about the various companies that are positive “evidence” that “democratic organizations” work?

Here’s the key point of this whole article: these cases of Democratic Organizations are not necessarily Democratic. At least not as people imagine it should be.

Actually, if you read my articles on animal groups, chaos leading to order, and self-organization, it makes full sense within limits. No organization needs to control even the color of the sock each employee wears. That’s not efficient or productive. How the team works, what hours, with what tools and codes of conduct, is something the team itself can decide.

Except that, to get to that point, the company’s goals were already defined: where it wants to go, what it wants to produce, how it wants to enter the market, and all the rest.

What the teams will decide “democratically” is how that strategy gets implemented. The “upper echelon” still exists, the company still belongs to its owners, and the strategy arrives ready. What’s left to the employees is the tactics, the implementation.

This is the natural path of organizations. If you want to call that “democratic,” fine. I prefer to slap no label on it and treat it as one more evolutionary step of “Organizations.”

Leaving “the way of implementing the strategy” in the hands of the people who will implement it is just another way that seems efficient to run a business. It doesn’t necessarily work for all of them. And in the companies treated as cases, I believe we find more evidence that they adopted more efficient means of production. The “democratic” label runs too big for what actually happens in them.

It’s like the Toyota Production System (TPS, or Lean for us) at Toyota. It doesn’t show up on the popular “democratic organization” lists and probably doesn’t even fit WorldBlu’s principles, but it does employ “democratic,” or “participatory,” tactics in the implementation of production. That’s the real goal.

And no, I also don’t think the right term would be “Republican Organizations,” just because I talked so much about Republic at the start. It’s neither one. Organizations are Private Properties that need to be managed. In its foundation, the whole organization cannot be “democratic” to employees who aren’t shareholders, by definition.

A publicly traded company can even have an almost “republican” structure. But the point where we want to study and intervene is the day-to-day management of teams, and there we have a “silo” with “participatory” implementations inside a wrapper that isn’t democratic.

The reason for this long explanation is simple: if you think too long about the colloquial definition of “Democracy,” it quickly slides into “Collectivism,” “Populism,” “Communism,” and “Socialism.” I’m absolutely against this line of evolution, it’s philosophically unhealthy. In the political realm, it should stop at “Republic.” In the business realm, it’s simply “Capitalism,” nothing added or taken away.

And the title “Democratic Organizations Don’t Work” is, of course, a provocation. It points to the fact that I’m correcting myself on the wrong use of “Democracy,” when in reality we’re talking about “Organizations with Efficient Processes,” especially the ones that grow organically with an evolutionary-Darwinist logic, like Lean/TPS.