Get Better at Estimating
The world of business loves its estimates. And it's amplified in startups.
How long will it take to add this feature?
If we don't build XYZ integration, the customer won't sign.
Sales keeps selling things we don't have, how fast can you build this?
Unfortunately, as you and I know, building software is hard to estimate. As engineers, we tend to underestimate - chronically.
When you underestimate, you harm your reputation. Product (or you yourself) has to scurry back to the stakeholders and make up excuses. The business loses trust in engineering.
The loss of trust widens the chasm between engineering and business. Engineering blames business for unrealistic goals. Business blames engineering for their inablility to ship.
But there is hope.
Why do we underestimate?
Here's a weekend project that took 6 weeks.
I wanted to build a new garden bed.
Requirements:
A box, made of wood, placed in the ground, and filled with dirt.
Simple, right?
Building this garden bed took one afternoon. Piece of cake. Dare I say, fun.
Installing it was hell.
I picked out the perfect spot in my yard. Lots of sun and adjacent to a drip line. I started digging one hole for the first post.
After topsoil, I discover 2 layers of weed barrier I have to remove. Ok, no problem. I cut that away. I start sweating.
I dig some more, sweat more, and swear mre. The soil underneath is hard clay. You could make pottery with it.
I dig again. My sweat is filling up the hole. My swears are scaring the neighbors.
Clunk.
My shovel hits something hard. It doesn't want to go further. I don't force it.
I take a small shovel and pick around the obstruction.
Lo and behold, I find the underground sprinkler line. This is a problem. If something goes wrong with that line, do I want an 8x3 foot garden bed full of dirt on top of it?
I quit for the day.
And then:
I agonize over an alternate location for days. Every other location has other problems.
A literal tornado sweeps through the neighborhood and knocks down a portion of the fence. (My house is fine.)
I redneck engineer the fence back to standing.
I agonize over the fence for days. Watching YouTube videos.
I decide to use the original location. I'll risk the sprinkler line breaking.
I shorten the posts so the bed is easier to uproot.
It's a lot of digging.
I make 2 trips to the hardware store because I didn't buy enough dirt the first time.
Finally, it's done.
P.S. That's my dog, Penny. She did not help.
So what does all this have to do with software estimation?
Plan time to dig
Software estimation is hard because of the unknown unknowns.
As the old saying goes: "You don't know what you don't know".
Unknown unknowns are surprises you can't envision, such as:
Performance bottlenecks
Data model incompatibilities
Security vulnerabilities and considerations
An API vendor that has crippling rate limits
Legacy code you forgot about that must be dealt with
Tech you thought worked out of the box doesn't work at all
Software systems become so large, it's impossible to keep it in your head.
Like the physical world, you don't know what you'll uncover until you explore a bit.
Therefore, as a software dev, you must ask for time to explore. Time to discover. Because you can't see everything.
You must ask for discovery and give it a timebox - a couple days, a week, a sprint.
How long you need depends on the urgency and scope.
When you explore that means you will:
1. Read lots of documentation.
2. Try out various libraries.
3. Build a POC (proof of concept)
4. (Nowadays) Chat with an LLM
But, be sure to produce a deliverable. A barely-working POC that you demo is great. Or, a document outlining what you discovered is another good option. But you must deliver something tangible.
Congrats! You've learned more and can more accurately estimate what it will take.
Fight for your right to discover
Discovery time is foreign to many startups. You need to fight for it.
If you can, make it a collaborative decision. Once your team and Product digest your deliverable (yummy), you should have feedback. Your teammates will see things you didn't.
Many brains are better than one.
You will still find surprises
There are two types of surprises.
The first is, sadly, you will still be wrong.
You can't unearth everything, otherwise you'll spend all your time digging instead of building. Imagine if I dug all the way to Earth's core to install a garden bed. A total waste of time (and I would be dead).
So the goal is to be less wrong. To use a machine learning analogy, you are minimizing a loss function. You want to be as minimally wrong as possible.
And you will be wrong. You may find a hidden "snake in the grass" you never saw before.
Once you find a surprise, even a showstopper, your job is to communicate the surprise to Product ASAP.
Your business will appreciate the transparency.
The second surprise is a new priority showing up and disrupting everything.
Like a tornado.
You can’t see these coming.
A new customer signed a huge contract, the database keeps crashing, you got featured on Hacker News, and so on.
Rapidly shifting priorities is par for the course in startup land.
These surprises are beyond your control.
When asked to switch gears to a new priority, your job is to say "yes", and it means you are de-prioritizing the current task at hand.
I can do this, but it means I can’t do that other thing right now. I can still do that other thing, but it will have to wait.
Let the business decide the priorities.
Wrap Up
Software estimation is hard. Startups will always want to know how long something will take.
You will never see the full picture.
Plan time to research and explore.
There will always be surprises.
Communicate!
If you enjoyed this post, please subscribe for more startup advice and pictures of Penny.