Introducing: The Yak Shaving Series

Sharing the pain of DevOps through short stories

Most programmers are terrible at estimating how long a project will take. “Let’s see, all I need to do is deploy a few servers in AWS,” you think to yourself, “I’ll have it done in a few hours.” Three weeks, two all-nighters, 457 cups of coffee, and three rolls of duct-tape later, when you finally have the thing up and running, you look back at what you did, and despite having gone through all that pain, you can’t quite remember why it took so long. And so your next estimate is just as wildly inaccurate as the first.

One of the reasons software estimation is so hard is that software projects have a nearly infinite level of detail, a bit like a fractal. When you are doing an estimate, all you think about is the big, zoomed out picture, which seems completely manageable:

But when you actually start doing the project, you begin to zoom in, and realize there is quite a bit of detail hiding in every corner. And each of those details seems to have more details attached to it, and each of those has more, and so on.

This sort of issue is often referred to as “yak” shaving. Seth Godin has a particularly good description of it:

Yak Shaving is the last step of a series of steps that occurs when you find something you need to do. “I want to wax the car today.”

“Oops, the hose is still broken from the winter. I’ll need to buy a new one at Home Depot.”

“But Home Depot is on the other side of the Tappan Zee bridge and getting there without my EZPass is miserable because of the tolls.”

“But, wait! I could borrow my neighbor’s EZPass…”

“Bob won’t lend me his EZPass until I return the mooshi pillow my son borrowed, though.”

“And we haven’t returned it because some of the stuffing fell out and we need to get some yak hair to restuff it.”

And the next thing you know, you’re at the zoo, shaving a yak, all so you can wax your car.

At Gruntwork, we do a lot of yak shaving. We’ve spent the last couple years building Infrastructure Packages (reusable solutions to common infrastructure problems, such as how to run a Docker cluster or how to manage your network topology in AWS), and along the way, we’ve shaved many, many yaks. And now, we’re going to start sharing some of these stories with you in a series of blog posts we’re calling The Yak Shaving Series.

We hope you’ll find these stories entertaining. We hope that by writing down the gory details of hair loss — both our own and that of the yak — we can capture just why building and estimating software is so hard. We hope that by sharing these stories, we can turn days of our frustration and pain into something we can all laugh about and perhaps even learn from.

So, without further ado, here are the first few entries in our Yak Shaving Series:

1. All I need is a little bit of disk space: a horror movie featuring Auto Scaling Groups, EBS Volumes, Terraform, and Bash.
2. A Tale of 12 Errors: Adventures with Terraform, AWS Lambda, and KMS
3. Inline blocks, conditionals, count limitations, and other Terraform yaks (coming soon)
4. Golang: the 1970’s called and they want their language back (coming soon)

Do you have your own stories of Yak Shaving? If so, share them in the comments!

Your entire infrastructure. Defined as code. In about a day. Gruntwork.io.