Configuration‌ ‌is‌ ‌a‌ ‌difficult‌ ‌problem,‌ ‌in‌ ‌part‌ ‌because‌ ‌it‌ ‌often‌ ‌lacks‌ ‌a‌ ‌clear‌ ‌definition‌ ‌–‌ ‌and‌ ‌even‌‌ when‌ ‌it‌ ‌has‌ ‌one,‌ ‌that‌ ‌definition‌ ‌encompasses‌ ‌both‌ ‌technical‌ ‌and‌ ‌social‌ ‌aspects‌ ‌of‌ ‌building‌ ‌and‌‌ operating‌ ‌software‌ ‌systems –‌ ‌there‌ ‌are‌ ‌many‌ ‌systems‌ ‌involved‌ ‌and‌ ‌many‌ ‌stakeholders.‌ ‌And‌ ‌it’s‌‌ an‌ ‌important‌ ‌problem:‌ ‌configuration‌ ‌errors‌ ‌are‌ ‌frequently‌ ‌cited‌ ‌as‌ ‌the‌ ‌causes‌ ‌of‌ ‌production‌‌ outages.‌

I‌ ‌have‌ ‌designed,‌ ‌built,‌ ‌and‌ ‌used‌ ‌a‌ ‌number‌ ‌of‌ ‌configuration‌ ‌systems‌ ‌in‌ ‌organizations‌ ‌both‌ ‌large‌‌ and‌ ‌small.‌ ‌Despite‌ ‌spending‌ ‌years‌ ‌studying‌ ‌the‌ ‌design‌ ‌and‌ ‌implementation‌ ‌of‌ ‌programming‌‌ languages,‌ ‌I‌ ‌was‌ ‌not‌ ‌adequately‌ ‌prepared:‌ ‌configuration‌ ‌is‌ ‌not‌ ‌merely‌ ‌about‌ ‌defining‌ ‌a‌‌ language‌ ‌but‌ ‌also‌ ‌includes‌ ‌how‌ ‌that‌ ‌language‌ ‌will‌ ‌be‌ ‌used‌ ‌(and‌ ‌by‌ ‌whom)‌ ‌as‌ ‌well‌ ‌as‌ ‌how‌‌ software‌ ‌systems‌ ‌will‌ ‌manipulate,‌ ‌create,‌ ‌and‌ ‌consume‌ ‌configuration.‌ ‌In‌ ‌this‌ ‌talk,‌ ‌I’ll‌ ‌present‌ ‌six‌‌ ways‌ ‌these‌ ‌configuration‌ ‌systems‌ ‌have‌ ‌failed‌ ‌(along‌ ‌with‌ ‌concrete‌ ‌examples)‌ ‌so‌ ‌that‌ ‌others‌ ‌can‌‌ avoid‌ ‌the‌ ‌mistakes‌ ‌I’ve‌ ‌made‌ ‌and‌ ‌the‌ ‌pitfalls‌ ‌that‌ ‌I‌ ‌have‌ ‌encountered.

Daniel “Spoons” Spoonhower is a co-founder at LightStep, where he’s building performance management tools for modern software systems. Previously, Spoons spent almost six years at Google where he worked on developer tools as part of both Google’s internal infrastructure and Cloud Platform teams. He has published papers on the performance of parallel programs, garbage collection, and real-time programming. He has a PhD in programming languages from Carnegie Mellon University but still hasn’t found one he loves.