Releasing software more often and innovating faster, while at the same time keeping risks manageable, has become wide-spread. Continuous experimentation guides development activities based on data collected on a subset of online users on a new experimental version of the software. It includes practices such as canary releases, gradual rollouts, or A/B testing. Unfortunately, our knowledge of continuous experimentation is currently primarily based on well-known industrial leaders such as Facebook or Microsoft. In this talk I will present key findings from an empirical study in which we interviewed 31 developers and release engineers to assess the actual state of practice in continuous experimentation. In addition, I will talk about how we can support software developers and release engineers conducting such experiments since manually administering them is often a cumbersome task (e.g., deciding about metrics to monitor, and whether to continue or discard experiments). We developed a middleware for automatically executing experiments in microservice-based applications. Our prototype introduces a performance overhead of around 8 ms and can run more than 100 experiments in parallel even on cheap cloud instances. I will conclude my talk giving an overview of ongoing work in which we try to apply concepts known from software product line research in the service domain.
Since September 2014 I'm a PhD student at the Software Evolution and Architecture Lab (seal) of the University of Zurich, supervised by Prof. Harald Gall. I received my master’s degree in computer science from the Vienna University of Technology in 2014. My current research focuses on continuous experimentation and continuous delivery/deployment. I am interested in how we can better support developers and release engineers in conducting experiments.