Continuous rewriting

Introduction

What would happen if your team rewrote every line of code in your application every six months?

Squirrel presented for discussion the thesis that "Continuous Rewriting" is an emerging theme for software development, a successor to better-known ideas "Continuous Integration" and "Continuous Deployment". Continuous Rewriting (CR) promises better responsiveness, faster feedback, improved understanding, and simpler code through regular rewriting of every part of a complex software product.

The session opened with discussion of the chart and two examples below:

	2000-3	2004-6	2007-9	2010-12	2012-
Continuous Integration	Integration hurts, so automate & do more often. - Daily Builds, Ant, CruiseControl	More tools! - JetBrains, Hudson, BuildForge	Adopted by advanced teams - CI books, CITCON	Tool shakeout, standard for new teams - Jenkins	Adopted by late majority?
Continuous Deployment			Delivery hurts, so automate & do more often. - one-click deploy, IMVU	Tools and culture! - DevOps, Puppet, Chef, Capistrano, Octopus	General adoption by advanced teams? - CD book
Continuous Rewriting				Rewrites hurt, so automate & do more often. - DRW (Dan North), Forward (Fred George)	????

Example 1: DRW

• Small, expendable, co-operating components
• Each fit for purpose
• Hard shell, soft centre
• Message = API
• Identifiable boundaries for experiment

References: Dan North talk, Slides

Example 2: Forward

• Small, short-lived apps
• No testing
• Continuous deployment

References: Fred George talk, Slides

Discussion

We discussed and criticised the thesis and examples. Points made included:

Why does rewriting hurt? You may have crack devs who push for a rewrite, but the new code doesn't quite match the old, and the rewrite causes a delay, and the new code becomes legacy quickly. If you have slobs, they may just live with the old code which is equally bad.

Red Gate use a strategy called "continuous monitoring".

Some strategies for piecemeal rewrites:

• when you get a change request, extract the relevant component and rewrite it
• the Strangler pattern

Will CR scale as CI and CD appear to? Way too early to know, but Fred George at Forward has large teams using their "programmer anarchy" model.

The model is similar to spaced learning, where you repeat things you want to learn at intervals to help you retain them.

Techniques

We moved on to discuss methods that might be used by teams adopting Continuous Rewriting. These included:

Dan North's "Spike and Stabilise" method, where you code something quickly, prove it is useful, then write tests that help you explore your code and stabilise it.

Hot-swapping or simultaneous running of old and new code - for example, both old and new code draw the same work items from a common queue and a voting or filtering system checks whether they got the same result and takes appropriate action if they didn't. Related example: NASA put four computers in the Space Shuttle that voted on how to steer the ship:

"On the Shuttle, four identical AP-101Bs would function simultaneously as a quadruple-redundant set during critical mission phases such as ascent and reentry, processing the same information, derived from completely separate data buses, in precise synchronization. If a conflict arose among the four primary computers, the majority would rule, voting the conflicting unit out of the loop. None of the computers, singly or en masse, could turn off any other—that step was left to the crew. An errant machine would announce itself to the crew with warning lights, audio signals, and display-screen messages—all suggesting that the crew might want to isolate (i.e.; turn off) the offending computer from the system." - NASA history

LMAX Disruptor and Event Sourcing are models that may be helpful for CR.

Problems and Questions

We raised these questions and thought of these problems with Continuous Rewriting:

Front-end, visual code could be a problem area; A-B testing could help.

Infrequent events - e.g. an annual audit or quarterly report - could form roadblocks to a CR-using team, as they won't easily have the opportunity to check that the rewritten code gets the same result as the old code in production.

A stack of unanswered questions:

• What about infrastructure? Retrofitting to existing software? Organisational acceptance? (Fast feedback and fix may help with the latter.)
• Are there types of application where this approach won't work?
• Can you get the same learning benefits from team rotation?
• Will increased readability for the rewriter mean better code, or will it mean less comprehension for others on the team?
• Will CR-using teams have tacit code ownership or will CR eliminate that ownership?

Benefits

We brainstormed these possible benefits of adopting CR:

• Code evolves with business understanding
• Eliminate code ownership
• Fast feedback and fixes
• Reduces team bus number, i.e. more people know more of the code
• Simpler, shorter code

Characteristics

We thought of these characteristics a system built using CR would likely have:

• Made up of small components (e.g. source code under 10K bytes)
• Components connected by messaging system
• Reduced usage of database, perhaps no database at all
• Deployment in stages - component live but isolated, then side-by-side with existing component, then fully deployed
• Polyglot - components written in a wide variety of languages
• Business metrics monitored, automated actions to address (similar to IMVU's continuous delivery system, which automatically rolls back if revenue drops)
• Team develop rewriting rules, goals, guidelines
• Business receptive to new ideas, driven to change quickly

Post-CITCON comments and ideas

Peter Zsoldos made several helpful comments after the conference; he asked

• what role acceptance tests might play in a CR world (do we keep them or delete them on a rewrite? Squirrel thinks we keep them and modify as needed)
• what might a CR advocate say to the points raised by Joel Spolsky in his article on rewrites?

Peter also wrote a blog post addressing some of the issues raised in this session and raising a new question: how might CR help teams rotate developers more effectively?

An article on the Obama campaign's software by Luc Perkins suggests that the Democrats used some of the techniques described here, such as combining many small components with a short half-life, though the article doesn't mention how much rewriting they did.

Andy Palmer drew comparisons with the micro-services ideas described by James Lewis of Thoughtworks. Here are some slides on micro-services.

Julian Gamble pointed to the continuous rewriting of a language by Arthur Whitney.