This is not really a post about tests though, it’s really about AI’s contribution (a lot) and what it took to make that contribution usable (also a lot).
Developers on this project had been quarantining tests with a :flaky label for several years. The strategy was to quarantine a small group which could be expected to fail randomly but could also be re-run easily and separately from the full suite. Apart from the flakiness, the test suite is comprehensive and gives us high confidence that if we merge something after tests pass, it works.
Over the years, several developers had tried for a week at a time to reduce flakiness, all resulting in failure. In our defense, the flaky tests centred around interactive pages using Stimulus or Hotwire, and online discussion of this topic is a combination of ideas we tried already, plus someone saying: “I tried a lot, it doesn’t work, I think there’s a bug”.
The most promising angle was adopting Playwright, which did improve some things but also left us with some tests that failed permanently and needed to be skipped. There’s a dissatisfying way in which this is better than tests that only fail some of the time.
The problem started to look more and more like a trap set for enthusiastic developers. As a manager I always had to urge caution: “sure, you can see some approaches that could help, but bear in mind the last five times anyone tried they found very promising angles that didn’t change the stats in github at all”. Developers whom I trust were seriously recommending deleting the entire group.
One night, Opus 4.6 running in Claude Code solved “the problem” by running the flaky test group hundreds of times and analyzing failures. There was some prompting to help Claude avoid premature conclusions and be aware that the problems could not be reproduced without repetition, plus a markdown file where it would record progress. Otherwise, no special magic.
I could see Claude’s progress over time because it needed to run the flaky group in larger and larger batches. At first, five times was sufficient because the errors it found occurred 20% of the time. As those were fixed, I had to tell it to use batches of ten, fifty, and then one hundred. Finally, it reached a point where zero errors were found.
A “nice” thing about needing such large batches is that I could leave Claude alone for hours at a time while my normal evening continued. Flaky specs may be a problem uniquely suited to coding agents in that way. There’s not even much token use: it just kicks off a long run and surfaces for an internal conversation, then kicks off the next batch.
This isn’t a post about test failure strategy, so I’ll spare you details of what was flaky and what fixes applied. Instead I’ll try to communicate some of the meta concerns I had with the resulting code changes.
Given a test that looked something like this:
1 create objects
2 visit page
3 click A
4 click B
5 expect expression 1 to be true
6 click C
7 expect expression 2 to be true
Unchecked, Claude would have turned it into something like this:
1 create objects in a slightly different way that makes no difference
2 visit page
3 explicit sleep
4 unnecessary scoping to a specific section of the page
5 click A
6 end of unnecessary scoping
7 click B, with 3 second wait passed as option arg
8 a clever improvement that should have been on line 3
9 expect expression 1 to be true
10 click C
11 an improvement that worked in other tests but was irrelevant here
12 expect expression 2 to be true
Ultimately the changes added up to a good improvement, usually because of one crucial addition per test (in our fictional example, line 8) that was on the wrong line and hidden in a mountain of garbage (lines 3, 4, 6, 7, 11).
It took two weeks to:
Some of this work was just a matter of applying good practice (e.g. any explicit sleep call is immediately suspect), and other times it was sending Claude back to hundreds of test runs to prove that something it had added made no difference.
I see in myself three reactions.
I think it would have been impossible without lots of experience working with Rails and rspec to move from what Claude was suggesting initially towards something sustainable2. The exact amount of experience necessary is uncertain, but I’m on more than ten years. It took a lot to move beyond the optimism and false positives, and it would have taken more if I didn’t already have a reasonable gut instinct about these things.
I would absolutely use (and recommend) Claude for analysing flaky tests again. I think it would be a mistake not to do so. Accurately running long processes with tiny changes in between multi-hour waits is not a strength for humans.
In addition, Claude did reason through code running in parallel processes in a way that no human had managed for years. That particular part of our code is complex, but has not had active work for years, meaning that no human has good context. Claude probably caught up in 10 minutes.
An interesting aside here is that I find Claude to do much better work when it has tests to help it reason about application code. The tests were flaky, but they were still a good record of what the code was supposed to do.
I could have taken the win, ignored the cruft, and gained two weeks. If I had, I would have lost those two weeks and more later on. Humans and AI agents would cargo cult the new (anti) patterns, falsely claiming victory over any future flakiness, and making it harder to identify the real problems.
As with all programming, eventually “tidy first, then do the work” ends up being faster than “just do the work”. There’s no escaping the tidying if I want good results, the question is whether I do it at a predictable time and pace or when there’s an emergency (like no-one being able to deploy any code because CI keeps failing).
That includes tidying up after AI.
]]>Although these conversations are frequently intertwined, building software away from a desk doesn’t have to come with a 4-digit daily bill. I don’t have that sort of cash, but I’ve still been able to get in on the fun.
Here’s a workflow that I’ve been using for unsupervised AI coding on a side project which will never justify serious focus or budget:
needs-elaboration to the issueready-for-dev labelagent-review label and GOTO 5The total cost is $20 for my Claude Code Pro plan and $10 for a Github Pro Plan (educators and open source contributors can often get this for free). There are usage limits to both plans1 but I mostly run out of ideas before I run out of limits. I’m not focused on making an agent work 24/7 while I’m doing other things; my only concern is that there is something new ready for me each time I have spare attention for this project.
In case it wasn’t clear, I’m not writing or even reading any of the code as part of this loop2. That would take focus that I can’t dedicate on a per-feature basis to this project.
This project wasn’t always an experiment in agentic coding. It was originally an exploration of JS web development with NextJS that I pursued during Friday investment time over a seven month period a few years back. This allows me to compare before and after, with striking results.
In the past month, with 10 minutes here and there on my phone, I’ve far exceeded what I did in the original seven months of Fridays:
Not bad for effort spent while out for a walk or on the London underground3.
When I say “Github starts Claude Code in the cloud” it may not be clear exactly what I mean. Here’s an example of a workflow file from my project’s .github/workflows folder:
name: Claude Issue Triage
on:
issues:
types: [labeled]
jobs:
claude-triage:
if: github.event.label.name == 'needs-elaboration'
...
steps:
- name: Checkout repository
...
- name: Run Claude Code
uses: anthropics/claude-code-action@v1.0.70
with:
claude_code_oauth_token: $
prompt: |
GitHub issue #$ ("$")
requires your analysis...
claude_args: "--max-turns 20 --dangerously-skip-permissions"
show_full_output: "true"
Think of it like running a CI workflow in Github actions, except that instead of running against a branch or pull request, this workflow runs when an issue is labelled needs-elaboration. First it checks out the code, then it runs Anthropic’s Claude Code action. The action passes a fixed prompt to Claude Code plus metadata from the Github workflow (like the contents of the labelled issue).
It doesn’t have to be Claude Code, I could go with Opencode or the Copilot agent instead.
If AI-coding terms like gastown, taskmaster, ralph loops, or minions mean nothing to you, these notes about what becomes different when we stop intervening in agentic code may be helpful.
First of all, the --dangerously-skip-permissions option on Claude Code may look scary, but it’s standard practice for folks who want to run Claude Code without any human intervention. I am less worried about doing this in a random github actions runner in the cloud than I am running it on my local machine. My laptop remains safe for my day job.
In unsupervised agentic coding, there’s a transition from “human in the loop” (approval of individual code changes that the agent makes) to “human on the loop” where the agent’s output over time is assessed and its standard instructions are improved. I’m not telling Claude which code to modify or how. Instead, it decides and I have to live with the results until I can give it what is needed to solve a problem in a better way the next time. This is a lot like being a manager of human coders (at least one who does not want to be a bottleneck to team output).
Some of the code in this project is really bad, for example a 1500-line JSX file. I only noticed this because of how Claude was struggling to implement changes to this code. Ideally I would have noticed this earlier, but the path to recovery is fairly straightforward. I’ll tell Claude to identify pieces of the component that can be split into separate components and files and have it create new github issues that the agents themselves can plan and implement. This is just how I would work as a developer myself (my threshold for breaking the code down would be somewhat earlier than 1500 lines though).
I wouldn’t trade the progress I’ve made in this month for perfect code. There’s a lightbulb moment for every developer when they work on software that they themselves want to use: suddenly the choice to pursue higher quality code has to fight for development priority with wanting to use a new shiny feature. The delicate balance is in knowing that with too much tech debt (e.g. 1500 line components), the shiny features will take longer and longer to build until eventually “improve quality then build new feature” takes less time than just “build new feature”.
A crucial factor is that the number of lines of code is no longer linked to the time it took to produce them. I often start by asking for options to address a problem. Out of five options, three may be easily rejected. When there are two or more realistic options, I don’t need to reason about them in the abstract: I can have Claude build both and see how each of them feels. There is no opportunity cost choice between one option and another.
The AI coding scene moves fast. As I was writing the first draft of this post I was puzzled that I hadn’t seen anyone else discussing a workflow like this. Between then and now Twill.AI announced a real product which seems like the same idea but with some professional features like more advanced sandboxing and integration with Slack, Whatsapp etc. that they’re betting are worth paying for.
Mine is a very low-stakes project which gives me freedom to experiment. There are serious gaps that I would need to close before applying this workflow to professional work, but I don’t think they’re insurmountable (certainly Twill.AI and many others are trying their best). I expect that over time I’ll be able to extend the agents’ work to close more and more gaps without breaking the bank. In the meantime, you can find me at the park with my daughter.
Your actions will just stop running if either Claude or Github plans run out. There are time-based limits that refresh over different periods. Claude is based on tokens and I use their cheaper tier of models (Sonnet) to make the limit go further. Github Pro Plans include an actions-based limit of 3000 minutes per month. I use much less than 3000 minutes per month on my side project. Github has recently changed their pricing model, but it is mostly with regards to AI token pricing, and in this case I’m getting my tokens directly from Anthropic. ↩
Before you worry about the safety of running code without reading: there is no backend, no personal information tracked, and all data is stored in a user’s browser or in their own Google Drive. Also note that I don’t read code per-feature during the loop, but I do get to look at the code as a whole over time. ↩
A few weeks back I was watching people play a game that made me think of a new feature. I took out my phone to write an issue, then carried on watching. By the time I was home, the feature was ready in production. ↩
The title was “How to improve your own and others’ code”.
A PDF of my slides is here.
You can find a recording of an earlier version of the workshop that I gave at the July 2025 London Ruby User Group meeting.
]]>Let’s imagine a plan that consists of five milestones. The organisation that wants this work knows far more about the project by the time they reach item three than they did when they wrote the plan. Tackling items three, four, and five exactly as they were specified in the plan is a choice to act based on outdated knowledge.
There is a better way to plan: we can explicitly acknowledge the vital role of learning by including experiments that we know will influence the plan. Experiments can range from something tiny like swapping between equivalent libraries for pagination, to something really big like trying different distributed architectures.
The benefits of experimenting include:
Below are some examples of real software-delivery experiments performed as small parts of a larger plan, taken from a variety of projects over the last ten years.
When a team found SQL-based search to be unforgiving for end users and very slow across multiple tables, Elasticsearch seemed like a solution worth evaluating.
Implementing it on one page was enough to reveal challenges about the infrastructure needed, client libraries, and the knowledge necessary for useful index and query design.
The experiment quickly demonstrated that the trouble to add new infrastructure and learn the necessary skills was greater than the urgency to solve the search problem.
Given company-wide adoption of Kaminari for pagination in Rails apps and a postgresql performance problem related to pagination, it seemed that the pagy gem (which advertises performance benefits over competitors like Kaminari) might help.
Adding it to only the problem page was faster than trying to swap it on every page. This led to the quick discovery that it offered no performance improvements.
The lack of improvement might have been predictable: the problem was related to database query performance, while pagy’s advertised benefits are about the application process and memory usage. However, trying pagy was faster and more convincing than engaging developers in a theoretical conversation.
When one mobile platform became unsuitable due to maintenance concerns, several alternatives were considered. The theoretically most promising option was evaluated by replicating small features in an app for a new platform.
The features were sufficient to prove the benefits the team hoped for (ease of porting existing code) and explore risks like how deployment would change.
Learning happened far faster than if they had attempted to implement every feature in the new platform before thinking about deployment. If the experiment had shown the new platform to be unsuitable, they could have stopped and evaluated other options instead.
A Ruby on Rails monolith became overwhelming for a team who needed to act with extreme caution in order to avoid database outages. A distributed architecture with integration via message broker was considered. The team moved clients used by logistics users to the new architecture, but other users such as customers and operations continued to be served by the original architecture.
Lessons from the experiment included how to host the message broker, how to monitor and debug, how it should behave when application processes restarted, and how to integrate with Rails. It was possible to wait until the single audience was well-served by the new API before considering any further adoption of the message broker.
In a web app a requirement for integration with a bluetooth printer was identified as an early risk. The feature that needed to print was far down the roadmap, but the developer built a “test print” button as one of the first actions in the project. This validated the high risk part of the feature (“print anything via bluetooth”) as early as possible. If the developer had waited to build the entire feature, the lessons from the printing experiment would only have been gained after completion of low risk items like assembling the correct data, UX for initiating printing, or print formatting.
Sometimes teams believe that due to a plan with a tight schedule, they have no time to experiment. This is self-defeating. The reality is that risks will materialise, and with experiments a team can face them quickly and with time to adjust. Without experiments, the team faces the same risks, but only when it is too late to do anything about them.
This article’s focus is largely on demonstrating the benefits of changing attitudes to planning - plans should be constantly updated based on what we learn during implementation - but there are some skills necessary to make this approach successful.
It’s useful to understand common patterns for experimentation that the tech industry follows, including:
It’s also important to be able to evaluate the success of experiments. Some businesses may have business intelligence teams and tools which can be enlisted for this, but should not become a crutch or bottleneck. Developers outside of those teams (or where no such team exists) can also learn how to evaluate the impact of their own work1.
Experiments help us to evaluate risks fast, choose between competing solutions, build consensus, and spread knowledge. They can be as small as a few hours, or take months to demonstrate how to work effectively with a new message broker. Building learning into a plan via experiments is far better than learning the same lessons at a time when it is too late to make different choices.
the linked article includes practical advice for Ruby on Rails developers, but the mindset it encourages is universal. ↩
Software teams often make incorrect assumptions about user feedback. Being aware of how differently one person can interpret something from another’s intent is powerful.
The bulk of this article is a list of many possibly incorrect assumptions. In most cases they are true, but assuming that they are always true leads teams to over- or under-reactions: dismissing a serious issue, or losing productivity in a scramble to meet imagined emergencies.
The list is in the tradition of many other lists of “falsehoods programmers believe”. In the genre a list of common simplifying assumptions is presented with the intention of making the reader aware of edge cases. You can see a collection of similar articles here.
The list below may seem absurd at times, but rest assured that they are based on real life circumstances. The common thread is that people in the detail (software teams) communicate very carefully about their software, and lose sight of communication patterns in other spheres. Insiders assume:
… that is not a given in normal communication.
Productive relationships form when there is some give and take where the most involved users start to adopt some of the team’s communication style, and the team learns to more carefully assess communication from those further away. This is doubly true with internal users and layers of management.
And finally, here is the list.
These statements might usually be true, but they are not always true. Ignore this at your peril.
]]>pg_stat_activity view) is telling you that one particular query in your Rails app is causing heavy load. Unfortunately, it already seems pretty fast. What are the odds that you can get it to run any faster?
I had a case where the query generating the heaviest load in an application took 20 milliseconds. It was also the fastest query in the application; some queries in the same application take entire seconds to run! Surely the fastest query in the system had little room for optimisation?
It’s possible to get really good, detailed answers about duration and optimisation strategies for specific queries using a database query planner, but sometimes it’s nice to have a rule of thumb that can guide one to a quick answer like: “this query’s duration is [or is not] very close in speed to the fastest my query could possibly run and so I likely therefore can [or cannot] optimise it”.
sometimes it’s nice to have a rule of thumb, a quick answer like: “this query’s duration is [or is not] very close in speed to the fastest my query could possibly run and so I likely therefore can [or cannot] optimise it”
Here’s a trick you can run in the Rails console of your production environment to find out how fast the fastest possible queries can run:
> ActiveRecord::Base.logger = Logger.new(STDOUT) # assuming you have SQL logging turned off
> ActiveRecord::Base.connection.execute("SELECT 1;")
D, [2025-03-21T14:03:35.765188 #8173] DEBUG -- : (0.7ms) select 1;
In this snippet we’re turning on ActiveRecord SQL logging (for this Rails console only) as it is normally disabled in production. Then we’re running a query that does not touch any real data. The resulting logging shows that our query takes 0.7ms.
SELECT 1; is the minimum work that can be done in the database1. The only work that happens is Rails assembling a SQL request, connecting to the database (which is normally on a different server), and returning an answer. Any “real” query must run slower than that because it has to search for and return real data.
It’s interesting to compare this number with the time it takes for a minimal network request to go from application server to database server.
If you have terminal acess to your application server, you could use the ping utility to get that number. If you’re in the cloud, you might need to rely on sites like cloudping.co (for Amazon) to tell you how long a network round trip takes. The 50th percentile cloudping time within my AWS region is reported to be 1.7ms, so this minimal query is faster than the average round trip time.
This was a real life example: I had previously thought that my 20ms query was pretty much as fast as things could get, but once I realised that a “fastest possible” SQL query could go much faster, I was emboldened to try optimise. I was able to drop the query time drastically: in fact, ActiveRecord now reports that the query takes 1 millisecond or less.
Improving my query time by 20x was a pretty good outcome for a case that started as “I’m not sure if it’s possible to do any better than what I have already”. Thank goodness for learning more about “possible”.
There is likely a database expert out there who can identify a query that does even less work. The point is that this query definitely does less than any useful query that my application could generate. ↩
config.active_record.query_log_tags_enabled. The feature adds comments to the SQL queries that Rails generates, like this:
/*action='index',application='MyApp',controller='myrecords'*/
SELECT "myrecords".* FROM "myrecords" WHERE "myrecords"."user_id" = 200 AND...
/*application='MyApp',job='myrecordjob'*/
SELECT "myrecords".* FROM "myrecords" WHERE "myrecords"."user_id" = 199 AND...
For readability I’ve split the comments and the queries into separate lines (but they’ll be on a single line in your DB logs) and I’ve turned on ActiveRecord::QueryLogs.prepend_comment = true so that the comments appear before the query. By default the logs are added at the end of the query.
These comments are visible in Rails application logs, but that’s not very helpful on its own because many production Rails deployments will decrease log levels to the point where ActiveRecord queries are invisible.
More importantly, the comments are actually sent with the SQL queries to my database server, so anything monitoring my queries will also see this metadata.
This is tremendously useful for working with database monitoring like Amazon’s RDS Performance Insights, where I can see what SQL queries are causing the biggest problems, but I can’t identify what application code generated those queries.
By default Rails will log the application, controller, controller action, and (under ActiveJob) job name1. That’s pretty good for most cases, but many Rails apps generate queries that aren’t usefully described by this data. The two I’ve seen so far are Sidekiq jobs running without ActiveJob, and GraphQL with the graphql-ruby gem, which will generate lines like this:
/*action='execute',application='MyApp',controller='graphql'*/
SELECT "myrecords".* FROM "myrecords" WHERE "myrecords"."user_id" = 200 AND "myrecords"."saved" = TRUE
No matter what graphQL operation is being executed, the generated SQL queries all receive the same comment, because GraphQL requests offer infinite variety from the same endpoint.
graphql-ruby offers tracing functionality, which can be enabled like this:
class MySchema < GraphQL::Schema
# normal schema stuff
mutation(Types::MutationType)
query(Types::QueryType)
# tell graphql-ruby what code to use for tracing:
trace_with(QueryTrace)
end
The module passed to trace_with just needs to implement a graphql-ruby lifecycle hook:
module QueryTrace
def execute_field(**args)
if args[:field].resolver
ActiveSupport::ExecutionContext.set(graphql_resolver: args[:field].resolver)
end
super
end
end
There is some nuance here about which of several GraphQL lifecycle hooks to choose that I’ll discuss later. Now that this data is in ExecutionContext, I can configure query_log_tags to use it:
# in config/application.rb
config.active_record.query_log_tags_enabled = true
config.active_record.query_log_tags << :graphql_resolver
Now my SQL queries generated by graphQL operations look like this:
/*...,graphql_resolver='Queries::AppVersions'*/
SELECT "app_versions".* FROM "app_versions"
/*...,graphql_resolver='Queries::SavedRecords'*/
SELECT "myrecords".* FROM "myrecords" WHERE "myrecords"."user_id" = 200 AND "myrecords"."saved" = TRUE
There are some things worth knowing before following this advice.
The QueryTrace#execute_field method is invoked once for every field in a query, but most likely only has an explicit resolver object to log about for root nodes:
query MyQuery {
savedRecords { # savedRecords has explicit resolver Queries::SavedRecords
# id and saved are implicitly resolved
id
saved
}
}
execute_field will execute once for each field here, but ActiveSupport::ExecutionContext.set calls are ignored when nil is passed as a value. With this GraphQL query, I’ll only get a tag for Queries::SavedRecords, but that should still be enough detail to narrow down my search a lot.
For what it’s worth, when I enabled this on a GraphQL API that receives 50 queries per second, logging per field did not increased the response times.
If your traffic is much heavier than that, or you’re letting clients send particularly heavy queries, you might want to investigate a lifecycle hook that is invoked less often than execute_field. execute_query or analyze_query (see the documentation) execute only once per query, but here’s why I don’t use them:
execute_query time does not include resolvers, so I’d have to log something like graphql_root: query.selected_query.selections.map(&:name), which is a lot of graphql-ruby implementation detail to depend on.graphql_root='savedRecords,appVersions'.GraphQL offers GraphQL::Tracing::ActiveSupportNotificationsTrace, which would then allow me to hook into ActiveSupport notifications like graphql.execute_field, rather than writing my own custom QueryTrace module. Unfortunately I found the timing of the ActiveSupport notifications to be unreliable, sometimes running after a query completed (and so the SQL query was generated without the added tags).
If I had my way, active_record.query_log_tags_enabled would be enabled by default in Rails, but I gather that it disables prepared statements, which are important for some applications. For my own applications, adding it is an automatic must, along with the additional configuration necessary for cases like GraphQL or sidekiq.
Recent versions of Rails (7.2 and 8+) also include support for optionally logging :source_location which will output file and line number of the query. I haven’t used this option hence not discussing it here, but I’d view it as complimentary to the logging suggested here rather than a replacement. The linked PR also contains a performance warning about enabling it in production. ↩
Many people use the word “refactoring” incorrectly. If we’re embarking on a change that is not really refactoring (for example looking at a bug or an adjustment after a third party change), we can’t fix it with refactoring.
What to do instead: we need to think and talk about it differently from refactoring. We can stop and consider how the system will change (from what to what) and raise it with teammates to discuss why it matters, what action to take, and when.
We often reach a point during refactoring where it seems like there is an easy improvement that applies to almost all cases. It’s usually better not to impose additional abstraction if it only matches “almost” all cases.
Here’s an example inspired by the Noisy Animals kata. Consider this code:
def make_noise(loud: true)
noise = {
'leopard' => { quiet: 'growl', loud: ['growl', 'growl'] },
'owl' => { quiet: 'hoot', loud: ['hoot', 'hoot'] },
'eagle' => { quiet: 'caw', loud: ['caw', 'caw'] },
'mouse' => { quiet: "\n", loud: ["\n"] },
'snake' => { quiet: 'slither', loud: ['hiss'] },
}.fetch(species)
puts noise.fetch(loud ? :loud : :quiet)
end
Those repeated strings on some lines might be irritating, but two lines don’t share the repetition. We’re better off leaving the code as is than trying something like this:
def make_noise(loud: true)
noise = {
'leopard' => 'growl'
'owl' => 'hoot',
'eagle' => 'caw',
'mouse' => "\n",
'snake' => 'slither'
}[species]
if loud && species == 'snake'
puts 'hiss'
elsif loud && species == 'mouse'
puts "\n"
elsif loud
puts noise, noise
else
puts noise
end
end
Here we’ve moved from a single ternary conditional to an if statement with four branches. Knowledge of how each species behaves was contained in a single line but is now split up over the entire method. The method is also almost double its original length.
What to do instead: perhaps there is a different code smell we can investigate, or maybe we need to stop where we are until a new requirement demonstrates the need to change.
We’re often confronted with more than one code smell at a time. Perhaps a piece of code seems like it should be extracted to a method, and inside that code is an unnecessary local variable. Create the method and replace all occurrences of the original code with the method call before you remove the local variable. Alternatively, remove the variable before you extract the code into a method. After you finish one, save, run the tests, and commit, all before you move on to the next.
Whatever you do, don’t try to do both at the same time. Doing them separately may seem like a hassle, but the risk of a typo and the benefit of knowing that you started each change from the safety of green tests are worth much more than the hassle (and there are techniques you can learn to reduce the hassle).
What to do instead: live with the code that isn’t ideal long enough to see the results of your current refactor. The current refactor may help you realise something new about the code that you would have missed if you tried to do both at the same time.
There’s no benefit to improving code that never changes, even if it is highly complex. This is the basis of Michael Feather’s process for identifying good refactoring targets (spoiler: code that is complex and changes often is the best place to look).
We should be aware in this case that it is really easy to apply wishful thinking: we imagine that the code that everyone is scared of will never change, simply because we cannot think of how to improve it.
What to do instead: find something else that is more likely to change to refactor, or pause refactoring and move on to the next requirement.
If we’re changing tests (or we have none), we have no guarantee that behaviour is unchanged. If we are possibly changing behaviour, we are not refactoring.
Here’s a delightful extract from Refactoring Javascript:
HOW CONVERSATIONS ABOUT REFACTORING SHOULD GO UNTIL TESTS ARE WRITTEN
“I refactored login to take email address and username.”
“No, you didn’t.”
“I’m refactoring the code to __”
“No, you aren’t.”
“Before we can add tests, we need to refactor.”
“No.”
“Refactoring th–”
“No.”
“Refa–”
“No.”
What to do instead: Instead of “refactoring” without tests, we can start by writing some tests! Writing new tests will help us learn about the code. After that, we can come back to refactoring.
Sometimes we tidy the code to a certain point and realise that there is something we don’t understand. That’s ok, we can stop where we are now. If we’ve been working in small low-risk steps the way that the refactoring community encourages, the tidying we did already was either valuable and should be kept, or is really safe and easy to revert.
From Martin Fowler’s description of the Refactoring book: “its essence is a series of small behavior-preserving transformations… You also avoid having the system broken while you are carrying out the restructuring - which allows you to gradually refactor a system over an extended period of time.”
What to do instead: Simply pause where we are and continue another time when we know more. That “more” may be something about our product that reveals that some code is overly complex or unnecessary. It might also be a new requirement that shows us a new way that the code is likely to change in future.
We’ve discussed several reasons not to refactor. Ironically, we might end up doing more refactoring as a result, because we won’t be trying to use refactoring to solve unsuitable problems. The refactoring we do will be faster, have a bigger impact, and be more rewarding.
]]>It is available in Ruby and Javascript. The code and instructions are all in the github repository.
The exercise owes a substantial debt of inspiration to the exercise and ideas in Sandi Metz and Katrina Owen’s 99 Bottles of OOP, as well as to the Gilded Rose kata.
When code is continuously updated with only the smallest change possible and never reorganised, it often degrades to a level equivalent to (or worse than) this exercise’s starting point.
Although the example is small and the domain of animal noises trivial, the form of the code and particularly the techniques for improving it are very relevant to professional code.
99 Bottles is an amazing book and this exercise is intended to expose some of its ideas (and those of the refactoring community in general) in a shorter form that nudges one towards refactoring without assuming the reader has ever heard of the concept.
99 Bottles is itself very approachable for beginners, but it tends to only be read and discussed by people who are already in (or are connected to others) in the refactoring community.
The Gilded Rose Kata is a wonderful exercise, but its World of Warcraft domain is less and less approachable with the passage of time, even in the tech community.
The exercise can no doubt be improved. If you think something about the exercise is less than ideal or you have a new idea for it, you’re welcome to open an issue or a pull request on the repository.
]]>Done well, refactoring is a low-risk investment in future productivity, by reorganising code without changing its behaviour. Often however, our conversations position work that doesn’t meet this description - and needs more scrutiny - as refactoring. Let’s look at what goes wrong when this happens.
As an aid to conversation, let’s start with some examples of misuse (based on real team conversations):
All of these conversations described important work that I don’t mean to discourage. Still, none of them was really about refactoring. This use of the word “refactoring” seems to signal that the speaker is thinking technically, but not about what refactoring really implies.
Refactoring is a technical word typically describing a series of quick, well-understood, easily reverted changes that a single developer (or pair) makes. German’s post describes this really well.
Misusing refactoring to describe work that doesn’t fit that description avoids important conversations about:
After a while, long periods of work with no result creates distrust. People learn to stop listening (or become hostile) when “refactoring” comes up. That increases the chance of failure even further, because real refactoring is a powerful tool, and teams lose out when they dismiss it.
Ideally we would be successful at both refactoring and other kinds of change. Below are two ideas that can help.
Remembering that the examples above are not refactoring but are still important work, here is some improved wording for a better conversation:
You may notice that there is more detail in the reworded examples. That’s intentional. When we stop using “refactoring” with an audience who don’t understand the word, we’re more likely to get into helpful specifics. A listener is more likely to ask: “why do I need this change and not something else?”, or “what does success look like?” when meaning is not hidden.
If we understand refactoring correctly, we’re not going to misuse it in conversation, and we stand more chance of success when we do need to refactor. Experience has taught me that refactoring without proper understanding is very likely to fail. For this I’ll point you for a third time to German’s blog post; for further reading it’s worth going into the books he cites.
We need to be careful not to claim that we’re refactoring when we’re really making bug fixes, adding new features, or otherwise changing the code’s behaviour and our product’s user experience. Disguising that work as refactoring sabotages important conversations and creates suspicion about the concept. We can fix this by getting more specific in our language and by learning more about what refactoring really is.
]]>