Otdfbt

I'm currently working on a bigger project which unfortunately has some files where software quality guidelines where not always followed. This includes big files (read 2000-4000 lines) which clearly contain multiple distinct functionalities.

Now I want to refactor these big files into multiple small ones. The issue is, since they are so big, multiple people (me included) on different branches are working on these files. So I can't really branch from develop and refactor, since merging these refactorings with other peoples' changes will become difficult.

We could of course require everyone to merge back to develop, "freeze" the files (i.e. don't allow anyone to edit them anymore), refactor, and then "unfreeze". But this is not really good either, since this would require everyone to basically stop their work on these files until refactoring is done.

So is there a way to refactor, don't require anyone else to stop working (for to long) or merge back their feature branches to develop?

You have correctly understood that this is not so much a technical as a social problem: if you want to avoid excessive merge conflicts, the team needs to collaborate in a way that avoids these conflicts.

This is part of a larger issue with Git, in that branching is very easy but merging can still take a lot of effort. Development teams tend to launch a lot of branches and are then surprised that merging them is difficult, possibly because they are trying to emulate the Git Flow without understanding its context.

The general rule to fast and easy merges is to prevent big differences from accumulating, in particular that feature branches should be very short lived (hours or days, not months). A development team that is able to rapidly integrate their changes will see fewer merge conflicts. If some code isn't yet production ready, it might be possible to integrate it but deactivate it through a feature flag. As soon as the code has been integrated into your master branch, it becomes accessible to the kind of refactoring you are trying to do.

That might be too much for your immediate problem. But it may be feasible to ask colleagues to merge their changes that impact this file until the end of the week so that you can perform the refactoring. If they wait longer, they'll have to deal with the merge conflicts themselves. That's not impossible, it's just avoidable work.

You may also want to prevent breaking large swaths of dependent code and only make API-compatible changes. For example, if you want to extract some functionality into a separate module:

1. Extract the functionality into a separate module.


2. Change the old functions to forward their calls to the new API.


3. Over time, port dependent code to the new API.


4. Finally, you can delete the old functions.


5. (Repeat for the next bunch of functionality)

This multi-step process can avoid many merge conflicts. In particular, there will only be conflicts if someone else is also changing the functionality you extracted. The cost of this approach is that it's much slower than changing everything at once, and that you temporarily have two duplicate APIs. This isn't so bad until something urgent interrupts this refactoring, the duplication is forgotten or deprioritized, and you end up with a bunch of tech debt.

But in the end, any solution will require you to coordinate with your team.

Do the refactoring in smaller steps. Lets say your large file has the name Foo:

1. Add a new empty file Bar - commit it to "trunk".




2. Find a small portion of the code in Foo which can be moved over to Bar. Apply the move, update from trunk, build and test the code, and commit to "trunk".




3. Repeat step 2 until Foo and Bar have equal size (or whatever size you prefer)

That way, next time your team mates update their branches from trunk, they get your changes in "small portions" and can merge them one-by-one, which is a lot easier than having to merge a full split in one step. Same holds when in step 2 you get a merge conflict because someone else updated trunk in between.

This won't eliminate merge conflicts or the need for resolving them manually, but it restricts each conflict to a small area of code, which is way more manageable.

And of course - communicate the refactoring in the team. Inform your mates what you are doing, so they know why they have to expect merge conflicts for the particular file.

You are thinking of splitting the file as an atomic operation, but there are intermediate changes you can make. The file gradually became huge over time, it can gradually become small over time.

Pick a part that hasn't had to change in a long time (git blame can help with this), and split that off first. Get that change merged into everyone's branches, then pick the next easiest part to split. Maybe even splitting one part is too big a step and you should just do some rearranging within the large file first.

If people aren't frequently merging back to develop, you should encourage that, then after they merge, take that opportunity to split off the parts they just changed. Or ask them to do the splitting off as part of the pull request review.

The idea is to slowly move toward your goal. It will feel like progress is slow, but then suddenly you'll realize your code is a lot better. It takes a long time to turn an ocean liner.

I'm going to suggest a different than normal solution to this problem.

Use this as a team code event. Have everyone check-in their code who can, then help others who are still working with the file. Once everyone relevant has their code checked in, find a conference room with a projector and work together to start moving things around and into new files.

You may want to set a specific amount of time to this, so that it doesn't end up being a week worth of arguments with no end in sight. Instead, this might even be a weekly 1-2 hour event until you all get things looking how it needs to be. Maybe you only need 1-2 hours to refactor the file. You won't know until you try, likely.

This has the benefit of everyone being on the same page (no pun intended) with the refactoring, but it can also help you avoid mistakes as well as get input from others about possible method groupings to maintain, if necessary.

Doing it this way can be considered to have a built-in code review, if you do that sort of thing. This allows the appropriate amount of devs to sign off on your code as soon as you get it checked in and ready for their review. You might still want them to check the code for anything you missed, but it goes a long ways to making sure the review process is shorter.

This may not work in all situations, teams, or companies, as the work isn't distributed in a way that makes this happen easily. It can also be (incorrectly) construed as a misuse of dev time. This group code needs buy-in from the manager as well as the refactor itself.

To help sell this idea to your manager, mention the code review bit as well as everyone knowing where thing are from the beginning. Preventing devs from losing time searching a host of new files can be worthwhile to avoid. Also, preventing devs from getting POed about where things ended up or "completely missing" is usually a good thing. (The fewer the meltdowns the better, IMO.)

Once you get one file refactored this way, you may be able to more easily get approval for more refactors, if it was successful and useful.

However you decide to do your refactor, good luck!