Breaking Backwards Compatibility is EVIL

By Tony Marston

18th December 2005
Amended 27th January 2006


Like many PHP programmers I was pleased with the new features that came with PHP 5, and I awaited the release of 5.1, with its wide array of 'improvements', with eager anticipation. Like many I was disappointed that the 5.1 release date seemed to be the subject of one delay after another. Then with new releases of PHP in the 4.4.x and 5.0.x series there started to be complaints that scripts which had run for years were now throwing up errors, that 'backwards compatibility' (known in some quarters as 'BC') was broken. Many sites did not relish the idea of having to spend an unknown amount of time in upgrading their scripts, so they either reverted to the previous working version or, after hearing the horror stories, never upgraded to the new version at all.

Then in various newsgroup posts and blogs I started to hear rumours about the future direction of PHP, and I have to say that I did not like what I heard. I started to read the PHP mailing list for the internals group where the core developers correspond between themselves. These are the people who actually work on the language core itself rather than those who write programs with it, like me. The attitude from some people seemed to be along the lines of:

Now that we've broken BC just a little bit why don't we go all the way and break it completely. We have a golden opportunity to get rid of all that legacy rubbish and make the language cleaner and purer and more up-to-date.

The fact that this would break thousands of scripts running on millions of sites did not seem to phase these people one little bit.

Those people with legacy code that won't run under version 5 can stick with version 4. They will then have plenty of time to rewrite everything 'properly' when version 6 comes out. Perhaps we could even fork the language into two separate branches so that those people with legacy code that they're too lazy to upgrade can still run with a 'dirty' version while the rest of us can work with a cleaner, purer version.

Amongst the suggestions have been:

In the remainder of this document the term 'you' is addressed to PHP's core developers, the producers of the language, not the users of the language like me.

If you want PHP to be considered as more than a 'toy' language then you must stop behaving like petulant children. If you want PHP to be accepted into the big league, to earn the reputation of being 'enterprise ready', then you must act like you belong in the big league instead of the local kindergarten.

You are not improving the language, you are killing it. If you are not careful you will end up just like the doctor who said 'The operation was a success, but the patient died'. PHP 5 has been available for 18 months, yet has only been accepted by about 5% of web hosts. Why is this? Simply put, it is the fear of breakages in backwards compatibility. Hosting companies will not upgrade if it means upsetting their customers.

You are small in number, maybe several dozen, but according to the Netcraft statistics there are over 22 million PHP domains out there. That means that you are outnumbered by several 100,000 to 1, so your personal opinions are no more significant than a grain of sand in the desert. The fact that you are unpaid volunteers is irrelevant. It is no excuse for an unprofessional or irresponsible attitude. You volunteered your time to assist the greater PHP community, so if you are unprepared to satisfy the needs of that community you should withdraw.

You are the producers of a product, the PHP language, and we developers are your customers. We in turn use PHP to produce a variety of software applications for our customers. In my many years in the IT industry I have been the customer of many software products such as operating systems, compilers and editing tools, and as a programmer I have been the producer of many applications which have serviced the needs of my customers. In that time I have experienced many upgrades, both as a consumer and a producer, and I can safely say that an upgrade is expected to contain only two things:

  1. Bug fixes.
  2. New features.

Please notice that 'breaking existing features' does NOT appear in that list. The ONLY time it is permissible to break an existing feature is where something is found to be the cause of an error, and the only way to deal with the error is to prevent it happening in the first place. In such circumstances both the cause and effect of the error should be fully documented and explained so that the customer knows exactly what is going on and why. In other words the developers must provide justification for breaking backwards compatibility. Where possible an alternative method should be identified, as in "doing it this way causes so-and-so problem, so we had to prevent it, but doing it in this other way is perfectly safe".

Note here that the removal of undocumented features can only be justified if it is balanced with the continuation of features which are documented. It would not be fair to have one without the other.

In my long experience of dealing with upgrades, both as a consumer and a producer, may I offer some golden rules that you would be well advised to give serious consideration:

The customer is always right
Customers are valuable. Without customers actually using a product, that product is useless and might as well not exist. If enough customers want a particular enhancement, then it is the developer's job to provide that enhancement. If different customers want different methods to achieve the same result, then it's the developer's job to provide those different methods. It is not acceptable for the developers to say "We're sorry, but we are only prepared to write enough code for one method, so more than one method is out of the question". It is the developer's job to write code to satisfy their users' requirements, not to invent excuses why they shouldn't write code.
One man's meat is another man's poison
The product exists to keep its customers happy, not to keep its developers happy. Providing a solution is the primary concern. Providing an elegantly-coded solution, or a technically pure solution, is secondary. Besides, different developers have a totally different notion of what is, or is not, "technically pure". What is neat to some is nauseating to others. To put it another way, One man's purity is another man's putrefaction. So saying that you want to break backwards compatibility for no other reason than "code purity" is unlikely to win friends among the user community.
If it ain't broke, don't fix it
This saying originated in the original engineering world before it was applied to the creation of software, but it is just as apt regardless of the setting. It means that you should only expend effort where effort needs to be expended. To put it another way, If it doesn't cause a problem it doesn't need a solution. If you not are modifying a piece of code to fix a bug or to add a new feature then what you may actually be doing is introducing a new bug. It is a recognised phenomenon in the non-software engineering world that a significant number of faults occur as a direct result of preventive maintenance (PM). This is a type of maintenance which, instead of being in response to a fault, is scheduled at otherwise quiet times to help prevent faults. Unfortunately the physical act of disturbing a delicate piece of engineering in order to give it a clean-up is also likely to disturb the gremlins. That is why an experienced engineer would rather have a dirty engine that works than a clean engine that doesn't.

Image two software products:

Which product has more value? Customer perception is much more valuable than developer perception, so building something that satisfies the needs of the users takes priority over building something that pleases the developers. If all software products were delayed until the developers were satisfied that it was technically "pure" I don't think many products would actually be released.

A product is developed for the benefit of its customers, not for the convenience of its developers. Working on a product which is technical crap but which sells like hot cakes is better than working on a product which is technically pure but which no one wants. You may, over a period of time, be able to clear up some of the technical mess, but if you break anything in the eyes of your customers then that is a step too far. Too many breaks and they will drop your product and take their business elsewhere.

As to that list of suggested "fixes", here are some of my personal comments:

To sum up, here are my Ten Commandments regarding software upgrades:

  1. Don't break backwards compatibility.
  2. Don't mess up the product for 99.9% of users just to provide a "cool" feature requested by 0.1% of users. Some "cool" features can be abused, by both novices and experts alike, to produce a real can of worms.
  3. Don't break backwards compatibility.
  4. If you have to change the way that something behaves then provide a switch (such as a setting in the ini file) so that existing code can still run with the 'old' behaviour instead of producing unexpected results with the 'new' behaviour. Make the 'old' behaviour the default so that those customers who don't want the 'new' behaviour don't get caught out because they forgot to flick the behaviour switch. It is the people who actually want the new behaviour who should be forced to flick the behaviour switch.
  5. Don't break backwards compatibility.
  6. If you cannot introduce new functionality without breaking existing code then consider creating a new set of API's for the new functions. Development (but not bug fixing) on the old API's should be stopped, but existing code using those functions will continue to run without any problem. If someone wants access to the new functionality then they must change their code to use the new API's. Any claim that this causes a maintenance burden is utter rubbish as only the new code should need any maintenance.
  7. Don't break backwards compatibility.
  8. Don't forbid something just because you don't think it should be done that way, or because a function is being used in a way other than originally intended. Unless it causes something to break it doesn't need fixing. You may think that it's a silly thing to do, but the customer wants to do it that way, and the customer is always right. I have seen developers waste time writing code that prevents a function from being used in a certain way even when that usage did not cause any breakages. Trapping imaginary errors is almost as annoying as not trapping real errors. The philosophy should be that everything is legal unless there is a good reason why it should not be so, and not the reverse where everything is illegal unless there is a good reason why it should be.
  9. Don't break backwards compatibility.
  10. Do not remove functionality just because the same result can be achieved by other means. Many languages provide alternate methods to achieve a particular result, and it is up to the customer, not the supplier, to decide which one to use. You may think that having two methods means that one of them is redundant, but if it's being used anywhere IT IS NOT REDUNDANT!
  11. Don't break backwards compatibility. (What? You've never heard of the 11th Commandment?)

One final word. Breaking backwards compatibility is not like breaking a promise, it is more like breaking a leg - very painful, and it puts you out of action for some while. So if you core developers promise not to break backwards compatibility with our favourite language, we, your customers, promise not to break your legs. Deal?

STFU - stands for Shut The F*** Up!

© Tony Marston
18th December 2005

Amendment history:

27th Jan 2006 Moved comments regarding case-sensitivity to a separate document.