Like a river flows: Upstream and Downstream

August 24th, 2007

Thanks Troy for blogging about this and saving me from having to say it first and turning into a flame magnet (although the responses from your blog tell me that I’m to paranoid). Now I can just blog and say that I’m reacting to you. :)

[Link to Troy's Ars Technica article]

The basic point of his article is that there exists a very complex relationship between distributions (downstream) and KDE or some other projects (upstream). Distributions make changes and apply patches to upstream software. This is reasonable as they might need to fix or change things in order to produce a fully working operating system. And presumably, in the spirit of openness and cooperation, distributions send these fixes and patches upstream so that the software writers could apply those fixes, which will eventually trickle down to other downstream projects, creating one big happy family. Right? Alas, life is more complicated than that.

According to Troy, the problem starts when distributions make heavy, not just minute bug fixes or changes, customizations to upstream software. This could include, among other things, replacing the default backends of some apps, replacing apps themselves, or big changes in the menus and user interface. Unfortunately, not all these changes arrive upstream, for a variety of reasons, creating a version of the software that is less and less like the default upstream version, usually called the vanilla version (which Troy amusingly refers to as “virgin” packages :P). And this creates a variety of problems:

  1. Distributions grow more divergent from each other, which isn’t really a bad thing in itself. It does, however, make user support and administration more difficult.
  2. Downstream versions also become very different from the original upstream versions, both internally and externally. Some would call it forking already.
  3. It creates a gap between downstream and upstream.

Sadly, this is one of those situations where both sides are right, yet still don’t see eye to eye. Distributions have the right to make changes and customizations. They are creating, distributing, and presenting a complete, functional, and integrated operating system, not just a jigsaw puzzle of software. Not only do they need to make sure that everything runs, they must also make sure to deliver a great (not just good) user experience for their intended audience. If that means creating tools and modifying apps, then their changes are justified if it means catering to the particular needs of their users. However, at the same time, upstream developers also have the right to express concern, anxiety, disappointment (or joy) over changes that are made downstream. It is their app, their pet project, their names, their integrity, that are also on the line. They also believe that the design decisions that they have made actually address the needs and concerns of their own target users. Distributions want to exert their right to modify and might see upstream as meddling or imposing. Upstream wants downstream to resemble upstream more and might see distributions as forking or too divergent.

So what really is the problem and what could be the solution? I haven’t been in the Free and Open Source Software world for long, but, in my mind, one key component of whatever solution would be is communication, on a technical level and on a personal/social level.

It’s probably not so easy to communicate between downstream and upstream. Downstream has the problem of having so many upstream sources and communicating with each and every one of those would be inefficient. Upstream has the problem of havin