There’s battle lines being drawn
Nobody’s right if everybody’s wrong
— Buffalo Springfield, “For What It’s Worth” (Stephen Stills)

Tom Adelstein, editor of Lxer.com and an open-source software consultant, makes a good point in this O’Reilly weblog post, Why does Microsoft Need a “Get the Facts” Campaign?:

If Microsoft offers a superior product why do they need a “Get the Facts” campaign? Almost any time a publication runs a Linux story, Microsoft places a “Get the Facts” ad right in the middle of the story. Do you ever wonder if publications run Linux stories just to garner Microsoft’s ad money?

Too many studies on the total cost of ownership (TCO) of Windows versus Linux have arrived at vastly different conclusions.

So… everything you know is wrong (with apologies to Firesign Theater). It’s difficult to trust any of the research — whether it is funded directly or indirectly by Microsoft or Microsoft partners and consultants, or by those claiming to be “independent” — because conclusions vary widely.

In the past, the press has reported many a study’s findings straight from a Microsoft press release without having access to the entire text of the study — and the fine print often told a different story. For example, in one white paper sponsored by Microsoft, tech industry analyst firm International Data Corp. (IDC) made at least three observations that were not favorable and were not reported by the press: Linux is more reliable, Linux is cheaper for web hosting, and Microsoft enjoys only a temporary advantage in support staffing. But Microsoft neglected to include these key findings in its official press announcement, so most people never read about them. (See Microsoft-sponsored study on Win2K vs. Linux is NOT all good news for Microsoft by Frank Scavo in The Enterprise System Spectator.)

As in politics, the battle over truth in this type of research is usually won by the biggest spender. Which explains why Microsoft — a company with a market cap of about US $266 billion and something like US $38 billion in cash — continues to perpetuate its embarrassing “Get the Facts” marketing campaign against Linux. It’s chief Linux competitors, Red Hat and Novell, have a combined market cap of less than US $7 billion. Microsoft can continue this propaganda indefinitely, as it is relatively cheaper than restructuring its server software business model.

Weird Comparisons from the Gold Mine

Historically Microsoft has been on the defensive about this marketing campaign. In Aug. 2004, the company had to pull its ads claiming that the total cost of ownership (TCO) with Linux was greater than with Windows. The Advertising Standards Authority in the U.K. ordered Microsoft to pull an ad based on a server test conducted by META Group because Microsoft’s claim — that Linux is more than 10 times more expensive than Windows Server 2003 — wasn’t supported by the tests. (See Microsoft Ordered to Pull Anti-Linux Ad in internetnews.com.) The test pitted Linux running on the more expensive and slower IBM z900 mainframe CPUs against Windows Server 2003 running on a set of cheaper, faster 900MHz Xeon CPUs. Of course it would have made more sense to run the two operating systems on the same hardware, but that wouldn’t have supported Microsoft’s claim.

Microsoft clearly enjoyed promoting its independent research, claiming that The Yankee Group had found that switching to Linux from Windows was “prohibitively expensive, extremely complex [and] provides no tangible business gains.” This analysis was contradicted in April, 2005. Yankee Group senior analyst Laura DiDio found that most U.S. businesses say there is almost no difference between the costs of maintaining Windows-based versus Linux-based corporate servers.

But wait, there’s more. According to Microsoft gets more ‘facts’ for anti-Linux campaign by Joris Evers, IDG News Service:

However, in real life Windows and Linux are not used for the same type of tasks, according to an independent Yankee Group survey of 509 IT users released on Monday [April 4, 2005]. The hourly cost of Windows downtime in actual user environments is three to four times higher than that of Linux downtime, according to Yankee Group.

Windows downtime is more expensive because the operating system is used for more crucial tasks in businesses when compared with Linux servers, according to the Yankee Group survey… Furthermore, according to Yankee Group, Linux is not displacing Windows in server rooms, but most users are installing Linux servers parallel with Windows servers.

Perhaps the most startling survey revelation was the fact that over 50% of the respondents said they had performed a thorough TCO analysis, but when asked to calculate their specific Linux and Windows capital expenditure and maintenance costs, 75% on average, could not answer explicit questions. Who knows for sure?

Hardening Research is Key to Increasing Brand Trust

The newest white paper on Microsoft’s “Get the Facts” site is from Gartner: Costs and Benefits Still Favor Windows Over Linux Among Midsize Businesses. The published version of the study doesn’t actually measure actual costs and benefits — it measures the “primary considerations” of midsize-business CIOs. “Concerns related to TCO, better security and reliability rank high as server OS investment considerations, while licensing costs are low on the list.” From this analysis, Gartner concludes that Windows will likely dominate the server OS market until 2010. In the meantime, according to Gartner, mid-size businesses are largely conservative adopters, and the switching costs aren’t worth it.

It seems that Gartner conducted this survey to bolster its position. It looked for conservative answers and found them. It interpreted normal, everyday concerns and turned them into insurmountable obstacles. But how useful is this analysis to businesses that are fed up with paying licensing fees for software that isn’t reliable?

Another report cited in Microsoft’s campaign is The Yankee Group’s 2005 North American Linux and Windows TCO Comparison Report, Part 2: Hardening Security Is Key to Reducing Risk and TCO. Here’s a relevant passage:

Users scored Microsoft security substantially higher in this year’s survey than in the 2004 survey. In 2004, users gave Windows XP, Windows 2000 Server and Windows Server 2003 security an average rating of 3.5 on a scale of 1 to 10, with 1 being the least secure and 10 representing the most secure. In 2005, Windows achieved an average security rating of 7.6 and substantially narrowed the gap between itself and Linux, which had an average rating of 8.3.

Linux’s security ranking slipped from an average 9.2 rating from last year’s survey to a still very respectable 8.3 in the current poll.

What’s interesting is that The Yankee Group acknowledges how fleeting these scores can be: “Yankee Group cautions that security is fluid, not static. An especially pernicious hack to either environment will cause either Windows or Linux’ respective ratings to plummet in users’ estimations.”

My conclusion would be that Linux is relatively stable, with only a minor fluctuation over two years in user estimates, while the Windows score seems more like a spike that could drop tomorrow.

The meat of The Yankee Group’s study is this:

It takes network administrators 30% longer — or approximately 4 hours — to bring their Linux servers back online following a security attack, compared to a Windows server. In the majority of the cases, the fault lies not with the underlying Linux operating system but with poor documentation and support.

That sounds bad, except that the study doesn’t take into account documentation and support from established Linux vendors:

There are few aggregate sites that identify cross-platform security issues that corporate Linux users can turn to for support. Companies like Novell, which offer subscription-based support services, can improve access to documentation, upgrades and ongoing maintenance.

OK, so what if the corporate Linux user takes advantage of Novell’s (or Red Hat’s) support and documentation? That’s not factored in.

What It is Ain’t Exactly Clear

Case studies are easy to bend to support a vendor’s position — I know; I used to write them for a living. In the Radio Shack case study on Microsoft’s “Get the Facts” site, the TCO was 30% lower in hardware costs because the implementation allowed Radio Shack to continue using its UNIX-based point-of-sale terminals. Radio Shack found it easier to use Windows Services for UNIX version 3.5 to support these terminals rather than Linux. Of course, Microsoft commissioned this case study. It’s a good one, except that there are very few if any businesses in the exact same predicament Radio Shack was in.

The Rayovac study, created by the technology partner that recommended the solution (Bedrock Managed Services and Consulting), neatly substantiates its recommendation with a detailed TCO — except that it includes icensing fees for an Oracle database, additional SAP consulting fees, additional costs to cross-train employees on Linux, and a salary and benefits for a full-time Linux system administrator. The study doesn’t mention whether any of these fees and costs would be incurred with Windows, or whether the company would have considered replacing Windows people with Linux people, rather than just adding them.

You might draw the conclusion from these studies that Linux provides smaller companies — those with customized vertical applications or who have no legacy networks — with a better TCO than Windows. You might also go along with the conclusion that the vast majority of customers — and especially those that are already Windows shops — can get a better TCO with Windows. That’s what Microsoft wants you to believe.

And that’s why research studies turn out to be unbalanced and unfair. It is nearly impossible to find a study that makes sense for a particular user or business, and to trust those underwritten by Microsoft or by partners who work with Microsoft software. You have to think outside the green shrink-wrapped box.