The Steam Platform
A guest article from reader Andy Simpson.
Valve recently announced that Steam has over 13 million active accounts, excluding those that haven’t been used for a month. Even if some users actively use more than one account, this is still a fearsome number of users and seems to confirm the position of Steam as the market leader for digital distribution, but there are other services vying for position in the market.
Gamespy/IGN are in the game with Direct2Drive, Turner with Gametap, EA with EA Link, GAME with a new on demand service powered by Metaboli, and probably a couple of others. Xfire is bringing gamers IM capabilities and server browsing, Gamespy have Comrade and Microsoft is trying to bring gaming under one umbrella with Games For Windows Live.
All these services fit into slightly overlapping areas, and it’s only a matter of time before an almighty competition breaks out between them, and so I would say that we’re at, or close to, a turning point in the history of gaming on the PC platform and I think now is probably a good time to examine the origins and long-forgotten promise of Steam.
Long Distant Origins
Steam was originally intended to be a shared set of technology components that could be used by any software publisher. Reading the 2002 Steam press release, you can see that many of the ambitions listed have gone unfulfilled.
Dave Rolston, Vice President of Engineering, ATI Technologies Inc:
We will use the tools Steam gives us to create an automatic driver update and configuration management facility to deliver the most current and complete graphics drivers directly to the user, and ultimately add to an immersive and outstanding gaming experience.
5 years on, and we’re seeing the same thing promised.We’re told that:
Steam is designed to support a wide variety of models for hosting, billing, and support. Allowing for any number of partnerships and implementation, Steam was built as a modular platform.
This too, has gone unfulfilled. There is one implementation: Valve’s. There’s only one method of billing currently being used, a bog-standard one-time purchase. Hosting is all on the Valve content servers and there are barely any partnerships to speak of. Even the innovative content streaming features of Steam mostly lie unused, even by Valve games. It’s depressing that the best example right now is the original Half-Life! Even Source games don’t properly take advantage of content streaming.
Nobody uses the content streaming because it requires deeper Steam integration.
The menu 3D backgrounds for instance, are a pretty good idea. They look great, show off the game immediately, and ensure a fairly relevant set of assets are already cached into memory before you even start the game. However, combined with Steam content streaming, they’re a disaster. Try it, if you like. Delete the local content for HL2 (or move the .gcfs out of SteamApps), then re-install it and try and run the game straight away. You’ll notice it allows you to start the game with it not fully downloaded, but then will appear to hang before it even gets to the menu. The engine has realised that the map file for the background isn’t downloaded, and instructs Steam to download it. Meanwhile, it continues to display a (static) loading screen. For the duration of the download, it simply looks like the game has hung.
There are two simple solutions: require 100% of the game to download before you allow the game to start, or require Source to put up a progress bar with approximate time remaining and a working cancel button. The second is by far the better design decision; the user is kept informed, knows the game hasn’t hung, and can cancel the process and continue the download on the desktop if they decide they don’t want to wait. It’s a well-known fact in user experience design that users are a lot more patient to wait for a long running-process to complete if they’re kept informed about what’s going on. The streaming features of Steam are also then properly leveraged to get the player into the game as soon as possible.
It gets worse – the third party games currently packing out the Steam platform don’t implement this streaming at all! The Steam client basically becomes a glorified download manager for loose files. Nobody uses the content streaming because it requires deeper Steam integration. If not even Valve are implementing the full features of their own platform, there isn’t much hope a third party will. Triton attempted to achieve content streaming essentially “for free” without requiring integration by using a virtual filesystem driver, however a similar attempt by Steam has been discarded as providing terrible performance.
The Secret of Success
So if Steam is being used only in the most basic fashion, what is its competitive advantage over every other service? The other digital distribution services are building up catalogues of games almost as impressive and in some cases possibly even superior. The functionality is more or less equivalent, minus Friends and Server browsing features. The answer has already been mentioned: Steam’s fantastically large user base.
By hitching Steam to the CS/HL2 wagon, giving Steam the killer app which all the other platforms lack, Valve ensured maximum exposure for Steam as well as ensuring it became fixed in the minds of the industry as a whole. Valve are pioneers not for trying something especially new – and as we’ve seen, what is new is not being used – but for trying it BIG, dragging digital distribution kicking and screaming into the mainstream. Before it was something on the fringe, now publishers like Eidos are putting their new games up on digital distribution services as a matter of course.
What is Steam’s competitive advantage over every other service?
Steam has become fixed in the gamer consciousness; PC Gamer UK for instance is quite happy in reviews listing the publisher as “Steam”, putting the price in dollars, and leaving it at that. There’s no need to qualify it by giving the Steam URL somewhere in the text. Everyone knows what Steam is.
The situation is analogous to the placing of a retail store. Excuse the London-specific example, but if you can you’re going to set up your shop somewhere like Oxford Street where the foot traffic is the highest. Passers-by will remember that your store is there if they want something you sell, even if they don’t walk in and buy something immediately.
Steam is also analogous to a department store, selling the wares of multiple different companies. Again, I apologise for the London bias, but I’d like to use a real-life analogy. Aquascutum make clothes, and they have a store location on Regent Street, just off Piccadilly Circus. A reasonably short walk away is the Selfridges department store on Oxford Street, which has an Aquascutum section, amongst others. The benefit to this arrangement is that in Selfridges they pick up the passing trade of people looking for a broad class of related items, in this case, menswear, even though they’re presumably doing some damage to sales from their own store down the road.
Introversion sell DEFCON both from their website and from Steam; presumably they make a larger margin on sales from their own site, but the advantage of Steam to them is that they pick up the passing “foot-traffic” which had previously helped propel Darwinia to new heights.
The Defcon Dilemma
So Introversion presumably were planning on distributing on Steam for a good portion of the development time of Defcon, needing the added exposure of Steam distribution to ensure good sales, and the finished game obviously has integrated Steam authentication, because the “key” you’re provided with in-game is prefixed with “STEAM”.
However, on launch day, people were able to buy, download, install and launch Defcon, only to be presented with an authentication failure when they attempted to play. Had the Steam auth servers, capable of handling the authentication of hundreds of thousands of Counter-Strike players every day, collapsed under the weight of the Defcon release?
No. Introversion had implemented their own “metaserver” system, which promptly collapsed under the load and forced the angry buyers to play the demo version until they fixed the problem. Why was this necessary? Why could Introversion not simply license the Steam authentication service for Defcon, from wherever it was purchased (Steam, their site, or retail) and for all platforms? What happened to the dream of a modular platform available for flexible implementation and partnership with anyone?
Somewhere along the line, Valve lost sight of the original plan, and this may come around to cost them.
A Call To Action
More and more publishers, retailers and others are encroaching on the territory that Steam first staked out. None of them quite do all of what Steam does, but I fear it’s simply a matter of time. I doubt it will be long before Xfire or Microsoft starts up a digital distribution service, or licenses one from one of the major players on the scene. The feature lists of Xfire or GFW Live already sound a lot like Steam’s feature list (especially if you include the upcoming Steam Community).
Microsoft generally have one modus operandi: keep going until no significant opposition is left standing. They’ve made mistakes, like tying Live to Vista, but they have deep pockets and a lot of patience. It’s only a matter of time until Vista overtakes XP in terms of install base, as unlikely as that seems today. If they succeed, we could see Steam slowly squeezed from the marketplace, reduced once again to a hard-core of the Half-Life games. Microsoft is a platform-builder; as a company it’s pretty much what they do, and in time they are going to build something very compelling to developers. The only advantage Steam has currently is momentum. If they lose that momentum to Live, or anyone else, then they’re back practically to square one.
This then is a call to action to Valve.
Valve has a chance, right now, to take the initiative. If they make the right moves, we could see the rise of Steam as the de facto platform for online gaming.
Otherwise, the alternatives are almost certainly either ignominious fragmentation or total Microsoft domination. A benevolent dictatorship by Valve seems to be the better alternative to either having tens of digital distribution schemes, friends lists, and server browsers, none quite able to outdo any of the others, or the reprehensible Microsoft tendency to suck ever more money from people.
The Road Ahead
The first step is, in my opinion, to decouple all the Steam functionality from the Steam client, and from each other as much as possible.
Steam features are useless to most developers and publishers.
Developers have very few options for using Steam right now. In practice, the only options are one-time billing/download manager/limited auth functionality, a full Source + Steam license or nothing at all. There’s practically no middle ground; there are the occasional exceptions like Red Orchestra which uses VAC and was considering integrating Friends, but they have one difference over most third party games on Steam: they distribute with Steam at retail.
This is the source of the Defcon dilemma – if you want to distribute your game without the Steam client, you can only use the most basic sub-set of Steam functionality, i.e simple downloading, very limited CD-Key style auth, and one-time billing. Most Steam features are therefore useless to most developers and publishers because they can’t use them, even if they wanted to. They don’t want to bundle the Steam client with their games at retail, for a bunch of fairly obvious reasons.
If you’re a publisher, you don’t want people at retail to install something full of ads for your competitors’ games, you don’t want to force your users to create a fairly unnecessary Steam account, and you don’t want to limit the ability of people to play their games without an Internet connection; all annoyances and niggles inherent in the Steam client. For people unaware of Steam, being forced to install and create an account with what at first sight seems to be extraneous and annoying software is not a great user experience.
It’s a shame, because there are some fairly obvious uses of the Steam functionality that would work well even with retail games. Take the Steam filesystem and content streaming, for instance. If implemented properly, it works well for streaming games to you from a remote content server. What if it was streaming files from the game’s DVD? You could put the DVD in the drive, the auto-run would pop-up and ask where you want to install, then mere moments after that you’re in-game, because there’s a Steam filesystem in the background streaming files from the DVD into a .gcf cache file on your hard disc. Halo 2 for Vista uses a system somewhat like this to give a console-like experience; put the DVD in and you’re away.
What if developers could license the master server / server browsing platform? They could host the servers themselves or rent them from Valve, either way, they’re using a robust system known to work for the world’s most popular online action game. I’d say that was worth licensing! What if they could license Friends? In-game and out-of-game chat with not only people who’d bought that game, but anyone who had any Steam Friends enabled game. I have no doubt that a Valve-provided server browsing back-end would be infinitely better than the technology being sold by the likes of Gamespy.
Steam Auth could be extended to create a new type of Steam account. Why only have the username/password combination to access a Steam account? Couldn’t any known unique token be used? For example, the CD-Key of a game? It’s a weaker protection as CD-Keys can be stolen, but plenty of games still use such a system. Valve could license out the auth server software for third parties to run themselves, federating with the main Steam auth servers, or just rent the service from Valve.
Resolution of the Dilemma
There are an innumerable number of ways Steam services could be used by third parties, and the Defcon dilemma would be rather swiftly exorcised.
Instead of building the metaserver system, Introversion could have bought a license to use the same Steam auth system for their retail copies as for their Steam ones. They’d save the time of building and debugging their own system, and the cost of hosting their own servers. Then it would make sense for them to also use Steam server browsing, another component they don’t have to build, with the added benefit that Defcon would automatically work with any third-party server browser that supported the Steam server protocols.
Valve could offer price breaks to developers who use more than one Steam service, and they could even offer say, the Steam filesystem for free or very cheaply as it’s an offline feature that costs Valve nothing for a developer to use in their game. This would make it easier and economic for a developer to implement content streaming as it would enable streaming from a retail CD as well, benefiting all their customers. For a game like Defcon it wouldn’t be useful, but for a modern AAA title it would certainly be beneficial.
You join MySpace because your friends have, not necessarily because of the quality of the service.
I’ve only really scratched the surface of what could be done with decoupled Steam services. There’s no reason purchases couldn’t be initiated in-game, for example, inside a demo like with Xbox Live Arcade games. Games could also manage their own auto-patching/distribution through their own in-game interface. A combination of those two would be especially impressive. Imagine playing a particularly good demo that let you purchase and play the full game without even quitting it!
You can see the advantages all round. The developer/publisher saves money and time by outsourcing, the consumer gets the benefits of the larger ecosystem, and Valve gets paid. None of this is obviously Steam-driven to the consumer too, no clunky Steam client to install, so the disadvantages of the client have effectively gone away. The services are transparent; the experience is just better. The use of these kinds of services can then see greater and greater uptake, and as we’ve seen by the explosion of third-party games on Steam, the growth rate is exponential.
The more people a service attracts, the more attractive the service becomes. You join MSN Messenger or Xfire or Facebook or MySpace because all your friends have, not necessarily because of the inherent quality of the service. The more friends are on one service, the more valuable that service becomes to you. It’s also about the developer mindshare; if you’re using Steam features, or developers you know at other companies are, you’re going to be more likely to use other features in future if using them is obviously beneficial.
The Role of the Client
So what role will the Steam client have in this new order? The answer is that it can integrate with the Steam services now built directly into the games. The Steam-enabled (but not necessarily Steam-delivered or auto-updating) games on your system will appear in the games window. You could use the Steam client’s desktop Friends and Server browser for your separately-installed Steam-enabled games. It’s currently a nonsense that all Steam games have to clustered under SteamApps, in the future maybe they could install like regular games, but seamlessly integrating with the auto-updating features of the client if they’re using the Steam filesystem.
The same games perform better when digitally distributed via the Client because the developer did the work to enable content streaming because it was beneficial to retail too. Valve can up-sell both users and developers to the digital vision.
It’s about flexibility and a great user experience. Steam, to be even more successful, has to offer opportunities, not limitations. It has to behave more sensibly, more pragmatically, and less rigidly. A prime example of pointless rigidity is in the auto-updating system. It has immense value, but also tremendous downsides. Once the update is started, you can’t use the game. If your Internet connection failed right then, you’d be unable to use that game in offline mode, because the update broke it. You’re limited to Steam to get updates, there’s no possibility of cover-CD distributed ones for people with dial-up and the like.
The solution is simple and obvious. Download patches into a temporary cache file. Once the download is complete, Steam auto-applies the patch. The game is always functional. This opens up options like: the patch being delivered a day or two before it’s activated, ensuring that the number of people wanting to play but unable to while the patch is downloading is kept to a minimum, offline game patching by allowing people to distribute (something like) the temporary cache outside of Steam, or pre-update betas being the norm rather than the exception.
A feature like this was originally intended to be a part of Steam, and was known as “trickle updating”. In the (probably paraphrased) words of Valve, “new content will just be there, when Valve says it will”. In reality, I can’t remember the feature ever being used, and it has apparently been dropped for operational reasons. It is more immediately practical for Valve just to push updates out monolithically, as most clients will download most updates quickly on their broadband connections.
The client will still exist for the reasons Valve initially conceived it. Having a permanently available application is valuable because residency has opportunities that transiency doesn’t; like trickle updating or community features, a single unified place to get at everything that matters to you in the gaming space.
I’m tempted to play the Devil’s Advocate and even suggest that the Steam Client should support in some fashion games which aren’t Steam enabled at all, for example providing XFire-like support for Friends and Server browsing and a listing in the Steam games list. This may look pointless, but it will make Steam more useful to end-users, which drives foot-traffic to Steam boosting sales, and gives Steam more mindshare. The community could be leveraged to develop code to enable server browsing for other engines. It would be worth it at least to get server browsing for Red Orchestra integrated into the Steam server browser!
It’s All About the Platform, Stupid
The initial vision of Steam could have been typical Valve over-ambition, cut short by technical challenges. I think there’s still a lot of value in that vision, though, especially if extended in subtle ways. The original vision was making Steam a platform that anyone could use, but it has since become somewhat more of a walled garden. There are advantages to the walled garden approach, and it’s certainly worked for companies like Apple who have successfully carved themselves out a niche with the iTunes + iPod system.
The company which has really been successful though is Microsoft, the platform builders. Rather than trying to sell their own software using their own operating system that runs only on their own hardware, you can run any application from any developer on Windows on standardised hardware you can buy from any hardware supplier. Microsoft has built a platform which bridges the gap, and they do very well out of being the middleman which supplies the glue.
By opening Steam up as a flexible platform, providing the “glue” between developers and gamers, they could be extremely successful. They could do a lot to change a lot of people’s minds. It’s all about getting critical mass, about getting mindshare.
Niggling edge cases are a pain to handle, but all-important.
Being a platform builder is hard. You have to ensure that your platform works 100% of the time for 100% of the people you’re aiming it at. Valve hasn’t done brilliantly at this so far. Part of the point is ensuring a good experience for everyone, and a good experience is all about the details, the niggling edge-cases which are a pain to handle correctly but are all-important. When Steam becomes acceptable to include in all games to everyone, not something a sizeable segment of gamers still see as something that has to be endured to play Valve games, then we could see a revolution.
The continued flakiness of Steam’s offline mode is one good example of this. The vast majority of the time it works, in a roundabout manner, but there are those edge cases it fails in immensely frustrating ways, like the “interrupted update” problem I mentioned earlier. In some ways this is a consequence of offline mode being an afterthought; the first release of Steam had no offline mode at all; Steam left beta in September 2003, but the release which added Offline Mode didn’t arrive until March 2004! Steam always has been designed exclusively for broadband connections and is painful in a lot of very un-obvious ways when you’re connected to a very good Internet connection in America. Woe betide you if only have dial-up and you want to install HL2 – Steam will force you to sit through a large update despite how the game works fine as it was published on CD. Woe betide you again if you live in Australia and have restrictive download limits and Byzantine networking arrangements.
Somewhere Valve forgot about the flexibility of their original vision, of a Steam that could be used fully by anyone, that would have made the experience universally better for everyone, even those on dial-up. It’s a crying shame that that flexibility and potential is long since lost, to do the easy thing and serve the majority, rather than the plurality.
Hopefully Valve will take this kind of thing seriously once they hire their new senior user experience director.
Basically, my message to Valve is this: Make it flexible. Make it open and inclusive. Think about all the people, both developers and end users, who could be using your technology for their benefit and yours who aren’t because of arbitrary limitations and restrictions. The removal of these limitations and the cutting of ties of the Steam services to the Steam client would have the end result of driving people to the client, not driving them away. There are technical hurdles, but they’re not insurmountable, and they’ll lead to a better experience not just for Valve and their customers but for everyone.