Red Orchestra load times traced to Steam
Since this post, the Steam virtual drive has been dropped and loading times are no longer abnormal.
The long load times that currently plague large numbers of Red Orchestra: Ostfront 41-45 users have been traced back to Steam and its new ‘virtual drive’ system, used by Ostfront to access its GCFs.
Red Orchestra: Ostfront is a
testbedfor new Steam technologies.
In a post on the official Ostfront forums, users experimented with extracting the contents of the game’s two GCFs and found that standalone versions, which can join local games when steam.dll is copied to the /system folder, saw load times three times shorter than the unextracted version of the game run through the Steam interface.
The Steam Virtual Drive was introduced with the release of Ostfront, currently the only game using it, to allow use of the GCF filesystem without developer input. Whereas the standard implementation involves replacing file calls in a game’s source code with Steam versions, the virtual filesystem allows the game to see its GCF as a collection of loose files on the disk
. Why developers Tripwire Interactive decided to use a new and untested system when the established one was readily accessible, especially when VAC integration shows they were prepared to implement other Steam features, was revealed by Tripwire rep Yoshiro, who described Red Orchestra as a testbed
for Steam technology during an impromptu IRC session last week.
When Tripwire staff replied to the thread, they defended their decision to distribute through Steam but did not mention loading times or their cause, suggesting that the users’ findings are accurate.
Let me say this right now – there would be NO Red Orchestra: Ostfront if it weren’t for Valve/Steam, period. Tripwire as a company exhausted all our possibilities for standard distribution initially. We were about to go out of business and just become a footnote in modding history – The team that won an engine license but faded away before they ever got a chance to do anything with it. With Valve/Steam we were able to get the game into gamers’ hands, attract the attention of a standard retail publisher, and become a sustainable game studio.
I’ll be the first to admit there has been some unexpected bumps while Steam expands to handle other 3rd party software just like there were some initial issues with Steam back when they first introduced it for their own games. The truth is though, the majority of the tens of thousands of people playing RO have a pretty smooth time with Steam. And for the people that do have issues, those issues are getting sorted out rapidly, just like the AOL issues, “connection failed” issues, ZoneAlarm issues, etc.
Tripwire’s ‘exhaustion’ of distribution options before talking to Valve echoes both Garry Newman’s situation, where Steam distribution gave the stalled development of Garry’s Mod a new drive, and Introversion Software’s near-bankruptcy before negotiating a Steam deal for Darwinia. How many other games and developers could be saved through digital distribution?
Having defragmented my hard drive, RO load times have now returned to normal on my computer.
36 Responses to this post:
Comments
Andy Simpson Says:
Hmm. Shows that Valve are going to have to start taking their responsibilities as a platform provider really really seriously. It’s one thing if it’s for your own use, but if you’re selling it on you need to make sure you’re sharp on things like performance.
Matthew Says:
I watch ur feed and can i say great post dude. I think the RO devs got it right with their reply … this is relatively new territory that is a lifeline to many aspiring dev houses.
And i totally agree with Andy Simpson, it reinforces how Valve has to take their position very seriously … its one thing to throw a lifeline but if it breaks under the strain, and customers aren’t impressed, then it could have come to nothing in the end.
But i tend to think Valve is on the right track.
RP Says:
Can someone elaborate more on that? I didn’t understand 🙁
Also I agree with what you said andy, It really gives small dev teams that have great games to get them into the public.
How sad it was if RO:OS wouldn’t manage to get thier game out, winning a 250,000$ prize and not being able to actually do something with it.
Tom Edwards Says:
It means they don’t have to do anything to get the game to look inside GCFs. They just run it and it works.
Andrew Timson Says:
True. On the other hand, they may have not even known that there was a performance drop; it’s possible that it only shows up when performing certain operations, and that Valve simply didn’t think to test for them.
The Source engine keeps each texture, sound, etc. within the GCF in its own file. Compare this to the Unreal engine’s bundling of textures/sounds into several large files (usually by “type”; one file for all the castle textures, another for industrial stuff, etc.), a much different scenario.
Depending on how the driver works, it’s quite plausible that Valve just never tried to use it the way that Red Orchestra does. Now that they know that there’s a performance hit, I’d expect to see them concentrate on improving it (especially if they want to expand their third-party offerings).
Tom Edwards Says:
That’s a good point and a very possible explanation, but I would think that someone would notice the precipitous drop. 😉
(As you can see to the right, I’ve enabled Gravatars.)
Helmut_AUT Says:
I’m the guy who came up with the thread on the RO forums linked here. Just wanted to elaborate on what Andy Simpson said – yes, Valve need to get on the ball. Also regarding things like customer support.
If you read the thread, you’ll see that I only started extracting .gcf files after I ran into problems launching the game over Steam. Tripwire does their best, but Steam Tech Support leaves you hanging. So the main reason for testing the extracted game version actually was to just be able to play – and then it turned out load times were cut in half as well.
All in all, I understand that without Steam, there might be no Red Orchestra Release – but still I think the delivery platform just adds a lot of bloated stuff and technical problems to the game.
wizpig64 Says:
My gravatar is still pending >_>;
Will it show up on this post when it is since by now I’ve signed up :o?
Well anyways, I’m glad there is an easy way for developers to convert their standard games to Steam enabled games, but if load times are gonna increase by up to 400%, I don’t see how that is going to attract customers when they find out. (Oh look, they just did.)
Tom Edwards Says:
Support are more for providing solutions to existing problems. They aren’t very good at diagnosing new ones: you’ve got to go to the devs for that stuff.
As long as you signed up with the same e-mail address as you use here.
I’m still waiting for them to auth the gravatar for my contact@steamreview.org account, which is the one I use for every comment except the one above. 🙁
They also seem to have screwed up my existing one, and I’m not alone. In fact the whole thing is pretty borked at the moment.
baff Says:
Steam is just another over-aggressive DRM.
The people at Steam hold the (illegal) attitude that after they sell a product, it is still theirs.
After you have paid for a product, you may only use it in a way that suits them and with their active consent.
They refuse to refund people (illegally) on principle.
I can’t play offline, I must buy multiple copies to play on LAN and if some 12 year old net stalker takes a dsilike to me he may get my account disconnected and reduce my game to worthless scrap.
What a pity.
Steam could have been so good.
All the good things people say about Steam are true.
Unfortunately their rampant greed has been allowed to ruin it.
I will never make a purchase from Steam.
Pretty soon they (or the developers that use them) will end up in court. At which point Steam will either mend it’s ways or fold.
Until then games producers will just have suffer the reduced sales associated with this kind of bad practise.
Avoid Steam like the plague. It has a long way to come before it becomes a useful service.
Tom Edwards Says:
Ignorance, hatred, and paranoia. Congratulations, you’ve got the whole spectrum covered!
DiSTuRbEd Says:
So true Tom.
Let me tell you, its so fun searching for all my games patches!
wizpig64 Says:
Multiple copies to just do lan-play? VAC can’t ban you from lan games. And how would some demented 12 year old stalker ruin your account? By guessing your password? That’s not Steam’s issue, that’s your own.
*changes password* O_o
Andy Simpson Says:
I think what Valve need to do is take a step back, really look at the core technology and work out all the ways they can really tune that to get it running the best they possibly can. Maybe Steam 3 will do this, but the roll-out of Steam 3 is either agonisingly slow, or it’s not actually making that much difference.
The problem is that Steam can just be pretty damn flakey. I mean, just the other day I observed some utterly braindead behaviour in Steam, and I probably should submit it as a bug report. A freshly installed copy of Steam will start up to the “Create a new account or login” screen, but will silently go into offline mode. So any attempts to actually create an account will fail. That’s pretty stupid. There are a host of errors caused by misconfigurations that can be solved just by flushing the clientregistry. Those should be fixing themselves, now. I can understand skimping on error handling to concentrate on features, but in a mature product like Steam they need to work on going back and making it more robust, making it fix itself more often than not.
They need to make sure they’re tight on performance and throughput. A little bit of optimisation will go a long way. The increases in load times reported by Helmut_AUT are appaling. You’d expect some performance hit, because Steam adds another abstraction layer but at the end of the day you’re still pulling bytes from a disk, and that should be what’s taking the longest. If the Steam overhead is at the same order of magnitude as physically reading magnetic stripes off a rotating disk then there is something very seriously wrong.
Tom Edwards Says:
I think you’ll find that that’s an issue with getting the registration information to the auth servers rather than anything to do with Offline mode. What did you do to fix it?
But yeah, I’m with you on the error reporting. It’s been dreadful for years. Make a game’s GCF read-only and when it needs updating you’ll be told that the Steam servers are too busy! How many ranting support tickets could Valve avoid if they just took the time to add a proper error system that a) gave the right messages and b) explained what they meant?
The good news is that Support have got the hang of things now. This sort of stuff just needs to be extended to the client.
Andy Simpson Says:
In my haste I kinda forgot to mention that there wasn’t a functional internet connection at the time, because Norton was being non-functional and automatically blocking Steam without providing any method of unblocking it. Glad it wasn’t my PC. Anyways, Steam failed to detect an internet connection and silently went into offline mode, but still offered the option to create a new account. Baffling.
And yikes. That’s awful. I can understand their reluctance to stick detailed technical information there, but without it trying to work out what’s causing the problem is a nightmare. Maybe they could have a “View details” button like with Windows error reporting. Just if the error messages dumped raw error codes it’d be better than saying it was a problem with the servers. It’s pretty much just lazy programming. I do it myself when I’m programing. Rather than providing an error handler for each little thing that could go wrong, you get lazy and bung in a generic one. Just for hobbyist programming that’s fine, but for a program deployed to millions of people? It’s unacceptable. The support guys should really be giving the devs hell about this, because they’re making the support guys’ lives a lot harder than they need to be.
Tom Edwards Says:
Heh, think about it for a minute. It said it was in offline, but how reliable is that? 😉 Offline mode is something that is either on or off when Steam starts (correct me if this has changed); you are just seeing a duff error message for limited connectivity.
Andy Simpson Says:
Well, yes. That’s another problem with Steam. Offline mode is more or less still a hacked-in afterthought. Steam is inherently designed to assume the Internet is always there, and is always 100% reliable. Which is why things like the welcome screen do things like still having the “Create account” button enabled. No-one thought, “Wait a minute, what if we’re not connected right now?” At the moment it does the Internet check on Startup, then disables every feature that would need the Internet. It’s not exactly sophisticated.
The sad thing is Windows actually provides APIs to tell you if there’s a connection or not. MSN Messenger is a stellar example of the form in this regard. It must have taken bucketloads of work, but the connection information page in the options of that even tells me the current signal strength on my wireless connection, and that I’m using a UPnP NAT. If Steam could do a similar thing it’d be great.
Andy Simpson Says:
The Network Location Awareness API.
Andy Simpson Says:
Whoops. *Wishes he could edit posts*. Stupid MSDN Library.
The actual link.
It’d be great if Steam could use it, anyway.
Andy Simpson Says:
Oh, hell. Only supported on Vista. I should really just shut up… There must be a way to do on XP anyway. MSN Messenger does it just fine…
Tom Edwards Says:
You can edit comments if you are logged into the account that created them. You’ve got an account, but don’t seem to be using it?
Andy Simpson Says:
Heh, seems I do, but I’ve been completely forgetting to login. Wow, I’m special today.
Anyways, after further investigation, it seems like that NLA API I eventually managed to post a link to is just a wrapper for a bunch of Windows Sockets functions, so Steam could be more network-aware if it wanted.
Anyways, I guess the point is they should be focusing more on stuff like this, and less on pretty pointless things like the new version of VGUI2 used in the new Steam UI and the Steam Media Player.
GamerDude Says:
A lot of gamers I know have the NLA API disabled, along with a long of other Windows services. Making Steam dependent on this would be a bad thing, in my opinion.
Tom Edwards Says:
I don’t think Andy is suggesting Steam be made reliant on it, just that it uses it. There’s no reason to make it compulsory.
wizpig64 Says:
Oh thats why there isn’t a Edit button! I’m not logged in! Doh!
steam life ruiner Says:
steam baned my account for hacking/cheating but i never did what can i do to get my account back that i rightfuly paid for if i do not get it back i will take steam to court cause i didnt hack n now my accounts blocked from playing vac servers ! im pissed !!!!!!!
wizpig64 Says:
Read the license agreement. You say you “rightfully paid for” your games, yet Valve reserves all rights to ban you from VAC. There is no way you can be banned from VAC without hacking (unless someone hijacked your account or someone at Valve has a grudge against you). The only way to get your online play back is to re-buy the game(s) with a different Steam account. Sorry.
Wizpig works for steam/affiliate Says:
if u see the user above WIZpig64, well he works for steam or affiliate, and looks for threads like this to collate on, be warey for moles, if u want to create a thread do it in a better place, all though steam games are good, its being run by small timers like him to reap alot of money from people, and screw u if it doesnt go there way, thus explaining why the mole is very aggressive towards all issues. read the above threads.
Werfetter Says:
I’m pissed off! I did read the label where it said net connection required for install or online play, but nowhere did it mention is must log in each time. I expected something like BF2, if I wanted to download patches, then I could’ve pirated the game! Mine will be going back tomorrow!
Tom Edwards Says:
You don’t need to log in every time. Either start Steam when you don’t have a connection, or choose
File > Go Offline....gamer Says:
Well i want to say it was a big mistake i cant get it from the internet i buy games from the store and red orchestra isnt a game that you can find in stores only if you download it from valve…think that i cant do so they loose huge ammounts of gamers and that means money because of these.
Tom Edwards Says:
Sure you can buy it shops. Here it is on Amazon.
3 Trackbacks/Pings
The Steam Review » Blog Archive » Virtual Drive no longer virtual
Virtual Drive no longer virtual - The Steam Review
Laid Back Gaming license Steam and Source - The Steam Review