{"id":845,"date":"2010-04-04T15:37:47","date_gmt":"2010-04-04T14:37:47","guid":{"rendered":"http:\/\/t-machine.org\/?p=845"},"modified":"2010-04-04T15:37:47","modified_gmt":"2010-04-04T14:37:47","slug":"assasins-creed-2-understatement-of-the-century","status":"publish","type":"post","link":"http:\/\/new.t-machine.org\/index.php\/2010\/04\/04\/assasins-creed-2-understatement-of-the-century\/","title":{"rendered":"Assasin’s Creed 2: Understatement of the Century"},"content":{"rendered":"

From the IGN walkthrough<\/a>:<\/p>\n

\n“If you have trouble grabbing the beam, just keep trying\u2014we promise it works, but lots of readers have told us it’s not always easy.”\n<\/p><\/blockquote>\n

<\/p>\n

I’m a pretty good AC player, but after 10 minutes of trying to do that one standing jump, I gave up and stopped playing for a long time in frustration.<\/p>\n

When game developers talk about “games should be so easy that all players can complete them; no-one should ever have to give up \/ fail to complete a game because something is too hard”, I usually disagree.<\/p>\n

But in this instance, where the game is extremely, excessively difficult on something that the designer obviously intended to be extremely simple – and where the player has spent hours being taught that this will be easy – you have something different going on. It’s a failure of the control scheme; in fact, it’s a bug.<\/p>\n

It’s a side-effect of the heuristics that AC uses to decide “what the player is trying to do” – heuristics that are far from perfect, while being very good.<\/p>\n

In the first game, it took me a long time to get past the intro – no, really – because if you *try* to jump over gaps, then you fail. The heuristics were so heavily weighted towards “allowing” you to jump off buildings that running over a small gap became very difficult – until you learnt that the character “automatically” jumps small distances.<\/p>\n

On the whole, I’m very impressed by the AC2 heuristics – compare it to Mirror’s Edge (a beautiful game, but feels a lot less fluid). I find them a bit too simplistic – I would love another 25% or so of user-control, and another 50% of precision on directional control – but (as ME shows) they got closer to perfect than any other game so far.<\/p>\n

BUT … what do you do about a bug like this, one severe enough to make me stop playing the game entirely?<\/p>\n

They had a huge QA team already (this is Ubisoft, after all), and such a vast amount of content in this game (multiple entire cities, modelled in fine detail), that there’s no way they could be sure to catch this bug.<\/p>\n

Or is there?<\/p>\n

This is the raison d’etre for a whole segment of in-game analytics \/ metrics: data-mining to discover undiscovered bugs.<\/p>\n

Good metrics for game designers are VERY hard to describe, and IME the vast majority of the industry doesn’t know how to carefully hand-pick the few numbers they really need out of the millions of stats availalbe. Here’s a good example of how to pick well.<\/p>\n

If the game reported<\/p>\n

\n“the quest-point at which people stopped playing”\n<\/p><\/blockquote>\n

…then you *might* discover this bug. But it’s too coarse-grained.<\/p>\n

If the game reported either\/both:<\/p>\n

\n“the segment on the map where people stopped playing”
\n“the segment on the map where people spent most-time during a mission”\n<\/p><\/blockquote>\n

…then you’d quickly and easily discover this bug. By “segment” I mean, essentially, a small patch of polygons approximately 6’x6′. This is relatively easy to measure algorithmically using greedy-polygon grabbing and hashing – although it would take a little care to make sure the measurement of the value didn’t take much CPU time (it could easily be pre-compiled for any given map, of course).<\/p>\n

I’m not 100% of the “stopped playing” part – this is a console game, and while that info would be useful, it would mostly stop evenly distributed over quest-end points. Where it was more \/ less likely, it would be obvious just from knowledge of the story. ALTHOUGH: still well worth doing *in case* there were anomalies there – that should set off alarm bells.<\/p>\n

However, the “spent most time during a mission” is more cut-and-dried.<\/p>\n

This probe gives you a set of local maxima. It’s categoriesed by mission, making it one level finer than doing it over the entire world-map (which is too much, too uncategorised info), and it’s also coarse enough to correlate closely with user-behaviour (it merges results mission-by-mission; recurring bugs are very likely to show up by people doing the same mission and getting stuck at the same point).<\/p>\n

The mission-based merge of results also has a nice side-effect: it tends to iron-out any anomalous results due to people wandering around the open-world game.<\/p>\n

So. With a little bit of probing, using probes that you could\/should have invented at the start of development (i.e. without knowledge of exact bugs that would occur) this bug could be ironed out. The three remaining questions are:<\/p>\n

    \n
  1. does Ubisoft do this level of automated-bug-detection,\n
  2. do their designers bother to look at the anomaly-date,\n
  3. and if so … why hasn’t the game been patched?\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

    From the IGN walkthrough: “If you have trouble grabbing the beam, just keep trying\u2014we promise it works, but lots of readers have told us it’s not always easy.” I’m a pretty good AC player, but after 10 minutes of trying to do that one standing jump, I gave up and stopped playing for a long […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13,47,28,9],"tags":[],"_links":{"self":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/845"}],"collection":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/comments?post=845"}],"version-history":[{"count":0,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/845\/revisions"}],"wp:attachment":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/media?parent=845"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/categories?post=845"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/tags?post=845"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}