Thanks to Mike Leahy for spotting this:
http://blog.gemserk.com/2012/02/02/how-we-use-box2d-with-artemis/
…a short blog post (with code) on how a team is integrating Box2D (a very well known open source physics lib) with Artemis (a java implementation of Entity Systems which starts from the same point as my Entity Systems posts, but fleshes it out)
For a couple of years now, Firefox has had a nifty feature where if it crashes – for ANY reason – the next time you run it, it gives you a “menu” of the windows you had open, and lets you selectively re-open them. As a bonus, this menu is just a plain window – you can ignore it (if you’re too busy right now), and open some other windows in parallel.
(even if it’s not Firefox that crashed – e.g. a laptop battery ran out, or someone tripped over your power cable – you still get this window. This is a huge help on OS X, which crashes quite a lot if you’re running Apple’s dev tools :))
I’d wondered a few times “what happens if it crashes again, before you’ve selected which things from that menu you want to re-open?”
I just found out the hard way: actually, it gives you a new menu, where one of the items is … the previous menu. It nests, all the way down (I had a crash-of-a-crash-of-a-crash – and I got everything back perfectly. Very useful, too – one of the deeply nested open windows was an important form I’d forgotten I’d not yet sent).
On the downside, if you close Firefox, and re-open it normally, the menu gets blown away – but that’s probably to do with the “five-years-and-counting still unfixed” bugs in Firefox where “remember open windows on startup” doesn’t work properly (there’s some epic threads in their bug tracker, many complaints, but no fix yet).
Still, for the most part, this is a very nice approach to application-crashing. One to remember when designing your own document-based applications…
In the ongoing, epic comments (300+ and counting!) for my Entity Systems posts, one of the recurring questions is:
What makes a good Component?
How should I split my conceptual model into Entities and Components?
How should I split my algorithms and methods into Systems?
It’s often difficult to answer these questions without concrete examples, which are thin on the ground.
Good news, then…
Paul Gestwicki runs a CS315: Game Programming course, and last year his students used an Entity System to implement their game – Morgan’s Raid. In a recent email conversation, he mentioned he’d been monitoring the actual number – and nature – of the Components and Systems that the teams developed and used on the project.
He’s now posted this list, along with some brief analysis.
Read Paul’s post – there are some caveats he mentions, and there’s a useful diagram showing roughly how many systems were using each component.
I strongly recommend you play the game too (it’s free, and quick to play) so you can get an idea straight away – just from the names – what data and code some of these contain.
To recap, here’s the list:
“For your reading convenience, here’s a simple tabular view of the systems and components
Systems Components BackgroundTileSystem
CityNameSystem
DestinationSystem
FadingSystem
GPSToScreenSystem
HoverableSystem
ImageRenderingSystem
IntInterpolationSystem
MinimumSleepTimeSystem
MorganLocationSystem
NightRenderingSystem
OnClickMoveHereSystem
OnScreenBoundingBoxSystem
RaidSoundSystem
RailwaySystem
ReputationSystem
RevealingTextSystem
SpeedSystem
StepwisePositionInterpolationSystem
SunSystem
TimePassingSystem
TimeTriggeredSystem
TownArrivalSystem
TownArrowSystem
TownUnderSiegeSystemAnimationRenderable
ArrivesAtTownIndex
BackgroundTile
BeenRaided
CentersOnGPS
CityData
CityImages
CityName
CityPopulation
CityTargets
CommandPoint
Destination
DoesCityHaveMilitia
DoesCityLoseGame
DoesCityWinGame
GPSPosition
GPSPositionList
HasMorganGPS
Hoverable
ImageRenderable
ImageRenderLayer
InGameTime
IntInterpolatedMinimumSleepTimeOverride
Morgan
MorganLocation
MovesOnClick
OnClickMoveHere
OnScreenBoundingBox
OnScreenBoundingBoxList
PositionInterpolated
Raidable
Raider
Railway
Reputation
ReputationValue
RevealingText
Road
RoadsToCity
RouteTaken
Speed
Sun
Terrain
TimeToRaid
TimeTriggeredEvent
TownAdjacency
TownArrow
TownUnderSeige”
Things I noticed straight away:
NB: I haven’t had time to look at the source code, but it’s freely downloadable here, so I’d recommend having a look if you have time.
I’ve generally started people off with: “aim for 1:1 ratio”. This is mainly to kick them out of the traditional class-based OOP mindset. In practice, there’s really no need to stick to that religiously – once you get the hang of ES design, you should be freely adding and subtracting components all over the place.
In reality, the pressures on “number of systems” and “number of components” are independent. Ideally, you add a new system when you have a major new concept to add to your game – e.g. “previously I was using hand-made jumping, now I want to add a complete physics-driven approach. This will mean changing collision-detection, changing the core game-loop, etc”.
Ideally, you add a new component when you have a new “dimension” to the game objects. For instance, if you’re adding a physics System, you may not need to add any new Components – it might be that all you need is Location (containing x,y,x position and dx,dy,dz velocity) and RenderState (containing screen-pixels x,y) – and that you already have those components.
One of the advantages of an ES is that old code can just fall off the radar and disappear. So I’m not surprised at all to see some components that appear to be unused – and it’s MUCH easier to simply delete this code from your project than it would be on a traditional OOP project. Does anything reference that data? If so, it’s a set of particular systems. For each system, you can look at MERELY the system and the component, and make a very quick decision about whether you still need this access – or if you can refactor to move (some of) it somewhere else. The amount of code you need to read to make such decisions safely is typically very small – i.e. easy, quick, and less error-prone.
This is fine. However, it can also be an early-warning sign of a design or code-architecture bug. Sometimes, there are components that – innately – are just needed all over the place. For instance, in a team-based game, the component saying which “team” a given object/player/item/building belongs to is likely to affect almost every piece of algorithm code across the board. It’ll be referenced by many systems.
On the flip-side, it may be a sign that you’ve put too much data into one component. There are two usual versions of this:
Of course, you get this exact problem in traditional OOP setups, but with an ES it’s trivial to fix. Split – or duplicate – the Component, change a few references in the Systems, and you’re done. If it turns out a week later that the split wasn’t necessary – or worse, was a step backwards (e.g. you find yourself frequently synching the data between those components) – it’s extremely cheap to swap it back.
By contrast, with OOP, this is a nightmare scenario, because you have to worry about every method on the original class. Does that method:
…this is just a lightning quick analysis, but I strongly invite you to do you own digging into the classes – and the codebase – and come up with your own thoughts and feedback. We have here a convenient, real-life, list of components/systems – something to dig our teeth into, and debate the rights and wrongs of each decision. And I’m sure the students involved on the project would be interested in your feedback on their approaches :)
Support me on Patreon, writing about Entity Systems and sharing tech demos and code examples
Ever seen this hard to notice bug in an iPhone / Mac project? Caused by a flaw (bug?) in Apple’s own API:
NSDictionary * map;
…
NSObject * key;
NSObject * value;
…
// NB: next line will frequently crash at runtime
// …but even if doesn’t crash, it probably won’t do
// what you’d expect it to:
[map setObject:value forKey:key];if( value != [map objectForKey:key] ) // HA!
{
NSAssert( 1 == 0, @”What the **** just happened??” );
}
NSDictionary is one of the nastier things I’ve seen in a programming language: in an Object-Oriented Language, it’s a class that refuses to take Objects as arguments, *but pretends to*. If you attempt it, it either crashes, or it invalidates your objects, breaking contracts all over the place.
In the days pre-OOP, a “dictionary” was something that mapped:
“a string”
to
“anything” (usually: basic datatypes – e.g. strings, integers, floats, etc).
In the days of OOP, the same thing is usually called a “map” (which is a better term) – although the terms are synonymous – and maps:
“an object”
to
“another object”
What did Apple/NextStep/Crazy-Guys-Behind ObjC do?
NSDictionary: maps:
“STRINGS ONLY” (no objects allowed!)
to
“OBJECTS ONLY” (no core datatypes allowed!)
Yep – but Apple’s internal implementation takes a *copy* of the object, and uses that as a key – rather than using the object that you gave it. This is a common problem in OOP languages and implementations of Map – e.g. Java does the same thing.
Unfortunately, this means that you can call “setObject:forKey:”, and then “objectForKey:” will return nil *for the same key*.
In Java and other OOP languages, you are required to implement a custom “isObjectEqualToObject” method. In ObjC too – except that that method is ILLEGAL if you’re using CoreData.
I’ve never seen this in other OOP languages, but in ObjC *additionally*: if you don’t manually add
How come? AFAICT, Apple’s header file is wrong:
– (void)setObject:(id)anObject forKey:(id)aKey // You lie!
it seems the implementation of that method is:
– (void)setObject:(id)anObject forKey:(id<NSCopying>)aKey; // the correct signature?
Net result? Code that happily compiles … will crash. ARGH!
…so what’s the other one?
Ah, just the one we see again and again on live projects, wasting hours and hours of time:
NSDictionary* dictionary = [NSDictionary dictionaryWithObjectsAndKeys:
object1, key1,
object2, key2,
nil];NSLog( @”dictionary = %@”, dictionary ); // but it only has one key/value pair. ?!?!?
Ah, well … that nil … what happens when object2 is nil? Oh, damn.
What about if key2 is nil? Now we’re really nasty … we’ve given it “half” of key/value pair. Nice!
It looked so good, finally – a decent Git client for Mac.
But, no. It just – simply – doesn’t work (version 1.2.1). Great for browsing, but sometimes you simply can’t commit changes to a repository! (a few hours earlier, it was committing fine. Nothing had changed in the meantime)
And what do you get as an error, when you try?
“A git error occurred.”
Seems the folks at GitHub know this is unacceptable. Because they also put a message onto the (invisible) system console:
-[GHErrorSheetController presentWithError:] [Line 84]
Presenting with error, but let’s do better guys: Error Domain=GHGitControllerErrorDomain Code=556 UserInfo=0x117f63fa0 “A git error occurred.”
For the record, there was nothing wrong, and no errors – all other clients worked fine. So it’s probably an internal problem with GitHub’s code. They know it’s a problem (they wrote an error for it) but they won’t tell you what that problem is. Classy :)
Two things here: if you run any Rails site, check out the security hole ASAP if you haven’t already. You might be safe – but given that even GitHub wasn’t, I’d double check if I were you. (The Rails community seemingly isn’t patching it – and there’s nothing recent on the Security list. Which leaves me going: WTF? The evidence is right there on GitHub of how bad this is right now, in the wild).
Secondly … what just happened? Apart from doom and gloom and “the end of every unpatched Rails site on the planet”, there’s a fun story behind this one. As someone put it “it’s the whitest of white-hat attacks” (i.e. the “attacker”‘s motives appear extremely innocent – but foolish and naive)
It seems that GitHub got hit by the world’s nastiest security hole, in Rails – trivial to take advantage of, and utterly lethal. The hole appears to allow pretty much anyone, any time, to do anything, anywhere – while PRETENDING to be any other user of the system. So, for instance, in the attack itself, someone inserted arbitrary source code into a project they had no right to.
Hmm. That’s bad. It effectively destroys GitHub’s entire business (it’s already fixed, don’t worry)
But it gets worse … it’s a flaw in the RoR framework, not GitHub itself (although apparently GitHub’s authors were supposed to know about the flaw by reading the Rails docs, as far as I can tell from a quick glimpse at the background). Rails authors have (allegedly) known about it and underestimated how bad it is in the wild, and left Rails completely open with zero security by default.
So, allegedly, the same attack works for most of the web’s large Web 2.0 sites – any of them that run on Rails.
WTFOMGBBQ!
Who was the perpetrator of this attack? Ah, well…
Classy. Dangerous (high risk of someone calling the police and the lawyers), but if people won’t believe you, and *close* your issues, claiming it’s not that important, what more amusing way to prove them wrong?
I can’t state this strongly enough: never attack a live system. Just … don’t.
Any demonstration of a security flaw has to be done very carefully – people have been arrested for demonstrating a flaw allegedly *at the owner’s request*, because under some jurisdiction’s it’s technically a crime even if you’re given permission. In general, security researchers never show a flaw on a real system – they explain how to, and do it on a dummy system, so no-one can arrest them.
(why arrest the researcher? Usually seems to be no reason beyond ass-covering by executives and lawyers, and a petty vindictiveness)
Homakov appears to have been ignorant of this little maxim, hence I’m writing it here, let as many people as possible know: never attack a live system (unless you’re very sure the owners and the police won’t come after you)!
On the plus side, they fixed it within hours, on a weekend. And then proceeded to tell every single user what had happened. And did so in a clever way – they put a block on all GitHub accounts that practically forces you to read their “here’s what happened, but we’ve fixed it” message. They could have kept it quiet.
Which is all rather wonderful and reassuring.
On the minus side, IMHO they rather misrepresented what actually happened, portraying it more as a malicious attack, and something they fixed, rather than what it was – the overspill from an argument between developers on some software that GitHub uses.
And they initially reported they’d “suspended” the user’s account. Normally I’d support this action – generally it’s a bad idea to let it be known you’ll accept attacks and not fight back. But in this case it appears that GitHub didn’t read the f***ing manual, and the maintainers apparently (based on reading their tickets on the GitHub DB) refused to accept it was a serious problem – and apparently didn’t care that one of their own high-profile clients was wide open and insecure. The attack wasn’t even against GitHub per se – it was against the Rails team who weren’t acting. IF it had e.g. been a defacement of GitHub’s main site, that would have been different, both in impact and in intent. Instead, the attack appears to be a genuinely dumb act by someone being naive.
Seems that GitHub agreed – although their reporting is a bit weak, it happened days ago, but they never thought to edit any of their material and back-link it.
“Now that we’ve had a chance to review his activity, and have determined that no malicious intent was present, @homakov’s account has been reinstated.
…and it’s pleasing to see that their reaction included a small mea culpa for being unclear in what they expect (although anyone dealing with security ought to be aware of this stuff as “standard practice”, sometimes it’s not security experts who find the holes):
“We haven’t been as clear as we should have been on how to responsibly disclose security problems, and for that I’m sorry. To prevent future confusion about security-related account suspension, and to make explicit our stance on responsible disclosure, we have added a section entitled Responsible Disclosure of Security Vulnerabilities to our Security policy.”
I’d expect: shame, weeping, and BEGGING the web world to forgive their foolishness. I’m not sure, but it’s going to be interesting to watch. As of right now, the demo’s of the flaw are still live. I particularly like one commenter’s:
drogus closed the issue 5 days ago
kennyj commented
5 days ago
“I’m closing it (again).
@drogus was close it, but it still open.
github bug?”Closed
kennyj closed the issue 5 days ago
“github bug?” LOL, no – massive security flaw :).
I’ve only played with this for a few minutes, but so far it seems to have an excellent, simple, clear GUI with the core features I’d expect. Way better than any other Git client I’ve seen for Mac. And it’s free!
GitHub for Mac v1.2.1 (NB: works with any Git server, not just GitHub.com!)
I’ve noticed some cosmetic bugs – e.g. it renders all the user-account avatars completely wrong (puts wrong image next to each commiter) – so I’d advise “use this with caution” until you’re sure it’s got no fatal bugs. Although, since it’s from GitHub.com, I expect it’s pretty robust.
There’s good reasons for adopting Mongo, I’m unconvinced (but open-minded) that performance is one of them. Here’s a ROFLMAO viewpoint on it:
“If your write fails, you’re ****ed”
Obviously, MySQL’s not perfect, but in most cases I’ve seen, it’s been lack of competence on the developer side, and the lack of basic DBA skills – not problems with MySQL itself – that’s broken scalability. In which case, I’m a little suspicious that a company that fails to scale MySQL will equally fail to write their code correctly on Mongo. In many ways, throwing away SQL makes it much easier to prevent scalability…
Looks like a “normal” KickStarter project for a new Tower Defence game.
Halfway through the demo video, it switches to “here’s how I’ve been using GA to detect game-design flaws, and to test ideas in tweaking game design”.
Something I’ve wanted to do for more than a decade, but could never find a company who’d take it seriously :). I really hope this iPad game does well – would be great to see a poster-child / real-world demonstration of a workable technique here.
As a free-time project, I’ve been writing a Risk clone (*) for iPad.
One of the bits I like best right now is that you can give it the URL of *any* SVG file on the web, and it automatically turns it into a Risk map.
(e.g. all the maps in Wikipedia articles are SVG files – it’s a common file format with good browser support)
This was one of those “interesting” technical challenges – I had to find an algorithm that would automatically work out which territories a human would “assume” were connected to each other.
I’m using an open-source SVG library which works fine for basic SVG files but has a lot of bugs with the more esoteric ones. I’ve already fixed a few of the major bugs (they’re now merged into the GitHub project) – but I’d like to get more SVG files to test with.
The one thing to bear in mind is that the colour-data gets wiped when it imports. So … SVG files that make heavy use of different colours or gradient-fills/pattern-fills lose detail when imported.
Also, files where none of the elements are close enough to be deemed “connected territories” … work poorly.
Everything else works fine.
So … if you’ve got any, please post a comment here with URL, or email them to me directly (address in the About link at top of this page).
(*) – I say “clone” because it’s the same genre – but the gameplay is “fixed” quite a lot. If you once loved Risk, but grew to hate it, you’ll see why I wanted to change the baic game design :).
What happens when you get 2 developers working together, sharing their source? What about 10? … or a 100?
There was a dream, 20 years ago, that the total would be greater than the sum of the parts. That developers could *re-use* each-other’s code.
Sadly, that dream – in 2012 – is poisoned.
What I’m going to describe here happens a lot – although in absolute terms, I hope it’s just a drop in the ocean. Maybe it’s nothing to worry about. Or maybe … well. In the last 15-odd GitHub projects I’ve tried to use, it affected more than a third of them. Such tiny stats are statistically meaningless, of course – but if you look at the causes of this, I think it’s more likely part of a general trend – and that really is worrying.
So. What’s going on?
I love GitHub, I’m a paying member (and I regularly sell it to clients and colleagues) but … in some ways, it’s IMHO actively preventing collaboration.
Just to be clear: it doesn’t have to be this way – you can run your own projects on GitHub and prevent this happening.
But … GitHub makes this the path of least resistance, and that means – in the world of Open Source – it’s the path that gets most followed
When you fix a bug on GitHub, you have to wait for the original project author to “accept” your fix.
If they don’t accept it, as far as collaboration goes: you’re screwed. There is no “plan B” for collaboration.
Your only option is to tell the world:
“Stop using his project! It sucks! Use my project instead! I promise I’ll be a better merger!”
But then … if *you* stop accepting fixes for a while, one of the developers fixing YOUR bugs will have to do the same thing.
And each of these “Stop! Use mine instead!” calls is one-way: once another developer who’s making use of the source moves to a sub-fork, they can never go back. In theory, the original Author could do a back-dated merge … but in reality, that won’t happen, because of the cost involved:
In practice, that’s more expensive than a normal person can afford, in terms of time and effort.
For each SubAuthor they want to back-merge with, they have to check every single change that person has made … against every change that they’ve merged already, from every single source. Otherwise they break the previously-merged code. Usually, each individual SubAuthor makes an incompatible change sooner or later – and so prevents the original Author from ever merging with them.
It’s no surprise – usually by this point the Sub Author has given up on the original Author (can you blame them? the Author has disappeared and ignored merge requests for months or years by this point)
So, in practice, very few GitHub authors (so far: none that I’ve seen) re-merge SubAuthor projects once the SubAuthor has really got going. On the projects I’ve been involved in, when a popular SubAuthor disappears for a while, there’s been a desperate scramble by the SubSubAuthors to find the guy/gal and beg/bribe/bully them into merging – otherwise we know that our combined efforts are about to be blown up.
What? Well …
Say you have Author “A”, and 3 people making changes and fixes to the code (“B”, “C”, and “D”).
At first, while A accepts merges quickly, B, C and D are all sharing code together – in practice, they are collaborating. However, they are not truly sharing code – GitHub does not allow this – they are sharing code with a Master (A), who is forwarding their work to all 3 of them.
When A disappears, B C and D can no longer collaborate. If A disappears with merges pending … then B/C/D find they have 3 distinct codebases, and no way within GitHub to do a simple cross-merge.
Now, the situation is not lost – if B, C, and D get in contact (somehow) and negotiate which one of them is going to become “the primary SubAuthor” (somehow), and they issue manual patches to each other’s code (surprisingly tricky to do on GitHub) … then they can resume collaboration. I’ve done this myself – it works. But it’s massively more complex than the process they were using before, which was *one-click-merge*.
In practice, at this point B/C/D will stop collaborating. Sad, but true. This happens over and over again on GitHub projects – when a SubAuthor arises, the other collaborators stop collaborating and become new SubAuthors in their own right.
Often it feels like watching a religion split, with each of the senior priests declaring themself “the new Prophet”, and going forth to spread (their) word…
In theory, GitHub is wonderful.
But the combination of its bad design around some core use-cases, and its intransigence when it comes to the VERY common case of a single person disappearing … have lead to the point where I believe it’s killing projects. This is a gross generalization – and not every project that loses its Author will get this problem – but I’ve encountered more and more “dead” projects on GitHub over the course of 2011.
Of course … the way GitHub is designed, *those projects do not appear to be dead*. Often they appear to be very much “alive” – there’s tonnes of activity.
But all that activity is going on in radically different and massively incompatible forks. It’s wasted time and energy, it’s programmers fixing the same bugs – multiple times – because they are NOT collaborating any more.
In the case I cited at the start, 100-plus developers have (probably) re-written the same fixes for the same problems.
i.e. the total effect of this project is tending towards ONE HUNDRED TIMES less than the sum of its parts.
Note: LESS … not more!
There’s some value there, still – anyone can come along and start from the original project and make their own fork. But it’s a sad and sorry fraction of what the Open Source world dreamed of when the word Collaboration was fresh and exciting.
This is UnCollaboration. And its becoming depressingly common.
(I’m prototyping a new game (working title: “ChessQuest”) – original post here)
Major changes:
http://www.java-gaming.org/topics/lwjgl16k/25093/view.html
“the LWJGL16k competition starts right here, right now.” – Cas
First deadline: 25th November 2011
First task: get a black screen running using LWJGL
“you’ve got 7 days. All I’m looking for at this stage from you is a blank window opening up and maybe a rotating square or whatever. Of course complete games are even more welcome but the idea is to get something shipped.”
Well, what are you waiting for?
If you’ve got Eclipse installed, all you need do is download the LWJGL library, copy/paste the 50-line minimal project from the Wiki, and submit your entry!
(I believe Cas is onto a good thing here … force people to realise how easy it is to make a game if you focus on small-but-visible steps done *quickly* – No more procrastination!)
UPDATE: So … it seems Suffusion has (buried deep inside the config) a way of displaying category pages with a higher-level workaround to the WP “missing feature”. Although so far I can’t find a way to set the settings on individual pages (which is what you generally need). So, I’ll leave this post up – it’s a good starting point for customizing Suffusion’s page-of-pages (which is also, it seems, how you would customize its Category pages.
Just to be clear, this is not a bug in Suffision, although it’s an oversight and I hope it’ll be included in a future version. The core problem is in WordPress: people have been requesting/expecting this feature for the past 5+ years, it’s pretty basic, but for now the WP folks haven’t added it.
This is VERY frequently requested by WP users: you want to have a Page on your site which lists the posts that are in a single Category.
WP has a very low-level way of doing this – which is very error-prone, hard to maintain (it’s hard-coded to database ID’s!), and requires you to break every single theme you own. Every time the theme is updated, you have to manually re-implement the fix!
Fortunately, there’s a minimal workaround (which is documented in the WP docs now (scroll to bottom)) which still (!) requires some theme editing.
Fortunately, Suffusion already has most of this workaround implemented, so we can make the fix without screwing around with the theme so much. The source code in Suffusion is almost identical to the sample provided by WP – but unfortunately it’s missing a key part. In Suffusion, you CAN list posts on a Page, but you CANNOT filter those posts by category.
So, edit the file “posts.php”, titled (in Suffusion): “Page of Posts”.
Where it has these lines:
$args = array(
'orderby' => 'date',
'order' => 'DESC',
'paged' => $paged,
);
…instead copy/paste the following:
$category_exclusive = get_post_meta($posts[0]->ID, 'category', true );
if( $category_exclusive )
{
$cat = get_cat_ID($category_exclusive);
$args = array(
'category__in' => array( $cat ),
'orderby' => 'date',
'order' => 'DESC',
'paged' => $paged,
);
}
else
{
$args = array(
'orderby' => 'date',
'order' => 'DESC',
'paged' => $paged,
);
}
As per WP’s official suggestion … if you want a page to filter by category:
One major thing here: We are being SAFE and non-destructive to the Suffusion theme.
Critically important is that we ONLY do a “filter by category” if the Page itself requests it. WP’s sample code does NOT have this protection (which is why the code above is slightly different).
This means that the rest of Suffusion (which doesn’t know about our new feature) is unaffected, and works as normal
Someone just wrote this comment on one of my StackOverflow answers:
Fundamentally, this question/answer pair is saying:
“Apple: one of your non-programming managers made a stupid mistake in one of your core tools, one used every day by hundreds of thousands of people; since you won’t fix it, here’s a (tricky) workaround that anyone can use”
Apple “doesn’t do” anything open, doesn’t do community support, or community development – you’re not even allowed to know if you’re the first person to report a bug, or the millionth.
But if they did, just imagine how much better their tools would be, and how much more productive the iOS developer community would be…
(I’m prototyping a new game (working title: “ChessQuest”) – original post here)
A couple more hours work, a few more changes:
Steve Yegge’s Google Platforms Rant is not so much a rant as a sign of someone fighting an inappropriate culture. We saw stuff like this a lot at NCsoft when people were trying to turn around the $100 million clusterf*ck that created hundreds of redundancies all the way to director level.
It’s a great article, but a couple of the key points resonated with my own experience of Google UK’s hiring practices a couple of years ago. There was clearly a lot wrong with the internal culture, but as an outsider I couldn’t quite put my finger on it. Here’s the crux of Steve’s post (but seriously – read the whole thing, it’s rich and meaty):
That one last thing that Google doesn’t do well is Platforms. We don’t understand platforms. We don’t “get” platforms. Some of you do, but you are the minority. This has become painfully clear to me over the past six years. I was kind of hoping that competitive pressure from Microsoft and Amazon and more recently Facebook would make us wake up collectively and start doing universal services. Not in some sort of ad-hoc, half-assed way, but in more or less the same way Amazon did it: all at once, for real, no cheating, and treating it as our top priority from now on.
But no. No, it’s like our tenth or eleventh priority. Or fifteenth, I don’t know. It’s pretty low. There are a few teams who treat the idea very seriously, but most teams either don’t think about it all, ever, or only a small percentage of them think about it in a very small way.
It’s a big stretch even to get most teams to offer a stubby service to get programmatic access to their data and computations. Most of them think they’re building products. And a stubby service is a pretty pathetic service. Go back and look at that partial list of learnings from Amazon, and tell me which ones Stubby gives you out of the box. As far as I’m concerned, it’s none of them. Stubby’s great, but it’s like parts when you need a car.
…and finally, reading that, it clicked for me what I saw that was so wrong:
“It’s a big stretch even to get most teams to offer a stubby service to get programmatic access to their data and computations. Most of them think they’re building products.”
That pair of sentences, back to back, is the problem: people outside Google would put the word “except” in between. Googlers put the word “because” in between. Google’s cultural definition of Product has got lost and perverted somewhere along the way, and now looks and smells like the real thing but is – to the rest of the world – a fake. Except Google – internally – can’t see this.
Every Googler I talked to worshipped at the altar of Product-as-King; three quarters of them would then – even in the same sentence – admit that they hated Product, didn’t believe in it, and felt it was a waste of time — “get out of my face with your product BS, and let me write beautiful code in my Ivory Towers, and leave me alone”.
They were smart people; they never said this explicitly (although one came very close – and you could see the moment when he had the thought: “oh crap; if anyone else hears I said that…”, then backtracked very hastily) – instead they frequently said mutually conflicting things, and dressed them up in enough abstractions that you could pretend that they weren’t conflicting. They were very good at it – I could tell there was a crack, but I couldn’t work out where the fault-line lay.
As Steve puts it later on:
Google+ is a prime example of our complete failure to understand platforms from the very highest levels of executive leadership (hi Larry, Sergey, Eric, Vic, howdy howdy) down to the very lowest leaf workers (hey yo). We all don’t get it. The Golden Rule of platforms is that you Eat Your Own Dogfood. The Google+ platform is a pathetic afterthought. We had no API at all at launch, and last I checked, we had one measly API call. One of the team members marched in and told me about it when they launched, and I asked: “So is it the Stalker API?” She got all glum and said “Yeah.” I mean, I was joking, but no… the only API call we offer is to get someone’s stream. So I guess the joke was on me.
Product. Platform. Since when have those been mutually exclusive? Not in this Millennium, I believe…
And even when Google gets over their hatred of Platform, even with something as simple as the pixels that their apps put on screen, they’ve jumped the shark:
You know how people are always saying Google is arrogant? I’m a Googler, so I get as irritated as you do when people say that. We’re not arrogant, by and large.
But when we take the stance that we know how to design the perfect product for everyone, and believe you me, I hear that a lot, then we’re being fools. You can attribute it to arrogance, or naivete, or whatever — it doesn’t matter in the end, because it’s foolishness. There IS no perfect product for everyone.
And so we wind up with a browser that doesn’t let you set the default font size. Talk about an affront to Accessibility. I mean, as I get older I’m actually going blind. For real. I’ve been nearsighted all my life, and once you hit 40 years old you stop being able to see things up close. So font selection becomes this life-or-death thing: it can lock you out of the product completely. But the Chrome team is flat-out arrogant here: they want to build a zero-configuration product, and they’re quite brazen about it, and Fuck You if you’re blind or deaf or whatever. Hit Ctrl-+ on every single page visit for the rest of your life.
It’s not just them. It’s everyone.
Any genuine Product person would run screaming from that situation – there’s nothing salvageable. It’s like someone coming to you with their design for a chocolate teapot: “Once you’ve had your tea, you can have a tasty chocolate treat too!”, leaving you wondering: where do I even start with trying to explain to this person what they’re missing?
With Scrum, you’re constantly focussing on:
How does the application look / work for the user *right now*?
…to the extent that you care more about “does this feature work for the user?” than “is the code/art/architecture for this feature ideal?”.
We regularly get situations where a feature *appears* to work, to a casual observer – but on deeper inspection, it’s clearly broken in one or more significant ways. Sometimes, the “broken” parts are so obscure that you’d need help to even find them. Other times, they’re obvious if you try to to use the feature more than just once or twice.
In Scrum terms, it’s pretty clear what’s gone wrong: the Product Owner didn’t describe the feature clearly enough (they implicitly included functionality they didn’t really care about, … or they described it too vaguely to be implemented well).
Scrum’s in-built check/balance against that is the Team. The developer who adopted the task should have rejected it during the Planning meeting, should have insisted on a clearer User Story (or a more explicit feature description).
But in the real world, this stuff happens. Leaving the issue: What do you do next?
Here’s an approach I’ve been experimenting with recently.
When it happens, you split the feature description in half, re-defining one half as the part which is done + working, and the other half as the part which isn’t working. Or into 3, 4, etc – if there’s multiple “player visible” ways in which it’s not working.
This seems to work pretty well – it lets us independently prioritise “the bit we’ve already got (hence: zero extra dev cost)” and “the hard stuff that’s not working”. And quite often, we end up redesigning some other part in a way that makes the broken edge-cases no longer exist – so we never need to fix them.
…but I’m still experimenting with it. I’m sure we could do our Planning meetings better – both from the PO side (more detailed descriptions, more PO planning) and/or from the developer side (more questioning, demands for more detail).
Response from ImageMagick folks, when I asked them to either re-instate the working binaries, … or stop building as Lion-only:
“We only host and maintain current versions of ImageMagick on one OS
release level. We have a small development team and do not have the
time to support multiple releases and multiple OS levels. The fix is to
download the MacPorts version of ImageMagick which runs under Leopard.
Another solution would be to donate a Mac with Leopard installed so we
can create binaries. We only have one Mac and it hosts Lion.”
Fine – it’s their software, they can do whatever they want. And I think they’ve done a great thing over the years by sharing this command-line tool with the world.
Except … their alternatives aren’t as reasonable as they sound.
Firstly, MacPorts is incredibly difficult to use (even as a former sysadmin and programmer, I find it painful). Simply put: I know it will take me at least a day to get that working, possibly several.
Secondly, deliberately deleting their own working software, and replacing it with non-working software, is deeply irresponsible. If this is how they approach the overall product, how long before you get “caught out” as a user when they pull some other rug out from under you? “Using ImageMagick today? Well – get it while you can, because tomorrow, they might arbitrarily delete it.” (this is what just happened to me: in the space of a few weeks, the first version I downloaded was deleted and replaced with a knowingly-broken version. My backup copy got corrupted, and I thought I could re-download from the web – nope!)
Asking them about this, they pointed out that the version from a few weeks ago had a bug which was a potential security hole. Fine, so they should discourage people from using it – but that doesn’t excuse *deleting* it, and providing only upgrade paths that are painful or expensive (Lion is not free).
It pains me to say this – as noted above, I think the IM product has been a great thing – but I have to conclude:
Don’t use ImageMagick. Just when you need it, it’s liable to let you down.
As for me, I see no other choice but to give Adobe more money, buying a more expensive copy of Photoshop that I don’t really need. I can’t afford to waste days fiddling around with MacPorts – and not even be guaranteed of success. I just need to do one, tiny, simple operation (an image resize!), but unless I can find a kind person who’s got an archived copy of ImageMagick, it’s not going to happen :(.
Oh, well.
The IM maintainers seem to be taking a leaf out of Apple’s book: if you don’t purchase the latest Apple OS upgrade (that most people don’t need), you can no longer use their software.
If you follow their 4-line install instructions, you’ll get:
dyld: Library not loaded: /usr/X11/lib/libpng15.15.dylib
Referenced from: …. /ImageMagick-6.7.2/bin/convert
Reason: image not found
…because that precise version of that library isn’t included in OS X generally, it’s only part of Lion, and it’s not included with ImageMagick – and ImageMagick for some reason has been compiled to refuse to run with anything except that precise version. Why? no explanation of this on the download page. I assume it’s just someone wasn’t paying attention, and went and linked a specific library. If so, it’s very frustrating that a simple noob mistake has just locked out anyone who’s not running Lion. Sigh.
Yes – I could go and download the source, and debug it, and fix it myself, because it’s Open Source. But if I’m going to consider that, then it would be cheaper to:
And what about if I were running this on a server (which, after all, is what IM is really here for)? Basically, I’d just be screwed :(. Unless I was happy building from source, with all the pain and suffering that entails.
Overall, it gives me the strong feeling: stop using ImageMagick. It’s too risky. Which is very sad, because in the past it’s been very popular in some (server) teams I’ve worked on, where it helped us do lots of great things (and, IIRC, some of the people I worked with contributed some small minor fixes back to IM. Although this was so long ago I might be imagining that).