UPDATED 24/09/13: Added some essential details to the class files at end of post, and corrected typos
UPDATED 24/09/13: Added Github project with full source from this article
This is Part 2. Part 1 was an overview of where Apple’s GLKit helps with OpenGL ES setup and rendering.
I’ve been using OpenGL ES 2 less than a year, so if you see anything odd here, I’m probably wrong. Might be a mix-up with desktop OpenGL, or just a typo. Comment/ask if you’re not sure.
Apple uses OpenGL extensively – their 2D desktop libraries (Quartz, CoreAnimation) are OpenGL-based, and iOS has inherited that link (along with some funky tricks you can pull).
December 2013: I’ve converted the sample code from these articles into a standalone library on GitHub, with the article-code as a Demo app. It uses the ‘latest’ version of the code, so the early articles are quite different – but it’s an easy starting point
iOS developers often don’t have an OpenGL background, and most of the ones I work with like the idea of OpenGL, but feel they don’t have the time to learn and master it. However, the combination of “new API (OpenGL ES 2)” with “Apple’s helper classes (GLKit)” makes OpenGL development on mobile suprisingly fast and easy.
A few years ago, Jeff LaMarche wrote a series of simple, easy tutorials on getting started with OpenGL ES 1. In the same spirit, I’m going to write about GL ES 2. In the land of Khronos (the OpenGL standards group), version “2” is fundamentally different to version “1” (this is a good thing: ES 1 was outdated from the start, based on 1990s OpenGL. ES 2 is a modern API, based on 2000’s OpenGL)
I’ll be covering this from an iOS perspective. There are lots of GL ES tutorials out there, but for non-iOS platforms the challenges (and solutions) are different.
With iOS 5.0, Apple introduced a new Framework: GLKit. GLKit’s apparent aim was to fix ES 2.0 to make it simpler and easier for non-OpenGL experts to use.
It’s not live yet, and I’d hoped Apple might update GLKit to fix some of the holes below. I don’t know what will be in the final iOS 7 release, but I get the impression GLKit hasn’t changed yet. If so, everything that follows applies equally to iOS 7.
| Porting GL ES 1.x code | |
| Full |
Apple provided a good, easy-to-use set of classes to make it trivial to port GL ES 1.x code. The documentation is very poor, though.
More importantly: it prevents you from using Shaders, which are one of the easiest and most powerful parts of OpenGL (once you get up and running). So, we’ll be ignoring GL ES 1.x Classes: anything with the text “Effect” in the class name (GLKBaseEffect, GLKEffectProperty, GLKNamedEffect, etc) |
| Vector math | |
| Full |
Every desktop OpenGL implementation assumes you have access to a good Vector Math library.
Until GLKit, Apple’s iOS was the exception: you had to make your own “vector” classes, you had to write all the Dot Product code, etc. (e.g. c.f. Jeff LaMarche’s first tutorial). Not any more :). Apple’s design here is good, and closely follows the conventions of their other frameworks (works the same way as CGRect, CGPoint etc from Quartz/CoreGraphics) Structs: GLKVector2, GLKVector3, GLKVector4 |
| Quaternions | |
| Full |
Quaternions have a bad rep. Many people find them incomprehensibly hard to understand/code, and yet … they are essential once you start “rotating 3D objects”.
Apple’s implementation of Quaternions is excellent: you don’t need to understand the mathematics, just use the pre-built methods |
| Matrix math | |
| Full |
Like Vector-math, Matrix math is tricky and time consuming to build for yourself and debug. Apple’s done all of it, with an good set of methods. Structs: GLKMatrix4, GLKMatrix3 |
| OpenGL Projection | |
| Partial (almost full) |
OpenGL uses 4-dimensions to deal with 3-dimensional rendering. That could get difficult, fast. Skipping the reasons for it, OpenGL used to be hardcoded to a set of special matries (M, V, and P – model, view, and projection).
GL ES 2 threw away the hard-coded matrices, and says “do it all yourself” (which, as we’ll see later, actually makes things easier in the long run). This is a pain … except Apple’s done it for us. Don’t go writing your own MVP stack code – use Apple’s. Structs: GLKMatrixStack |
| Texture loading | |
| Partial (poor) |
See post: “Part 6”
Before GLKit, you had to write long and complex methods, using CoreAnimation and Quartz, to convert “file.png” and upload it to the graphics chip as an “OpenGL texture”. That code was hard to debug, and most iOS programmers aren’t familiar with CA/Quartz. Apple wrote a proper Objective-C texturing system that does the work of Quartz and y-flipping for you. For most “simple” code, this is perfect. …but: they screwed up in a few places, some major bugs. When it works, it’s fine – and it only needs two lines of code! – so we’ll use it in the early articles, but we’ll throw it away and write a “fixed” version for the later articles. Classes: GLKTextureInfo, GLKTextureLoader |
| Mixing UIKit with OpenGL | |
| Partial (OK) |
See post: “Part 2”
There’s a lot of FUD on the web that says this is “impossible” or “slow” or … etc. Don’t believe it. There are bugs in Apple’s CALayer / Quartz / CoreAnimation classes that make them slow *independent* of whether you’re running OpenGL. It’s just that the things people want to do when using UIKit with OpenGL are usually the ones that show up the bugs in UIKit/Quartz. We’ll cover the main gotchas, and look at how to avoid or improve them. But for the most part: it works automagically. (there’s a reason for this: UIKit is implemented on top of OpenGL, so it’s already integrated to a high standard. It’s just that Apple hid the integration points) |
| Shaders (vertex, fragment) | |
| None |
See post: Part 3 See post: “Part 6” GLKit pretends that shaders don’t exist. The most important feature of OpenGL ES 2.0 – and Apple ignored it. Sad, but true. We’ll fix that. |
| Multithreading, context-switching | |
| Full |
OpenGL supports multi-threading, background loading, all sorts of funky stuff.
Although it’s not strictly part of GLKit, Apple has re-used their old EAGLContext class to provide access to all of this. This is probably because it worked fine in the first place. However, to be clear: if you’re used to EAGLContext, it’s still used EVERYWHERE by GLKit. Classes: EAGLContext |
| Multi-pass rendering | |
| None |
See post: “Part 2”
You can make a cube appear on screen, textured, using single-pass rendering. Just about everything else you ever want to do … needs more than one pass. Apple provides no support for this, so you have to write it yourself (and there’s a surprisingly large amount of boilerplate you need to write here). |
| 3D models, animation, data formats | Partial (very little) |
See post: Part 3 See post: Part 4b GLKit does one great thing with 3D data formats: it provides a Vector class that all iOS apps/libraries/source can use, and be fully compatible with each other. But it provides zero support for the rest: meshes, 3D models, VAOs, VBOs, etc. |
| Error handling and state management | |
| None |
See post: Part 4
When you have a bug in your code, GL does nothing. Nor does Apple. Sooner or later you’ll find yourself weeping in frustration. |
| Performance analysis | |
| Partial (some) |
Apple makes a lot of noise about how good Instruments is with OpenGL.
This is true, it’s good. But Apple also blocks you from accessing the hardware-manufacturers own performance tools, which may be better. If you already know Instruments inside-out (e.g. you know about the invisible RHS-panel…), you’ll be right at home. |
If you know nothing at all about OpenGL, I recommend skim-reading the first 8 of Jeff LaMarche’s posts on GL ES 1.
NOTE: a *lot* of the detail in Jeffs’ posts is now unnecessary (or superceded). But it all helps with understanding the newer/cleaner GL ES 2 (and GLKit) versions. If you get stuck or struggle, skim that post and move on to the next. Each of his posts works well standalone.
Then head on to Part 2 – Drawing and Draw calls.
Also posted to #AltDevBlog at www.altdevblogaday.com/2013/08/29/glkit-to-the-max-opengl-es-2-0-for-ios-part-1-glkit-features/
I’ve just posted some screenshots + notes for Reddit’s Screenshot Saturday, for the iPad game I’ve been working on for almost 2 years now.
Doesn’t look like much given how long it’s been in development :(, but I’m hoping it’ll speed up from here!
Along the way, I’ve:
Here’s a great post-mortem on Growtopia (launched 2012, developed by a team of two)
It’s slightly buried in there, but I spotted this:
My blog posts are info-rich and spam-poor. Most of the “enter your email address” plugins are designed for spam – covered in bling, in-your-face animations, background music, all sorts of crap.
There’s nothing out there, so I made one, using a GPL’d existing project. Feel free to use this yourself.
note: this is an image, not a form!
(see also the first set of SVGKit recipes)
[objc]
CALayer* layerUserClickedOn = …
if( [layerUserClickedOn isKindOfClass:[CAShapeLayer class]] ) // should ALWAYS be true, but just in case you write your code wrong…
{
CAShapeLayer* shapeLayer = (CAShapeLayer*) layerUserClickedOn;
shapeLayer.fillColor = [UIColor redColor].CGColor;
}
[/objc]
NB: Apple/ObjectiveC uses “id” as a keyword, so we had to rename the SVG name “id” to “identifier”.
If your SVG file is legal, and has a unique “id” attribute for every SVGElement, then you can find the CALayer directly from the SVGElement, using the “id” attribute. If your SVG file has no “id” attributes, or the SVGElement you’re searching has no “id” element, then you’ll have to do more work – or fix the SVG file (edit it, add an “id” attribute to the SVGElement that matters, save it … then use this recipe)!
[objc]
SVGKImage* svgImage = …
SVGElement* element = … // see previous SVGKit recipes
CALayer* layer = [svgImage layerWithIdentifier:element.identifier];
… // do something with it…
[/objc]
First of all, find the part of the SVG you want to clone, e.g. using “Search an SVG file for particular tags / nodes / elements” from the first set of SVGKit recipes.
That gives you an (SVGElement*). Use the previous recipe to get the CALayer. Finally, clone it and use it:
[objc]
#import "CALayer+RecursiveClone.h" // requires SVGKit version 1.1.0 or later
…
-(void) someMethod
{
SVGKImage* svgImage = …
SVGElement* elementToClone = … // see previous SVGKit recipes
CALayer* layerToClone = … // see recipe above
CALayer* newLayer = [layerToClone cloneRecursively];
newLayer.frame = layerToClone.bounds; // reset it to be positioned at (0,0)
// if you want the clone to start in same position as original
// add this code
if( calculatePositionOfOriginalLayer )
{
CGFloat xOffsetOriginal = 0.0f;
CGFloat yOffsetOriginal = 0.0f;
CALayer* parentLayer = layerToClone;
while( nil != (parentLayer = parentLayer.superlayer) )
{
xOffsetOriginal += parentLayer.frame.origin.x;
yOffsetOriginal += parentLayer.frame.origin.y;
}
CGPoint positionOfOriginalLayer = CGPointApplyTransform( layerToClone.position, CGAffineMakeTranslation( xOffsetOriginal, yOffsetOriginal ) );
}
}
[/objc]
Every “shape” is stored as some form of Apple CGPath object.
[objc]
CALayer* layerInSVG = … // use a previous recipe to get the layer from the SVG element
CAShapeLayer* shapeLayer = (CAShapeLayer*) layerInSVG; // if you chose wrong, this will error at runtime
CGPath pathToEdit = shapeLayer.path;
… // edit the path, and create a replacement path
shapeLayer.path = myEditedPath;
[/objc]
Apple doesn’t provide an API to directly edit CGPath objects. They have a C API that let’s you create a callback function that turns the CGPath into a list of draw-calls, that you can then work with. But it takes quite a bit of code to read this, and then convert it back into a CGPath that Apple can draw.
So far, no-one has open-sourced their code for this (I’ve got a private version I wrote for a previous project, but it’s thousands of lines of code). Either petition Apple to add the missing APIs, or find someone to write it for you / license it to you, or write it yourself.
Today I found a new article on implementing a Bomberman clone using an Entity System (in C#) – but I feel it doesn’t quite work.
It’s a well-written article, with some great illustrations. But the authors “Components” struck me as over-complicated and too behaviour-focussed. If you’re designing a game based on an ES, this isn’t a great way to do it – it feels too much towards old-style OOP gameobjects.
This is an opportunity to talk about good and bad directions in Component design. I really like the author’s idea of using Bomberman as a reference – it’s simple, it’s a well-known game, but it’s got a lot of depth. Read the original article to refresh your memory on Bomberman and how it works. Then come back here, and we’ll do a worked example on choosing components for a game. I’ll completely ignore the programming issues, and only look at the design issues.
OK, a completely different approach this time (based on the “new technique for Render to Texture” that I mentioned last time). And it works a lot better – lighting, shaders, etc is all there for free (but something is wrong with the side-to-side)
UPDATE: see end of post for another idea…
One of the ways they make you buy Unity Pro is gimping the non-Pro version by taking away Stencil Buffer and Render-to-Texture (needed for … many (most?) of the truly interesting effect you can think of). Except … it’s only needed by old-school OpenGL programmers, because everything RtT does can be mimicked in Shaders. Stencils … maybe also?
Anyway, I’m working on a hobby project and don’t need or want the Pro version for now (I’d like to support Unity, but these days the jump in price is way too expensive, at $X000 (they price-gouge you if you’re purchasing from within EU) for a simple hobby project). If/when we’re using it at work, and I have lots of cash, sure – but for now: can’t find that much money.
For the most part, I’m happy to go without RtT for now – generally speaking, hobby projects don’t “need” those kind of effects (and if they do: maybe it’s not a hobby any more?). There’s just one exception: reflections.
I never imagined I’d reach anything even close to 10k rep. Lots of thoughts and some analysis to come on this in a future post – but I’ve got two deadlines coming up, so very rushed right now.
Everyone has had that dreaded experience: you open up the task manager on your computer… and there’s a program name you don’t recognize. It gets worse when you google the name and can’t find a concrete answer on what it is and why it’s there. It gets even worse when you remove it from Autoruns and it comes back. It gets terrible when you realize it has keylogger functionality. The icing on the cake, however, is when the mystery program is also eating up all your RAM.
…
The application does not store or transmit or even display the information polled. It does nothing. I spent the better part of two hours scouring the obscure corners of the binary, thinking surely I must be missing some cleverly hidden method that actually uses this data. I couldn’t find one.
…
that still leaves the question of why it’s doing this at all. The clues are there, vestigial remnants of removed code, exciting to any Executable Archaeologist: The generically named “Form1” of the application contains several widgets which are never actually displayed: a start button, a stop button, a place for displaying mouse coordinates, and a text box for displaying some other unspecified data. I believe this was originally a debugging utility used by “Spacer” engineers to calibrate the accelerometer so that it would not go off when one simply tapped on the touchscreen (triggering a mouse event or keyboard event). They didn’t bother to rigorously prevent memory leaks because it was never intended to run for more than a few minutes at a time. Somehow, through some miscommunication, a copy of this program with the logic for rendering the visuals stripped ended up on the list of utilities that needed to be kept in the final version of “Spacer’s” Windows 8 image for this model of tablet.
…and the last working version of Parse.com’s SDK isn’t listed on their website any more.
So … if you see this (which you probably will, on any non-trivial project):
Undefined symbols for architecture i386:
“_FBTokenInformationExpirationDateKey”, referenced from:
…then you MUST download and install the Facebook API into your iOS / Xcode project. Especially if you’re not actually using Facebook!
Parse.com for iOS is currently setup so that you CANNOT use ANY LIBRARIES AT ALL, unless you ALSO use the Facebook library. Oops.
A little bit of naive linking by the Parse.com engineers. C-linking is a PITA to get right, so I don’t blame them.
1. Facebook won’t let you download their API / library any more.
Instead you have to “install an application” on your system that spews to random places on your machine (where? Well … the app won’t tell you, but on the Facebook website they say it all goes in ~/Documents) – and you’re not allowed to change them.
Wrong place, wrong installer (shouldn’t be hard-coded, shouldn’t “hide” the location). And a pain to deal with, when all that was needed or wanted was a simple ZIP file…
2. Facebook’s latest SDK requires iOS 6 to even compile it – even if you’re not using iOS 6. No-one should be hard-coding to iOS 6, though – so I’m surprised that FB is targetting it as default. iOS 5 is the main target version of iOS for now.
3. Once you find a 6 SDK, you have to add a bunch of extra frameworks which don’t exist except on iOS 6, and set them to “Optional” in the “Project Settings > Build Phases > Link with Libraries” phase.
Details are on the FB iOS Getting Started page, although they’re pretty hard to find (they’re hidden inside a drop-down with an unrelated title).
(incidentally: the FB iOS install page has always been way too long, so I suspect someone decided to “tidy it up” by hiding 95% of it. I think a better solution would have been to remove all the cruft, and fix the install process :))
…anyway, once you get past all that, things go smoothly. NB: when I wrote this, I was on a hack-day at Facebook’s offices, and it took 30 minutes to get Parse’s API installed, because of the above problems. It would have been even longer if I’d not used Facebook in the past, and knew how to navigate their install page.
With my recent fixes to “auto-scaling” in SVGKit, I can now take in images, apply various effects, and lay them out in a grid:
…next step: get this stickers-game working…
This is very easy, but lots of people find this difficult with good reason!
Here’s a quick guide to how image-scaling is implemented with SVGKit, so you can effortlessly scale your images.
In app development, the most common thing people do with an SVG is “render it to fit a specific area on screen”. This is very wise – it’s making use of the core feature of Scalable Vector Graphics: resolution independence.
Unfortunately, achieving that aim is a lot less obvious than you would expect. Most of the SVG Spec is extremely well written, but for this aspect the authors “had a bad day”, and wrote some rubbish prose that’s not technically possible. This leads to a lot of confusion…
I’m trying to make SVGKit for iOS/Mac be 100% standards compliant, and the lack of specification in this area has made it very difficult. Worse … people using the library are often confused by simple items like “how do I make the SVG file fit into a small area on screen?”.
It’s been difficult to work through, and I’ve decided to document what I *think* the authors intended – and how I’m implementing it in our open-source library.
(from 1982. Blogged now because … the named individual who apparently came up with this scam)
(for those that haven’t been following: four years after Langdell tried to bully an award-winning iPhone game into giving him their money, using his invalid trademark to threaten legal action … the USPTO has finally started cancelling each of his trademarks. Trademark law is FUBAR: 4 years for a fraudulent(*) TM to be cancelled? Ouch.)
(*) – my opinion, but: read the case notes … he apparently committed blatant fraud to keep-alive a trademark that legally had already expired
For the last couple of months, one of our dev machines has been literally incapable of opening a simple Android project. It crashes every time, on startup, while displaying the Eclipse logo:
Re-installing everything had no effect. We tried everything, and the only thing that worked reliably was to keep deleting the project and re-synching from SVN every time we wanted to start Eclipse
Today I finally discovered the cause: Mylyn
Apple has finally released drivers for Windows 8 (NB: because Apple takes standard PC hardware and then customizes it, Apple customers are reliant on Apple for “custom” drivers; the manufacturer drivers don’t work)
Also: the built-in copy of BootCamp has been updated, if you upgrade OS X to version 10.8.3 or later.
To recap:
Even being generous, that’s an 11 month delay between “the world discovering that Apple hardware needed a new set of drivers, or Windows 8 wouldn’t run” and “Apple delivering the drivers”. Many Apple machines were fine, but a large proportion were effectively blocked from running Windows 8 at all. It’s good to see this finally released, but … that’s a pretty poor service from a company the size of Apple.
Worst was the iMacs, where even machines less than 18 months old were (allegedly, according to the Apple.com forums) unusable on Windows 8. You could install it (with some hacks), but then the graphics card was disabled. This means: most major software won’t run (because the graphics card is used so heavily on modern computers you got corruption of on-screen info, or just massive drops in performance, so that apps were unusable), and definitely: no games.
…which, after all, is one of the main reasons for Mac users to dual-install Windows.
Interestingly, it seems this was caused by Apple’s modifications of the ATI graphics card, so that the ATI drivers were convinced there was an external monitor (which Apple didn’t provide a socket for, so there was nothing you could do – even if you owned a second monitor).
Interesting because: in the PC world, ATi used to be famous for writing low-quality, poorly-tested graphics drivers. ATi owners were accustomed to poring over every new minor version update “just in case” it fixed the glaring bugs in the previous one – and spent a lot of time de-installing and re-installing the older versions (when the newest version frequently introduced major new bugs).
So … although it’s ostensibly Apple to blame here in being so late to fix it, my suspicion is that it was some shoddy code in ATi’s driver that (accidentally) only affected Apple-modified cards.
3rd Party Device drivers: every OS-developer’s nightmare!
| Default.png | Default-568@2x.png | ||
|---|---|---|---|
| Model | Size | Model | Size |
| 3 + 3GS | 320×480 | ||
| 4 + 4S | 640×960 | ||
| 5 | 640×1136 | ||
| Default.png | Default@2x.png | ||
|---|---|---|---|
| Model | Size | Model | Size |
| 1 + 2 | 768×1004 or 1024×748 |
||
| 3 + 4 | 1536×2008 or 2048×1496 |
||
| No status bar | with status bar | with status bar + navbar | |||
|---|---|---|---|---|---|
| Model | Size | Model | Size | Model | Size |
| 3 + 3GS | 320×480 | 3 + 3GS | 320×460 or 480×300 |
3 + 3GS | 320×416 or 480×268 |
| 4 + 4S | 640×960 | 4 + 4S | 640×920 or 960×600 |
4S | 640×832 or 960×536 |
| 5 | 640×1136 | 5 | 640×1096 or 1136×600 |
5 | 640×1008 or 1136×512 |
| No status bar | with status bar | with status bar + navbar | |||
|---|---|---|---|---|---|
| Model | Size | Model | Size | Model | Size |
| 1 + 2 | 768×1024 | 1 + 2 | 768×1004 or 1024×748 |
1 + 2 | 768×960 or 1024×704 |
| 3 + 4 | 1536×2048 | 3 + 4 | 1536×2008 or 2048×1496 |
3 + 4 | 1536×1920 or 2048×1408 |
| App | iTunes | Spotlight | |||
|---|---|---|---|---|---|
| Model | Size | Model | Size | Model | Size |
| 3 + 3GS | 57×57 | 3 + 3GS | 512×512 | 3 + 3GS | 29×29 |
| 4 + 4S + 5 | 114×114 | 4 + 4S + 5 | 1024×1024 | 4 + 4S + 5 | 58×58 |
| App | iTunes | NewsStand (if Newsstand app) | |||
|---|---|---|---|---|---|
| Model | Size | Model | Size | Model | Size |
| 1 + 2 | 72×72 | 1 + 2 | 512×512 | 1 + 2 | >= 512 |
| 3 + 4 | 144×144 | 3 + 4 | 1024×1024 | 3 + 4 | >= 1024 |
| Spotlight | Settings | ||
|---|---|---|---|
| Model | Size | Model | Size |
| 1 + 2 | 50×50 | 1 + 2 | 29×29 |
| 3 + 4 | 100×100 | 1 + 2 | 58×58 |
I’m writing an iPad game that’s designed to be self-evident and trivial to use.
Today, I made a re-usable “tell the user how many fingers to use” animation that would work well with any app. So, I decided to open-source it and stick it on github.
Screenshot from my current app (NB: you need to photo YOUR fingers – not mine! – if you want it to look like this)
Instructions are in the source code, but my intention is for this to be ULTRA SIMPLE for you to use, so here goes:
[code]
// Add this to the main UIViewController for your app – the one that displays
// when you’re ready for user interaction
-(void)viewDidAppear:(BOOL)animated
{
VFingersHelp* helpView = [[[VFingersHelp alloc] init] autorelease];
/** This will position the helpview in bottom-left corner. Bottom left/right corners are where
users prefer to "touch" (c.f. Disney’s style guide for toddlers).
While they’re watching the help, we don’t want them touching, so we deliberately place in bottom corner.
(feel free to move to top corner if you prefer).
NB: it’s easy to take a photo of your fingers and display at bottom of screen…
*/
helpView.autoresizingMask = UIViewAutoresizingFlexibleTopMargin | UIViewAutoresizingFlexibleRightMargin;
helpView.center = CGPointMake( 0 + helpView.bounds.size.width/2.0, 2.0 + self.view.bounds.size.height – helpView.bounds.size.height + helpView.bounds.size.height/2.0);
[self.view addSubview:helpView];
}
[/code]