{"id":3014,"date":"2013-12-15T00:41:25","date_gmt":"2013-12-14T23:41:25","guid":{"rendered":"http:\/\/t-machine.org\/?p=3014"},"modified":"2013-12-24T15:41:29","modified_gmt":"2013-12-24T14:41:29","slug":"opengl-es-2-textures-2-of-3-texture-mapping","status":"publish","type":"post","link":"http:\/\/new.t-machine.org\/index.php\/2013\/12\/15\/opengl-es-2-textures-2-of-3-texture-mapping\/","title":{"rendered":"OpenGL ES 2 &#8211; Textures 2 of 3: Texture Mapping"},"content":{"rendered":"<p>Recap: Texturing in OpenGL can be achieved in two separate ways, using different API&#8217;s and hardware features. The preferred, modern approach &#8211; procedural texturing &#8211; uses nothing more than a Fragment shader (this is really what Fragment shaders were created for: Texturing).<\/p>\n<p><a href=\"http:\/\/t-machine.org\/index.php\/2013\/11\/29\/opengl-es2-textures-1-of-3-texturing-triangles-using-shaders\/\">Part 5 (Textures: 1 of 3)<\/a> covered that in detail &#8211; but there&#8217;s another option: Texture-Mapping (also known as &#8220;UV Mapping&#8221;, it&#8217;s identical).<br \/>\n<!--more--><\/p>\n<h2>Cheaper ways to texture an object<\/h2>\n<p>Procedural Texturing (from the previous post) requires a lot of processing power on-board the GPU. OpenGL launched in 1992 &#8211; 5 years before GPU&#8217;s even existed for the PC. Texturing was mostly useful as a hack to make up for the poor raw power of hardware. Texture-Mapping in OpenGL comes out of that mindset &#8211; and it works a lot differently from the &#8220;simple, obvious&#8221; texturing we have today.<\/p>\n<blockquote><p>\nTexture-Mapping requires almost zero processing power, instead it sacrifices huge amounts of RAM. But the API is much lower-level than it needs to be.\n<\/p><\/blockquote>\n<p>Everything in texture-mapping comes out of this quest for performance at low cost. Texture-mapping is theoretically a subset of procedural texturing &#8211; but <strong>even modern GPU&#8217;s still contain custom hardware dedicated to texture-mapping<\/strong>. This has three main effects:<\/p>\n<ol>\n<li>Some things that can theoretically be done &#8220;either way&#8221; run faster when done using texture-mapping\n<li>Texture-mapping saves GPU processing power so that you can &#8220;spend&#8221; more of it on procedural-textures (or other Shader effects) elsewhere\n<li>Some features of OpenGL are <strong>only supported with texture-mapping, not with raw Fragment Shaders<\/strong>\n<\/ol>\n<p>Texturing has been added to OpenGL in fits and starts. I will try to consistently use these terms:<\/p>\n<blockquote><p>\n<strong>Texturing<\/strong>: the general concept of putting pixel-level detail &#8211; from any source! &#8211; onto geometry.<\/p>\n<p><strong>Texture-Mapping<\/strong>: a specific kind of Texturing that uses a source image <em>usually but not always 2D<\/em> and maps it to a 3D geometry<\/p>\n<p><strong>Texture(s)<\/strong>: the &#8220;image&#8221; part of Texture-Mapping (on CPU: an image file; on GPU: an OpenGL TextureObject)\n<\/p><\/blockquote>\n<h2>What is texture-mapping?<\/h2>\n<p>Google for more info, but the essence is:<\/p>\n<ol>\n<li>Take a piece of geometry (e.g. a triangle)\n<li>Take a 2D image (e.g. PNG file)\n<li>Make a cookie-cutter with the same number of corners as your 3D shape (i.e. another triangle)\n<ul>\n<li>NOTE: the cookie-cutter triangle DOES NOT need to be the same size\/angles &#8211; so long as it has same number of corners\n<\/ul>\n<li>When you draw the triangle to screen, use the cookie-cutter to cut out a piece of the 2D image\n<li>&#8230;and STRETCH AND SQUISH it to fit: i.e. make the three corners line up\n<\/ol>\n<p>Texture-mapping often creates distortion of the original 2D image, unless you deliberately choose your cookie-cutter to prevent it.<\/p>\n<p>Note that only the final step (the stretch\/squish) uses processing-power on the GPU. For the rest of it, the GPU can &#8220;cut out&#8221; the shape from the 2D image when the app starts running, and keep &#8220;stamping&#8221; it onto the triangle in each frame.<\/p>\n<p>Also, if you zoom-in on the triangle &#8230; the texture will get more and more blocky, because the 2D image has fixed resolution.<\/p>\n<h3>&#8230;Mapping?<\/h3>\n<p>This process of stretching and squashing the image until the corners match-up is mathematically known as a &#8220;map&#8221;, because each x,y co-ordinate in the image is &#8220;mapped&#8221; to a different x,y co-ordinate in the 3D-projected pixels.<\/p>\n<p>There are two x&#8217;s and two y&#8217;s, which gets confusing. So, traditionally, the x and y co-ords in the <em>source image<\/em> are renamed &#8220;u&#8221; and &#8220;v&#8221; &#8211; so that in code and in docs you always know which x is which. Hence the other name for Texture Mapping: &#8220;UV Mapping&#8221;.<\/p>\n<p>Before the invention of Shaders, you had almost no influence of the mapping function &#8211; it directly mapped u\/v to x\/y according to the corner-co-ordinates you attached to the 3D geometry. With Shaders, of course &#8230; the Fragment Shader &#8220;replaces&#8221; the mapping function, and lets you do any kind of weird and funky *even animated* &#8216;map function&#8217; you want.<\/p>\n<h3>Vertex Textures and GL ES 2<\/h3>\n<p>There&#8217;s more to texture-mapping than meets the eye. Some of the hardware features of TextureMapping mean that you can achieve certain special effects and high performance boosts by <strong>storing 3D geometry inside special textures<\/strong>.<\/p>\n<p>GL ES 2 supports Vertex Textures for this (textures you can access inside the Vertex Shader), but allows vendors to say &#8220;number supported = 0&#8221;. Apple used this to disable Vertex Textures on all iPhone\/iPod\/iPad&#8217;s &#8211; even though the hardware supported it. They &#8220;accidentally&#8221; enabled it for one version of iOS only (4.3), and then quickly killed it again.<\/p>\n<p>Until now. iOS7 not only adds GL ES 3 support (for the iPhone 5 only :( ), but also &#8220;re-unlocks&#8221; Vertex Textures on all devices.<\/p>\n<h2>How do you TextureMap in OpenGL?<\/h2>\n<p>First, make sure <a href=\"http:\/\/t-machine.org\/index.php\/2013\/11\/29\/opengl-es2-textures-1-of-3-texturing-triangles-using-shaders\/\">you fully understand the previous post on Procedural Texturing<\/a>, since TextureMapping is a special-case of that.<\/p>\n<p>Then there are several stages to Texture-Mapping:<\/p>\n<ul>\n<li>Preparation:\n<ol>\n<li>Upload the image to the GPU <em>(similar to uploading 3D geometry \/ vertices, creating a BufferObject\/VBO)<\/em>\n<li>Convert the image format (e.g. PNG, JPEG, etc) to the GPU&#8217;s internal format <em>(similar to telling the GPU how to interpret the contents of a BufferObject\/VBO)<\/em>\n<\/ol>\n<li>Rendering:\n<ul>\n<li>Attach each texture you need to a virtual &#8220;TextureUnit&#8221;\n<li>Enable all the TextureUnits you&#8217;re using <em>(together, these steps are similar to switching to the next VAO)<\/em>\n<li>Open the Texture inside a Shader &#8211; i.e. texture-map it to your triangles <em>(similar to invoking a Draw call)<\/em>\n<li>Read a different value of the Texture for each pixel in the triangle <em>(VERY similar but NOT identical to the trick we did earlier of using &#8220;two&#8221; varying&#8217;s to give us a texture that varied across the whole 2D surface)<\/em>\n<li>Prettify the output <em>(unique to texturemapping: hardware tricks that make them prettier, or faster, or both)<\/em>\n<\/ul>\n<\/ul>\n<p>From reading that list you may already have guessed at what we&#8217;ll use to implement this technically &#8211; it&#8217;s similar but not the same as ordinary Procedural Texturing. We&#8217;ll need:<\/p>\n<ol>\n<li>A &#8220;texture upload&#8221; system (sadly: SHOULD but DOESN&#8217;T re-use VBO&#8217;s and VAO&#8217;s)\n<li>A system for dealing with the weird beast that is TextureUnits\n<li>A way to tell the GPU which &#8220;Texture&#8221; goes inside which &#8220;TextureUnit&#8221;\n<li>A way to tell the GPU which &#8220;TextureUnit&#8221; is being used by which Shader variable\n<li>Two varying&#8217;s in our Shader (or a single vec2) that give us a 2D co-ordinate\n<\/ol>\n<p>Items 3 and 4 above require us to attach some data to the ShaderProgram &#8211; but we don&#8217;t want to duplicate this for every Vertex (huge waste of VRAM!). So we need a way of storing some variable data on the ShaderProgram itself&#8230;<\/p>\n<h2>Shader Variables: Uniforms<\/h2>\n<p>Uniforms are a special-case of Attribute. Where an Attribute has a separate value for every Vertex, a Uniform has the same value for every Vertex &#8211; they hold &#8220;global&#8221; data that affects the entire Draw call.<\/p>\n<blockquote><p>\nA Uniform is constant <strong>but only while executing a single Draw call<\/strong>. Each successive Draw call &#8211; even within the current Frame &#8211; can change the constant&#8217;s value.\n<\/p><\/blockquote>\n<p>I&#8217;ll come back to this in a later post, but the typical use of Uniforms (apart from Texture Mapping) is to efficiently re-use geometry and source code. It simplifies your source (less copy\/paste code), and reduces memory usage (e.g. you can use one &#8220;copy&#8221; of a Monster on the GPU, but draw it in many different places with small changes between them (e.g. arms and legs at different positions)).<\/p>\n<p>For now, we&#8217;re going to skim the topic lightly. A future blog post will cover them in more detail.<\/p>\n<h3>Compiling\/Linking Shaders: Store the Uniforms<\/h3>\n<p>Uniforms are very similar to Attributes; just as we had to &#8220;store&#8221; all the Attributes when we linked our ShaderProgram, we must do the same with Uniforms. The code is identical, except that the method names and Constants use the text &#8220;Uniform&#8221; or &#8220;UNIFORM&#8221; instead of &#8220;Attribute\/ATTRIBUTE&#8221;.<\/p>\n<p>I won&#8217;t go into it here, look in the GitHub source if interested. The net result is a new method on GLK2ShaderProgram:<\/p>\n<p><em>GLK2ShaderProgram.h<\/em>:<br \/>\n[objc]<br \/>\n&#8230;<br \/>\n-(GLK2Uniform*) uniformNamed:(NSString*) name;<br \/>\n&#8230;<br \/>\n[\/objc]<\/p>\n<p>GLK2Uniform itself is modelled on our existing GLK2Attribute class:<\/p>\n<p><em>GLK2Uniform.h<\/em>:<br \/>\n[objc]<br \/>\n@interface GLK2Uniform : NSObject &lt;NSCopying&gt; \/** Apple&#8217;s design of NSDictionary forces us to &#8216;copy&#8217; keys, instead of mapping them *\/<\/p>\n<p>+(GLK2Uniform*) uniformNamed:(NSString*) nameOfUniform GLType:(GLenum) openGLType GLLocation:(GLint) openGLLocation numElementsInArray:(GLint) numElements;<\/p>\n<p>\/** The name of the variable inside the shader source file(s) *\/<br \/>\n@property(nonatomic, retain) NSString* nameInSourceFile;<\/p>\n<p>@property(nonatomic) GLint glLocation;<br \/>\n@property(nonatomic) GLenum glType;<br \/>\n@property(nonatomic) GLint arrayLength;<\/p>\n<p>@property(nonatomic,readonly) BOOL isInteger, isFloat, isVector, isMatrix;<\/p>\n<p>-(int) matrixWidth;<br \/>\n-(int) vectorWidth;<\/p>\n<p>@end<br \/>\n[\/objc]<\/p>\n<p>Please note:<\/p>\n<blockquote><p>\nI have implemented NSCopying on GLK2Uniform &#8211; this is very, very useful (allows you to use them as Keys in an NSDictionary) &#8211; but easy to get wrong if you&#8217;ve not done it before in Objective-C. See the source of GLK2Uniform.m for details of <strong>the three methods<\/strong> Apple requires you to override.\n<\/p><\/blockquote>\n<h3>Setting the Uniform with GL calls<\/h3>\n<p>With Attributes, the amount of data scales up with complexity of the 3D object, and quickly becomes measured in megabytes for complex 3D levels and characters\/objects. It&#8217;s so rare to send small amounts that OpenGL only provided methods for &#8220;sending a whole load at once&#8221; &#8211; this gave us all the complexity of BufferObjects \/ VBO&#8217;s.<\/p>\n<p>In contrast, the number of Uniforms is low &#8211; typically only a few tens per Draw call. So GL lets you set individual Uniforms with a single call:<\/p>\n<blockquote><p>\nglUniform1\/2\/3\/4\/i\/f\/v: sets a Uniform for the currently-selected ShaderProgram\n<\/p><\/blockquote>\n<p>GL knows what the uniform is, but you still have to (over-)specify it. For Texture-Mapping, only one of those methods is used &#8211; glUniform1i &#8211; because we&#8217;re only using it to set a single &#8220;tag&#8221; (magic number) at a time.<\/p>\n<h2>Creating and uploading Textures<\/h2>\n<blockquote><p>\nTexture-Mapping is <em>old<\/em>. It pre-dates VBOs, VAOs, etc &#8211; otherwise it would <em>probably<\/em> have re-used those features. Instead, it <em>reproduces them, with subtle differences<\/em>\n<\/p><\/blockquote>\n<p>Today most software uses JPG, PNG, etc &#8211; but GPU&#8217;s save money by using simple hardware that doesn&#8217;t support these. So &#8230; Uploading textures to the GPU requires you to handle every low-level detail of parsing the image format and converting it into a &#8220;raw bytes&#8221; format that GL understands.<\/p>\n<p>Lots of GL ES tutorials have you write a texture-loader that will convert e.g. PNG or UIImage into raw bytes, using arcane invocations of CALayer \/ CoreAnimation.<\/p>\n<p>But &#8230; Apple&#8217;s already done it for you. Check out GLKit&#8217;s <a href=\"https:\/\/developer.apple.com\/library\/ios\/documentation\/GLkit\/Reference\/GLKTextureLoader_ClassRef\/Reference\/Reference.html\">GLKTextureLoader<\/a> class. As Apple put it: &#8220;The GLKTextureLoader class simplifies the effort required to load your texture data.&#8221;<\/p>\n<h2>Texture Units and &#8230; kill me now<\/h2>\n<p>I hate texture-units. Ideally you would say to GL:<\/p>\n<blockquote><p>\n&#8220;For this Draw call, use this Texture&#8221; (<strong>I wish!<\/strong>)\n<\/p><\/blockquote>\n<p>Unfortunately, they didn&#8217;t invent the GPU-side &#8220;Texture&#8221; object until OpenGL 1.1. OpenGL launched with a much crappier system: the TextureUnit.<\/p>\n<p>Instead, you have to say:<\/p>\n<blockquote><p>\n&#8220;For this Draw call, take a Texture, put it in a TextureUnit. But only while rendering. When not rendering, don&#8217;t. Take a number &#8211; that is not the TextureUnit&#8217;s name, but &#8220;implies&#8221; the TextureUnit, and put it in a Shader-Uniform. In the Shader, use the Uniform in a magical way as if it&#8217;s a function (not a variable!). (<strong>I HATE TEXTURE UNITS<\/strong>)\n<\/p><\/blockquote>\n<p><strong>TL;DR: We need a way to assign a magic-number to the ShaderProgram, that OpenGL understands to mean a specific TextureUnit. We also have to &#8220;load&#8221; that TextureUnit with the right Texture &#8230; and repeat this before every Draw call<\/strong><\/p>\n<h2>Access a texture-map inside a Shader<\/h2>\n<p>Now we flee from the evils of TextureUnits, and zoom forwards in time by 10 years, to the advent of Shaders. Shaders do texture-mapping in a sensible and easy way.<\/p>\n<p>Previously, to access an attribute, you simply read its value:<\/p>\n<p>[c]<br \/>\nmediump vec4 attribute positionAttribute;<br \/>\n&#8230;<br \/>\n{<br \/>\ngl_Position = positionAttribute;<br \/>\n}<br \/>\n[\/c]<\/p>\n<p>On the CPU, Uniforms and Sampler2D&#8217;s are identical, and are &#8220;set&#8221; the same way: A Sampler2D is defined as a 1-dimensional integer Uniform.<\/p>\n<p>On the GPU, &#8220;uniform&#8221; and &#8220;sampler2D&#8221; variables are again the same &#8211; except that sampler2D variables can be passed to the Shader function &#8220;texture2D(&#8230;)&#8221;, allowing it to &#8220;find and read from&#8221; the correct Texture on the GPU.<\/p>\n<p>Shaders assume:<\/p>\n<ol>\n<li>Your Sampler2D &#8220;Uniform&#8221; is merely a pointer &#8230;\n<li>&#8230; to the contents of a Texture Unit.\n<li>The TextureUnit contains a Texture &#8230;\n<li>&#8230; which is a 2D image you previously uploaded\n<li>When you invoke &#8220;texture2D(&#8230;)&#8221;, OpenGL accesses the named TextureUnit, reads the Texture &#8230;\n<li>&#8230; and Maps it to the current Draw call&#8217;s geometry\n<li>&#8230; and gives you back the Colour for the current Pixel in the Fragment Shader\n<\/ol>\n<p>All of that is represented in Shader source code by:<\/p>\n<p>[c]<br \/>\ntexture2D( samplerForMyTexture, vec2( x, y ) ); \/\/ returns a Colour at position (x,y) inside the Texture<br \/>\n[\/c]<\/p>\n<p>Simples.<\/p>\n<h2>Putting it together: A TextureMap instead of a Procedural Texture<\/h2>\n<p>First, create a new Draw call, setup 6 vertices (for 2 triangles), and setup X,Y values for our Varying (all as per last time):<\/p>\n<p><em>ViewController.m<\/em>:<br \/>\n[objc]<br \/>\n-(NSMutableArray*) createAllDrawCalls<br \/>\n{<br \/>\n&#8230;<br \/>\n\t\/* draw a TEXTURED pair of 2 triangles onto the screen, arranged into a square (&quot;quad&quot;)<br \/>\n\t *\/<br \/>\n\tGLK2DrawCall* drawTexturedQuad = [[GLK2DrawCall new] autorelease];<br \/>\n\tdrawTexturedQuad.numVerticesToDraw = 6;<br \/>\n&#8230;<br \/>\n\t&#8230;NB: must create ShaderProgram here, and call glUseProgram, as in previous posts<br \/>\n&#8230;<br \/>\n\tGLKVector3 cpuBufferQuad[6] =<br \/>\n\t{<br \/>\n\t\tGLKVector3Make(-0.5,-0.5, z),<br \/>\n\t\tGLKVector3Make(-0.5, 0.5, z),<br \/>\n\t\tGLKVector3Make( 0.5, 0.5, z),<br \/>\n\t\tGLKVector3Make(-0.5,-0.5, z),<br \/>\n\t\tGLKVector3Make( 0.5, 0.5, z),<br \/>\n\t\tGLKVector3Make( 0.5,-0.5, z)<br \/>\n\t};<br \/>\n\tGLKVector2 attributesVirtualXY [6] =<br \/>\n\t{<br \/>\n\t\tGLKVector2Make( 0, 0 ), \/\/ note: we vary the virtual x and y as if they were x,y co-ords on a right-angle triangle<br \/>\n\t\tGLKVector2Make( 0, 1 ),<br \/>\n\t\tGLKVector2Make( 1, 1 ),<br \/>\n\t\tGLKVector2Make( 0, 0 ), \/\/ note: we vary the virtual x and y as if they were x,y co-ords on a right-angle triangle<br \/>\n\t\tGLKVector2Make( 1, 1 ),<br \/>\n\t\tGLKVector2Make( 1, 0 )<br \/>\n\t};<\/p>\n<p>\tdrawTexturedQuad.VAO = [[GLK2VertexArrayObject new] autorelease];<br \/>\n&#8230;<br \/>\n\tGLK2Attribute* attributePosition = [drawTexturedQuad.shaderProgram attributeNamed:@&quot;position&quot;];<br \/>\n\t[drawTexturedQuad.VAO addVBOForAttribute:attributePosition filledWithData:cpuBufferQuad bytesPerArrayElement:sizeof(GLKVector3) arrayLength:drawTexturedQuad.numVerticesToDraw];<\/p>\n<p>\tGLK2Attribute* attXY = [drawTexturedQuad.shaderProgram attributeNamed:@&quot;a_virtualXY&quot;];<br \/>\n\t[drawTexturedQuad.VAO addVBOForAttribute:attXY filledWithData:attributesVirtualXY bytesPerArrayElement:sizeof(GLKVector2) arrayLength:drawTexturedQuad.numVerticesToDraw];<br \/>\n&#8230;<br \/>\n}<br \/>\n[\/objc]<\/p>\n<h3>Part 1: Upload a texture using GLKit<\/h3>\n<p>Unlike OpenGL, Apple supports PNG and JPG. Grab any image file you like, and add it to your Xcode project. Then send it to the GPU. In this case, I&#8217;ve named my file &#8220;textureFile.png&#8221;:<\/p>\n<blockquote><p>\nGOTCHA: Apple hasn&#8217;t documented the formats they support. I filed a bug and got confirmation that PNG and JPG are supported &#8211; others might too, but be careful.\n<\/p><\/blockquote>\n<p><em>ViewController.m<\/em>:<br \/>\n[objc]<br \/>\n-(NSMutableArray*) createAllDrawCalls<br \/>\n{<br \/>\n&#8230;<br \/>\n\t[drawTexturedQuad.VAO addVBOForAttribute:attXY filledWithData:attributesVirtualXY bytesPerArrayElement:sizeof(GLKVector3) arrayLength:drawTexturedQuad.numVerticesToDraw];<br \/>\n&#8230;<\/p>\n<p>\tNSError* error;<br \/>\n\tGLKTextureInfo* appleTextureMetadata = [GLKTextureLoader textureWithContentsOfFile:[[NSBundle mainBundle] pathForResource:@&quot;textureFile&quot; ofType:@&quot;png&quot;] options:nil error:&amp;error];<\/p>\n<p>\tNSAssert( appleTextureMetadata != nil, @&quot;Error loading texture: %@&quot;, error);<br \/>\n&#8230;<br \/>\n}<br \/>\n[\/objc]<\/p>\n<blockquote><p>\nNB: textures can and do fail to load &#8211; and when they do, you&#8217;ll waste hours trying to trace the wrong bugs in your app. Much, much safer to put this nil check at load time, and Assert immediately if a texture failed for any reason.\n<\/p><\/blockquote>\n<p>Done! You don&#8217;t even need to &#8220;convert&#8221; the image from the incoming file-format &#8211; Apple does all this automagically. When it works, GLKTextureLoader is a wonderful utility.<\/p>\n<h3>Apple Engineering: Hmm&#8230;<\/h3>\n<p>Apple&#8217;s texture-loader creates Textures on the GPU, but the things it returns are GLKTexture<strong>Info<\/strong> objects. These are not &#8220;textures&#8221; themselves, but a handle that lets you access the GPU-side texture later on. This is great &#8211; but for some reason they decided to make the TextureInfo class private and non-extensible.<\/p>\n<p>This is tragic. Every GL app needs to &#8220;create&#8221; TextureInfo objects &#8211; Apple has only implemented one out of the 5 or more different ways to upload GL texture data. I don&#8217;t mind them failing to implement those &#8211; but blocking us from implementing them in a compatible way? That&#8217;s bad.<\/p>\n<p>So, we have to create a class that reproduces the features of GLKTextureInfo &#8211; I&#8217;ve called it GLK2Texture (which says more about how we&#8217;ll end up using it, than it does about how we use it today). For this post, we&#8217;ll simply wrap the GLKTextureInfo object, but later on we&#8217;ll use it to create our own texture-loading:<\/p>\n<p><em>GLK2Texture.h<\/em>:<br \/>\n[objc]<br \/>\n@interface GLK2Texture : NSObject<\/p>\n<p>+(GLK2Texture*) texturePreLoadedByApplesGLKit:(GLKTextureInfo*) appleMetadata;<\/p>\n<p>\/** OpenGL uses integers as &quot;names&quot; instead of Strings, because Strings in C are a pain to work with, and slower *\/<br \/>\n@property(nonatomic, readonly) GLuint glName;<\/p>\n<p>\/** Creates a new, blank, OpenGL texture on the GPU.<\/p>\n<p> If you already created a texture from some other source, use the initWithName: method instead<br \/>\n *\/<br \/>\n&#8211; (id)init;<\/p>\n<p>\/** If a texture was loaded by an external source &#8211; e.g. Apple&#8217;s GLKit &#8211; you&#8217;ll already have a name for it, and can<br \/>\n use this method<br \/>\n *\/<br \/>\n&#8211; (id)initWithName:(GLuint) name;<\/p>\n<p>@end<br \/>\n[\/objc]<\/p>\n<p>&#8230;so you can modify the ViewController code above, too:<\/p>\n<p><em>ViewController.m<\/em>:<br \/>\n[objc]<br \/>\n&#8230;<br \/>\n\tNSAssert( appleTextureMetadata != nil, @&quot;Error loading texture: %@&quot;, error);<br \/>\n\tGLK2Texture* texture = [[GLK2Texture texturePreLoadedByApplesGLKit:appleTextureMetadata] retain];<br \/>\n&#8230;<br \/>\n[\/objc]<\/p>\n<h2>Part 2: Modify the DrawCall to track Textures and TextureUnits<\/h2>\n<p>You can&#8217;t use a texture directly, but you do have to create and manage them. TextureUnits also have to be managed, on a Draw-call-by-Draw-call basis. I went for the &#8220;least possible coding&#8221; approach &#8211; but it&#8217;s still got a lot of boilerplate code here.<\/p>\n<p><em>GLK2DrawCall.m<\/em>:<br \/>\n[objc]<br \/>\n@interface GLK2DrawCall()<br \/>\n@property(nonatomic,retain) NSMutableArray* textureUnitSlots;<br \/>\n@property(nonatomic,retain) NSMutableDictionary* texturesFromSamplers;<br \/>\n[\/objc]<\/p>\n<p>The number of TextureUnits is fixed and small &#8211; on most iOS devices, it&#8217;s 8. This is how many textures can be used <strong>during a single Draw call<\/strong>. You can have thousands of textures on-screen &#8211; they just have to be rendered by different Draw calls. Note that there is a significant performance cost to &#8220;switching&#8221; textures in and out of TextureUnits (this is partly why they exist &#8211; so you can optimize your switching).<\/p>\n<p>So &#8230; we pre-create an array with exactly that many slots. Each slot &#8220;represents&#8221; a TextureUnit.<\/p>\n<p>We also create a dictionary mapping <em>&#8220;Uniform from Shader&#8221;<\/em> to <em>&#8220;the Texture that the Uniform is connected to&#8221;<\/em>. For reasons we&#8217;ll come onto later, Uniforms used by TextureMapping have a different type in Shader source: &#8220;sampler2D&#8221;. So we use the term &#8220;samplers&#8221; from here on in.<\/p>\n<p><em>GLK2DrawCall.m<\/em>:<br \/>\n[objc]<br \/>\n&#8230;<br \/>\n-(void)dealloc<br \/>\n{<br \/>\n\tself.texturesFromSamplers = nil;<br \/>\n\tself.textureUnitSlots = nil;<br \/>\n&#8230;<br \/>\n}<br \/>\n&#8230;<br \/>\n&#8211; (id)init<br \/>\n{<br \/>\n\tself = [super init];<br \/>\n\tif (self) {<br \/>\n&#8230;<br \/>\n\t\tself.texturesFromSamplers = [NSMutableDictionary dictionary];<\/p>\n<p>\t\tGLint sizeOfTextureUnitSlotsArray; \/\/ MUST be fixed size, and have an entry for every index!<br \/>\n\t\tglGetIntegerv( GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS, &amp;sizeOfTextureUnitSlotsArray );<br \/>\n\t\tself.textureUnitSlots = [NSMutableArray arrayWithCapacity:sizeOfTextureUnitSlotsArray];<br \/>\n\t\tfor( int i=0; i&lt;sizeOfTextureUnitSlotsArray; i++ )<br \/>\n\t\t\t[self.textureUnitSlots addObject:[NSNull null]]; \/\/ marks this slto as &quot;currently empty&quot;<br \/>\n&#8230;<br \/>\n}<br \/>\n[\/objc]<\/p>\n<p>Next comes a big, fat, ugly chunk of boilerplate code&#8230;<\/p>\n<p>What this will do is take a pair of &#8220;Uniform (Sampler2d)&#8221; and &#8220;Texture&#8221;, and save references to both of them in this Draw call.<\/p>\n<p>It will also iterate across our &#8220;slots&#8221; (one per TextureUnit) until it finds one that is &#8220;empty&#8221; (i.e. not being used for a Texture on this Draw call already). If it can&#8217;t find one &#8211; you&#8217;ve tried to use too many textures, and we Assert.<\/p>\n<blockquote><p>\n<strong>NB:<\/strong> GL ridiculously uses two different incompatible ways of referencing TextureUnits &#8211; be very careful if you re-write this code manually. Pay close attention to TextureUnit name vs. TextureUnit index\n<\/p><\/blockquote>\n<p><em>GLK2DrawCall.m<\/em>:<br \/>\n[objc]<br \/>\n&#8230;<br \/>\n-(GLuint)setTexture:(GLK2Texture *)texture forSampler:(GLK2Uniform *)sampler<br \/>\n{<br \/>\n\tNSAssert( sampler != nil, @&quot;Cannot set a texture for non-existent sampler = nil&quot;);<\/p>\n<p>\tif( texture != nil )<br \/>\n\t{<br \/>\n\t\t\/** do we already have this sampler stored? *\/<br \/>\n\t\tint indexOfStoredSampler = -1;<br \/>\n\t\tint i=-1;<br \/>\n\t\tfor( GLK2Uniform* samplerInUnitSlot in self.textureUnitSlots )<br \/>\n\t\t{<br \/>\n\t\t\ti++;<\/p>\n<p>\t\t\tif( (id)samplerInUnitSlot != [NSNull null] &amp;&amp; [samplerInUnitSlot isEqual:sampler])<br \/>\n\t\t\t{<br \/>\n\t\t\t\tindexOfStoredSampler = i;<br \/>\n\t\t\t\tbreak;<br \/>\n\t\t\t}<br \/>\n\t\t}<\/p>\n<p>\t\t\/** store the texture locally *\/<br \/>\n\t\t[self.texturesFromSamplers setObject:texture forKey:sampler];<\/p>\n<p>\t\t\/** choose a textureunit slot, if not already assigned for that sampler *\/<br \/>\n\t\tif( indexOfStoredSampler &lt; 0 )<br \/>\n\t\t{<br \/>\n\t\t\ti = -1;<br \/>\n\t\t\tfor( GLK2Uniform* samplerInUnitSlot in self.textureUnitSlots )<br \/>\n\t\t\t{<br \/>\n\t\t\t\ti++;<\/p>\n<p>\t\t\t\tif( (id)samplerInUnitSlot == [NSNull null])<br \/>\n\t\t\t\t{<br \/>\n\t\t\t\t\t[self.textureUnitSlots replaceObjectAtIndex:i withObject:sampler];<br \/>\n\t\t\t\t\tindexOfStoredSampler = i;<\/p>\n<p>\t\t\t\t\t\/** Inform the embedded shader program that this slot is the new source for this sampler *\/<br \/>\n\t\t\t\t\t\/\/ save the current program, since GL requires this magicness&#8230;<br \/>\n\t\t\t\t\tGLint currentProgram;<br \/>\n\t\t\t\t\tglGetIntegerv( GL_CURRENT_PROGRAM, &amp;currentProgram);<\/p>\n<p>\t\t\t\t\tNSAssert( self.shaderProgram != nil, @&quot;Cannot set textures on a drawcall until you&#8217;ve given it a shader program (it&#8217;s possible, but not implemented here)&quot;);<br \/>\n\t\t\t\t\tGLint textureUnitOffsetOpenGLMakesThisHard = [self textureUnitOffsetForSampler:sampler];<\/p>\n<p>\t\t\t\t\t\/** Set the program, set the Uniform, then restore the program *\/<br \/>\n\t\t\t\t\tglUseProgram(self.shaderProgram.glName);<br \/>\n\t\t\t\t\tglUniform1i( sampler.glLocation, textureUnitOffsetOpenGLMakesThisHard );<br \/>\n\t\t\t\t\tglUseProgram(currentProgram);<br \/>\n\t\t\t\t\tbreak;<br \/>\n\t\t\t\t}<br \/>\n\t\t\t}<\/p>\n<p>\t\t\tNSAssert( indexOfStoredSampler &gt;= 0, @&quot;Ran out of texture-units; you cannot assign this many texture samplers to a single ShaderProgram on your hardware&quot; );<br \/>\n\t\t}<\/p>\n<p>\t\treturn indexOfStoredSampler;<br \/>\n\t}<br \/>\n\telse<br \/>\n\t{<br \/>\n\t\t[self.texturesFromSamplers removeObjectForKey:sampler];<\/p>\n<p>\t\tint i=-1;<br \/>\n\t\tfor( GLK2Uniform* samplerInUnitSlot in self.textureUnitSlots )<br \/>\n\t\t{<br \/>\n\t\t\ti++;<\/p>\n<p>\t\t\tif( (id)samplerInUnitSlot != [NSNull null] &amp;&amp; [samplerInUnitSlot isEqual:sampler])<br \/>\n\t\t\t{<br \/>\n\t\t\t\t[self.textureUnitSlots replaceObjectAtIndex:i withObject:[NSNull null]];<br \/>\n\t\t\t\tbreak;<br \/>\n\t\t\t}<br \/>\n\t\t}<\/p>\n<p>\t\treturn -1;<br \/>\n\t}<br \/>\n}<\/p>\n<p>-(GLint)textureUnitOffsetForSampler:(GLK2Uniform *)sampler<br \/>\n{<br \/>\n\tint i=-1;<br \/>\n\tfor( GLK2Uniform* samplerInUnitSlot in self.textureUnitSlots )<br \/>\n\t{<br \/>\n\t\ti++;<\/p>\n<p>\t\tif( (id)samplerInUnitSlot != [NSNull null] &amp;&amp; [samplerInUnitSlot isEqual:sampler])<br \/>\n\t\t{<br \/>\n\t\t\treturn i; \/\/ NB: sometimes you need i, sometimes you need GL_TEXTURE0 + i. OpenGL API is evil. Don&#8217;t mix them up!<br \/>\n\t\t}<br \/>\n\t}<\/p>\n<p>\treturn -1;<br \/>\n}<br \/>\n[\/objc]<\/p>\n<p>Now we can use this method to &#8220;add&#8221; a texture to our Draw call:<\/p>\n<p><em>ViewController.m<\/em>:<br \/>\n[objc]<br \/>\n-(NSMutableArray*) createAllDrawCalls<br \/>\n{<br \/>\n&#8230;<br \/>\n\tGLK2Uniform* uniformTextureSampler = [drawTexturedQuad.shaderProgram uniformNamed:@&quot;s_texture&quot;];<\/p>\n<p>\t\/**   &#8230; store the sampler and texture in the Draw call, and configure it to use them *\/<br \/>\n\t[drawTexturedQuad setTexture:texture forSampler:uniformTextureSampler];<br \/>\n&#8230;<br \/>\n}<br \/>\n[\/objc]<\/p>\n<h2>Part 3 + 4: Use the texture inside the Fragment Shader<\/h2>\n<p>First, we need a Varying (just as with Procedural Texturing).<\/p>\n<p>Unlike Procedural Texturing &#8230; the values of this Varying are special, and you have to use numbers pre-ordained by the GL spec:<\/p>\n<ol>\n<li>You will have two floating-point values, one for &#8220;X inside the texture&#8221;, one for &#8220;Y inside the texture&#8221;\n<li>X and Y are 0 at the image&#8217;s top-left corner, and 1.0 at the bottom right corner\n<li>If you render a pixel with X or Y less than 0, or greater than 1, GL gives you multiple options on how it converts those numbers into the 0..1 range. The default is to &#8220;clamp&#8221; (anything less than 0 becomes 0, anything greater than 1 becomes 1)\n<\/ol>\n<p>For the Vertex Shader, we&#8217;ll re-use our existing &#8230; <strong>except:<\/strong> we have to change the range of Varying values back to the original 0..1:<\/p>\n<p><em>VertexTextureMappingUnprojected.vsh<\/em>:<br \/>\n[c]<br \/>\nattribute vec4 position;<br \/>\nattribute vec2 a_virtualXY;<\/p>\n<p>varying mediump vec2 v_virtualXY;<\/p>\n<p>void main()<br \/>\n{<br \/>\n\tv_virtualXY = a_virtualXY;<br \/>\n\tgl_Position = position;<br \/>\n}<br \/>\n[\/c]<\/p>\n<p>The Fragment Shader needs only one line changing, we&#8217;ll make a new Shader:<\/p>\n<p><em>FragmentTextureMapOnly.fsh<\/em><br \/>\n[c]<br \/>\nuniform sampler2D s_texture; \/\/ this name MUST match the one used for &quot;uniformNamed:&quot; above<br \/>\nvarying mediump vec2 v_virtualXY; \/\/ MUST match the one in Vertex Shader<\/p>\n<p>void main()<br \/>\n{<br \/>\n\tgl_FragColor = texture2D( s_texture, v_virtualXY );<br \/>\n}<br \/>\n[\/c]<\/p>\n<p>&#8230;and use the new Shader pair in our app:<\/p>\n<p><em>ViewController.m<\/em>:<br \/>\n[objc]<br \/>\n&#8230;<br \/>\n-(NSMutableArray*) createAllDrawCalls<br \/>\n{<br \/>\n\t&#8230;this gets inserted just after we create the GLK2DrawCall itself,<br \/>\n\t&#8230;so that we have the ShaderProgram active before we try to read<br \/>\n\t&#8230;Uniforms, Samplers, etc out of it&#8230;<br \/>\n&#8230;<br \/>\n\tdrawTexturedQuad.shaderProgram = [GLK2ShaderProgram shaderProgramFromVertexFilename:@&quot;VertexTextureMappingUnprojected&quot; fragmentFilename:@&quot;FragmentTextureMapOnly&quot;];<br \/>\n\tglUseProgram( drawTexturedQuad.shaderProgram.glName );<br \/>\n&#8230;<br \/>\n}<br \/>\n[\/objc]<\/p>\n<h2>Finally: update the RenderFrame method to use Texture Mapping<\/h2>\n<p><em>ViewController.m<\/em>:<br \/>\n[objc]<br \/>\n&#8230;<br \/>\n-(void) renderSingleDrawCall:(GLK2DrawCall*) drawCall<br \/>\n{<br \/>\n&#8230;<br \/>\n\tfor( GLK2Uniform* sampler in drawCall.texturesFromSamplers )<br \/>\n\t{<br \/>\n\t\tGLK2Texture* texture = [drawCall.texturesFromSamplers objectForKey:sampler];<br \/>\n\t\tNSLog(@&quot;RENDER: Binding and enabling texture sampler &#8216;%@&#8217;, putting texture: %i into texture unit: %i&quot;, sampler.nameInSourceFile, texture.glName, GL_TEXTURE0 + [drawCall textureUnitOffsetForSampler:sampler] );<\/p>\n<p>\t\tglActiveTexture( GL_TEXTURE0 + [drawCall textureUnitOffsetForSampler:sampler] );<br \/>\n\t\tglBindTexture( GL_TEXTURE_2D, texture.glName);<br \/>\n\t}<\/p>\n<p>\t\/** Finally: kick-off the draw-call, telling GL how to interpret the data we&#8217;ve given it (triangles, lines, points &#8211; or a variation of one of those) *\/<br \/>\n\tglDrawArrays( GL_TRIANGLES, 0, drawCall.numVerticesToDraw );<br \/>\n}<br \/>\n[\/objc]<\/p>\n<p>&#8230;which is where all that boilerplate code in GLK2DrawCall saves our butts: the &#8220;management&#8221; of the texture\/texture-unit stuff is all happening there, and our Render call remains nice and simple (as it should be).<\/p>\n<h3>But, but! &#8230; my texture is upside-down!<\/h3>\n<p>iOS&#8217;s implementation of OpenGL really messes with the concept of &#8220;up&#8221;. You can Google for &#8220;OpenGL image upside down&#8221; and find many hits that explain the basic problem (OpenGL draws from y=0 at BOTTOM of screen, UIKit draws with y=0 at TOP of screen).<\/p>\n<p>&#8230;but then you discover that UIKit *sometimes* automatically does the switch for you.<\/p>\n<p>&#8230;and that CALayer (which UIKit uses internally) uses the OpenGL version of &#8220;up&#8221; (because it&#8217;s implemented on top of OpenGL, in fact).<\/p>\n<p>The very very short answer is this: change the U,V numbers you attached as vertex attributes, to &#8220;flip&#8221; your texture upside down. i.e. wherever the second value is &#8220;1.0&#8221; replace it with &#8220;-1.0&#8221;.<\/p>\n<h3>Did this post help you?<\/h3>\n<p>If you&#8217;re finding these OpenGL ES tutorials useful, enter your email, and I&#8217;ll let you know the next time I post one. I&#8217;ll also send you some info about my current personal-project, a 3D game\/app which uses these techniques:<\/p>\n<div style=\"background: #beb; border: thin gray solid; margin: 5px auto; text-align: center; width: 350px; \">\n[smlsubform emailtxt=&#8221;&#8221; showname=&#8221; mailinglist=&#8221;pif-opengles2.6&#8243;]\n<\/div>\n<h2>Source code&#8230;<\/h2>\n<p>In case you hadn&#8217;t noticed, I&#8217;ve converted this series of blog posts into <a href=\"https:\/\/github.com\/adamgit\/GL2KitExtensions\">a complete, standalone, library and hosted it on GitHub<\/a>.<\/p>\n<p>For this post, I&#8217;ve created a Branch containing <strong>the exact source of each file used here<\/strong>: <a href=\"https:\/\/github.com\/adamgit\/GL2KitExtensions\/tree\/Part6TextureMapping\">https:\/\/github.com\/adamgit\/GL2KitExtensions\/tree\/Part6TextureMapping<\/a><\/p>\n<p>(note: the &#8220;master&#8221; branch on GitHub has some code from the next few (unpublished!) blog posts; to avoid confusing yourself, use the branch above. But going forwards, you might want to play with the master branch sometime too)<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Recap: Texturing in OpenGL can be achieved in two separate ways, using different API&#8217;s and hardware features. The preferred, modern approach &#8211; procedural texturing &#8211; uses nothing more than a Fragment shader (this is really what Fragment shaders were created for: Texturing). Part 5 (Textures: 1 of 3) covered that in detail &#8211; but there&#8217;s [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[51,20],"tags":[],"_links":{"self":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/3014"}],"collection":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/comments?post=3014"}],"version-history":[{"count":14,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/3014\/revisions"}],"predecessor-version":[{"id":3056,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/3014\/revisions\/3056"}],"wp:attachment":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/media?parent=3014"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/categories?post=3014"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/tags?post=3014"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}