{"id":3781,"date":"2016-01-19T05:08:29","date_gmt":"2016-01-19T04:08:29","guid":{"rendered":"http:\/\/t-machine.org\/?p=3781"},"modified":"2016-01-19T05:08:29","modified_gmt":"2016-01-19T04:08:29","slug":"making-a-night-vision-shader-for-animals-in-unity3d","status":"publish","type":"post","link":"http:\/\/new.t-machine.org\/index.php\/2016\/01\/19\/making-a-night-vision-shader-for-animals-in-unity3d\/","title":{"rendered":"Making a night-vision shader for animals in #unity3d"},"content":{"rendered":"<p>In the FPS roguelike I&#8217;ve been working on, a core feature is that each ability modifies how you interact with the world &#8211; you see things differently, you move differently, etc.<\/p>\n<h2>Night Vision<\/h2>\n<p>I&#8217;m working on a tiny level demo to show some of my ideas together. I want one character that can see in the dark, something like the &#8220;InfraVision&#8221; of D&amp;D.<\/p>\n<p>Spec:<\/p>\n<ul>\n<li>Character can see independent of the existence of lights in scene\n<li>Vision is imperfect, significantly weaker than daylight vision, in terms of details, colours, etc\n<li>Vision enables easy location + identification of other creatures\n<\/ul>\n<h2>Screenshots<\/h2>\n<p>Here&#8217;s my orc (a test avatar I already had) sitting on top of a building:<\/p>\n<p><a href=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.56.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.56-1024x894.png\" alt=\"Screen Shot 2016-01-19 at 03.23.56\" width=\"625\" height=\"546\" class=\"aligncenter size-large wp-image-3782\" srcset=\"https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.56-1024x894.png 1024w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.56-150x131.png 150w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.56-300x262.png 300w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.56-624x545.png 624w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.56.png 1107w\" sizes=\"(max-width: 625px) 100vw, 625px\" \/><\/a><\/p>\n<h3>Step 1: cancel the lighting<\/h3>\n<p>We use Unity&#8217;s &#8220;GetComponent<Camera>.SetReplacementShader( Shader, &#8220;&#8221; )&#8221; to change the shader to a simple &#8220;Unlit&#8221; shader (Unity provides a built-in one).<\/p>\n<p>[csharp]<br \/>\n\t[ContextMenu(&quot;Enable dark vision&quot;)]<br \/>\n\tpublic Shader shaderDarkVision;<\/p>\n<p>\tpublic void Start()<br \/>\n\t{<br \/>\n\t\tif( shaderDarkVision == null )<br \/>\n\t\t\tshaderDarkVision = Shader.Find(&quot;Unlit\/Texture&quot;);<br \/>\n\t}<br \/>\n\tpublic void EnableDarkVision()<br \/>\n\t{<br \/>\n\t\tGetComponent&lt;Camera&gt;().SetReplacementShader( shaderDarkVision, &quot;&quot; );<br \/>\n\t}<br \/>\n[\/csharp]<\/p>\n<p><a href=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.45.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.45-1024x904.png\" alt=\"Screen Shot 2016-01-19 at 03.23.45\" width=\"625\" height=\"552\" class=\"aligncenter size-large wp-image-3783\" srcset=\"https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.45-1024x904.png 1024w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.45-150x132.png 150w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.45-300x265.png 300w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.45-624x551.png 624w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.23.45.png 1107w\" sizes=\"(max-width: 625px) 100vw, 625px\" \/><\/a><\/p>\n<h3>Step2: Add a vignette to nerf it<\/h3>\n<p>I want the DarkVision to be significantly less perfect than daylight vision, so let&#8217;s add a simple vignette (Unity has downloadable examples of this shader in their docs, but I only found it after I&#8217;d got mine working!).<\/p>\n<p>This took me a long time to get working, just to get accurate screen-positions of shader pixels &#8211; so much of Unity&#8217;s semi-proprietary Shader language is still undocumented, after many years. I knew all the maths, and had implemented the effect in OpenGL \/ GLSL in minutes, but it took an hour or so in Unity. And once it was working, I got sidetracked in tweaking the exact curve of darkness of vignette.<\/p>\n<p><a href=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.24.22.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.24.22-1024x884.png\" alt=\"Screen Shot 2016-01-19 at 03.24.22\" width=\"625\" height=\"540\" class=\"aligncenter size-large wp-image-3784\" srcset=\"https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.24.22-1024x884.png 1024w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.24.22-150x130.png 150w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.24.22-300x259.png 300w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.24.22-624x539.png 624w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.24.22.png 1105w\" sizes=\"(max-width: 625px) 100vw, 625px\" \/><\/a><\/p>\n<h3>Step3: Convert to grayscale (infravision should be colourblind)<\/h3>\n<p>This was easy &#8211; simply average the r\/g\/b components of your Color, somethign like:<\/p>\n<p>[c]<br \/>\navg = (color.r + color.g + color.b ) \/ 3f;<br \/>\ncolor = float4( avg, avg, avg, color.a );<br \/>\n[\/c]<\/p>\n<p><a href=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.21.43.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.21.43-1024x852.png\" alt=\"Screen Shot 2016-01-19 at 03.21.43\" width=\"625\" height=\"520\" class=\"aligncenter size-large wp-image-3785\" srcset=\"https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.21.43-1024x852.png 1024w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.21.43-150x125.png 150w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.21.43-300x250.png 300w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.21.43-624x519.png 624w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.21.43.png 1103w\" sizes=\"(max-width: 625px) 100vw, 625px\" \/><\/a><\/p>\n<h3>Step 4: several hours of experimenting with wild effects<\/h3>\n<p>I had some problems with this:<\/p>\n<ol>\n<li>It was too easy to see &#8220;everything&#8221;\n<li>With no lighting-calcs, it was fugly\n<li>With no lighting-calcs, it was VERY hard to do depth-perception. In a complex street, you got lost trying to tell which objects were near, far, blocking your way, or not.\n<li>WAY too much detail on the textures is showing through: DarkVision should lose most detail, while keeping overall shapes clear\n<\/ol>\n<p>Here&#8217;s some of the things I tried and discarded. Note how the screenshots are from many positions &#8211; I ran around a lot, testing how effective the different shaders were for hunting a quarry (or fleeing an angry mob of AI&#8217;s).<\/p>\n<table>\n<tr>\n<td><a href=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.57.08.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.57.08-300x250.png\" alt=\"Screen Shot 2016-01-19 at 02.57.08\" width=\"300\" height=\"250\" class=\"aligncenter size-medium wp-image-3786\" srcset=\"https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.57.08-300x250.png 300w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.57.08-150x125.png 150w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.57.08-1024x855.png 1024w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.57.08-624x521.png 624w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.57.08.png 1109w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/td>\n<td><a href=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-01.34.27.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-01.34.27-300x250.png\" alt=\"Screen Shot 2016-01-19 at 01.34.27\" width=\"300\" height=\"250\" class=\"aligncenter size-medium wp-image-3787\" srcset=\"https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-01.34.27-300x250.png 300w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-01.34.27-150x125.png 150w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-01.34.27-1024x852.png 1024w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-01.34.27-624x519.png 624w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-01.34.27.png 1096w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/td>\n<td><a href=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.26.02.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.26.02-300x276.png\" alt=\"Screen Shot 2016-01-19 at 02.26.02\" width=\"300\" height=\"276\" class=\"aligncenter size-medium wp-image-3788\" srcset=\"https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.26.02-300x276.png 300w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.26.02-150x138.png 150w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.26.02-624x575.png 624w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-02.26.02.png 1015w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/td>\n<\/table>\n<h3>Step5: FAILED: use depth-buffer shading<\/h3>\n<p>Unity staff: please write clear, correct, up-to-date documentation on depth-buffering <em>as supported (or not) by your proprietary Shader Lab language<\/em>, and with clear details on how this works in Unity5, not some hand-waving Unity4-maybe-Unity5-not-sure gumph.<\/p>\n<p>I simply couldn&#8217;t get it to work. Using the proprietary built-in depth-buffer samplers either failed outright (0.5 across the sampler), or gave bizarre results.<\/p>\n<p>Rumours suggest that we have to write 3 shaders: 2 for your proprietary system, and a third where we re-implement depth-buffer access by hand. This seems to me unlikely &#8211; I am sure Unity can handle that for you. This is the kind of feature that Unity&#8217;s ShaderLab does well (but no-one documents, and so it goes under-used by devs).<\/p>\n<p>When you find or reverse-engineer instructions, ShaderLab is excellent. It&#8217;s such a pity that so often &#8230; you can&#8217;t.<\/p>\n<h3>Step6: Combine object-normals with plain greyscale<\/h3>\n<p>Ultimately, I fell back on &#8220;Fine, I&#8217;ll use the most basic bits of Shaders that even Unity can&#8217;t make confusing&#8221; &#8211; object normals. I faked shading Old Skool: each surface is darkened by dot-producting the forwards vector ( which by definition is (0,0,1) after the MVP multiplication stage) with the normal vector. That gives a float from 0 to 1, showing how much the surface is facing towards\/away from the viewer.<\/p>\n<p><a href=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.26.07.png\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.26.07-1024x951.png\" alt=\"Screen Shot 2016-01-19 at 03.26.07\" width=\"625\" height=\"580\" class=\"aligncenter size-large wp-image-3789\" srcset=\"https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.26.07-1024x951.png 1024w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.26.07-150x139.png 150w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.26.07-300x279.png 300w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.26.07-624x579.png 624w, https:\/\/t-machine.org\/wp-content\/uploads\/Screen-Shot-2016-01-19-at-03.26.07.png 1090w\" sizes=\"(max-width: 625px) 100vw, 625px\" \/><\/a><\/p>\n<p>I&#8217;m not delighted with this &#8211; it&#8217;s not quite the effect I was aiming for (I REALLY wanted to wipe more of the diffuse detail, but&#8230;). However, it does work quite nicely, and for now it&#8217;s more than good enough.<\/p>\n<h2>Benefits of final version<\/h2>\n<p>Here&#8217;s why I like it:<\/p>\n<ol>\n<li>Runs 100% independently of my main lighting &#8211; I never have to re-write it when changing lighting.\n<li>Provides huge advantages at night time, or in low-light situations. More so the less lighting there is\n<li>Provides depth-cues to the player, so that running quickly through streets (or tunnels) is still pretty easy to do\n<li>Limits the over-powered nature by cutting down peripheral vision\n<li>Runs very fast on GPU, despite crappy hand-written shader code (mostly because it doesn&#8217;t use any lighting calculations)\n<\/ol>\n","protected":false},"excerpt":{"rendered":"<p>In the FPS roguelike I&#8217;ve been working on, a core feature is that each ability modifies how you interact with the world &#8211; you see things differently, you move differently, etc. Night Vision I&#8217;m working on a tiny level demo to show some of my ideas together. I want one character that can see in [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"_links":{"self":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/3781"}],"collection":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/comments?post=3781"}],"version-history":[{"count":1,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/3781\/revisions"}],"predecessor-version":[{"id":3790,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/3781\/revisions\/3790"}],"wp:attachment":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/media?parent=3781"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/categories?post=3781"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/tags?post=3781"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}