{"id":2477,"date":"2013-06-11T20:45:19","date_gmt":"2013-06-11T19:45:19","guid":{"rendered":"http:\/\/t-machine.org\/?p=2477"},"modified":"2013-06-13T13:28:34","modified_gmt":"2013-06-13T12:28:34","slug":"unity-custom-editors-dragging-handles-in-3d","status":"publish","type":"post","link":"http:\/\/new.t-machine.org\/index.php\/2013\/06\/11\/unity-custom-editors-dragging-handles-in-3d\/","title":{"rendered":"Unity Custom Editors: dragging handles in 3D"},"content":{"rendered":"<p>Here&#8217;s two quick tips for people writing custom 3D editor controls (something you should be doing a lot of in Unity, because it&#8217;s very good at it, and it saves a lot of dev time and design time!)<br \/>\n<!--more--><\/p>\n<h2>Find the 3D point on a plane where the user&#8217;s mouse clicked, in Editor mode<\/h2>\n<p>Bizarrely, Unity doesn&#8217;t give you access to their core libraries for interpreting mouse position when you&#8217;re in Editor mode. They only exist for &#8220;play&#8221; mode. Hmm. Oh well&#8230;<\/p>\n<p>So, one of the common questions is:<\/p>\n<blockquote><p>\nFind the 3D position on an object corresponding to where the mouse is currently positioned on screen\n<\/p><\/blockquote>\n<p>Most of the source code I&#8217;ve seen on the Unity3D forums, and on the web in general, for calculating this is wrong.<\/p>\n<p>There&#8217;s a couple of problems you have to solve, not least: Unity&#8217;s mouse position is reported upside-down in the Editor. But do NOT use &#8220;Screen.height&#8221; to reverse it &#8211; Screen.height is NOT the height of your visual screen!. Instead, you want &#8220;Camera.pixelHeight&#8221; (read the Camera docs carefully, and you&#8217;ll eventually find the bit that explains that your visual screen is defined as Camera.pixelWidth x Camera.pixelHeight).<\/p>\n<p>[csharp]<br \/>\n\/** calculate the point in 3D where the mouse clicked,<br \/>\n    using the plane of the &quot;floor&quot; of your building (assuming your 3D arrow<br \/>\n    is on the floor *\/<br \/>\nRay rayCameraThroughMouse = editorCamera.ScreenPointToRay( new Vector2(e.mousePosition.x, (editorCamera.pixelHeight &#8211; e.mousePosition.y)) );<\/p>\n<p>Plane planeOfFloor = new Plane( myObject.transform.up, myObject.transform.position );<br \/>\nfloat distanceAlongRay;<br \/>\nplaneOfFloor.Raycast( rayCameraThroughMouse, out distanceAlongRay );<br \/>\nVector3 worldPointForMousePositon = rayCameraThroughMouse.GetPoint( distanceAlongRay );<br \/>\n[\/csharp]<\/p>\n<h2>Trick 2: If you want an Arrow that&#8217;s draggable in 3D, but only along it&#8217;s direction<\/h2>\n<p><span style=\"color: blue\">NB: if you don&#8217;t know how to make draggable, interactive, 3D editors in Unity, this isn&#8217;t the solution; this is only a small part of what you need to do. Most of the rest is undocumented. If I get time to write-up the missing docs myself, I&#8217;ll replace this paragraph with a link<\/span><\/p>\n<p>If you do this by hand (as I used to do), your code is something like this:<\/p>\n<p>[csharp]<br \/>\n\/**<br \/>\n * This code works great &#8211; and it works outside the Editor &#8211; but it&#8217;s long-winded<br \/>\n *\/<br \/>\n\/** 3d position of arrow start *\/<br \/>\nVector3 arrowBase = myObject.transform.position + myObject.transform.localRotation * myComponent.localOffset;<br \/>\n\/** 3d position of arrow tip *\/<br \/>\nVector3 worldCapPosition = arrowBase + myObject.transform.localRotation * myComponent.localOutDirection;<br \/>\n\/** 3d direction of arrow *\/<br \/>\nVector3 worldOutDirection = myObject.transform.localRotation * myComponent.localOutDirection;<\/p>\n<p>\/** calculate the point in 3D where the mouse clicked,<br \/>\n    using the plane of the &quot;floor&quot; of your building (assuming your 3D arrow<br \/>\n    is on the floor *\/<br \/>\nRay rayCameraThroughMouse = editorCamera.ScreenPointToRay( new Vector2(e.mousePosition.x, (editorCamera.pixelHeight &#8211; e.mousePosition.y)) );<br \/>\nPlane planeOfFloor = new Plane( myObject.transform.up, myObject.transform.position );<br \/>\nfloat distanceAlongRay;<br \/>\nplaneOfFloor.Raycast( rayCameraThroughMouse, out distanceAlongRay );<br \/>\nVector3 worldPointForMousePositon = rayCameraThroughMouse.GetPoint( distanceAlongRay );<\/p>\n<p>\/** Find the distance along the 3D line that the mouse &quot;latest&quot; position equates to,<br \/>\n    compared to its initial position *\/<br \/>\nVector3 localOffsetInWorld = worldPointForMousePositon &#8211; worldCapPosition;<br \/>\nfloat distance = Vector3.Dot( localOffsetInWorld, worldOutDirection );<\/p>\n<p>\/** add this to current handle&#8217;s position to move it on-screen *\/<br \/>\nVector3 currentHandleDragOffset = distance * worldOutDirection;<br \/>\n[\/csharp]<\/p>\n<p>But it turns out you can throw away more than half of that code, and get something much simpler, easy to read (and less likely to have bugs in it!). Unity has a built-in method for doing this that seems to work well. The official Unity docs are no help here:<\/p>\n<blockquote><p>\nHelper function for doing arrows.\n<\/p><\/blockquote>\n<p>(@Unity team: if you&#8217;re not going to write something useful, why bother writing it at all?)<\/p>\n<p>&#8230;but reading between the lines, this method was invented to solve the exact problem outlined above. Look how much shorter the code is:<\/p>\n<p>[csharp]<br \/>\n\/**<br \/>\n * This code is much cleaner\/shorter &#8230; but you can&#8217;t use this in your game,<br \/>\n * Unity won&#8217;t allow you to (it&#8217;s Editor-only). Seems a bit short-sighted?<br \/>\n *\/<br \/>\n\/** 3d position of arrow start *\/<br \/>\nVector3 arrowBase = myObject.transform.position + myObject.transform.localRotation * myComponent.localOffset;<br \/>\n\/** 3d position of arrow tip *\/<br \/>\nVector3 worldCapPosition = arrowBase + myObject.transform.localRotation * myComponent.localOutDirection;<br \/>\n\/** 3d direction of arrow *\/<br \/>\nVector3 worldOutDirection = myObject.transform.localRotation * myComponent.localOutDirection;<\/p>\n<p>\/** Let Unity&#8217;s one-liner do all the hard work for you *\/<br \/>\nfloat distance = HandleUtility.CalcLineTranslation( currentDragStartPoint, e.mousePosition, arrowBase, worldOutDirection );<\/p>\n<p>\/** add this to current handle&#8217;s position to move it on-screen *\/<br \/>\nVector3 currentHandleDragOffset = distance * worldOutDirection;<br \/>\n[\/csharp]<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Here&#8217;s two quick tips for people writing custom 3D editor controls (something you should be doing a lot of in Unity, because it&#8217;s very good at it, and it saves a lot of dev time and design time!)<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[67,69],"tags":[],"_links":{"self":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/2477"}],"collection":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/comments?post=2477"}],"version-history":[{"count":3,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/2477\/revisions"}],"predecessor-version":[{"id":2481,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/posts\/2477\/revisions\/2481"}],"wp:attachment":[{"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/media?parent=2477"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/categories?post=2477"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/new.t-machine.org\/index.php\/wp-json\/wp\/v2\/tags?post=2477"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}