Edge Detection Shader Deep Dive! Part 1 – Even or Thinner Edges?

It has been a while since the last update. I think this may actually be the longest gap between updates on this devlog in quite some time….

I spent the last week and a half deep diving into shaders, writing multiple edge detection algorithms, connecting them to MIDI controllers for better tweaking, so my mind was down that rabbit hole. I feel like I’ve finally emerged and am seeing sunglight for the first time in a while.

Anyway, I’m going to talk about what I did during this past week, and the different approaches I tried. What worked, and what didn’t.

Basic Shader Knowledge

Edge detection in Unity relies on using the depthNormal texture from the camera.

I’ve written about this before, so recommend checking it out just to get a basic understanding:

Unity Shaders – Depth and Normal Textures – Part 1

Even / Thinner Edges

The edge detection shader I had been using for most of the past year, relied on this algorithm:

- Sample surrounding pixels
- Compare the normal values of opposite surrounding pixels (if values are close, it’s not an edge, else, it is)
- Compare the depth values of opposite surrounding pixels (if values are close, it’s not an edge, else it is)

So basically we test for both normals and depth. The pixel only has to pass one of the tests to be considered an edge. Some edges the depth test miss, some the normal test miss.

This is what it looks like:


I describe this version as using uneven lines, as you can see that some of the lines are slightly thicker than the others. These are always at the place where there is both normal and depth difference, so it’ just that the edge there is being drawn by both the normal and depth test.

I decided to try another approach, inspired by Lucas Pope’s approach for the edge detection shader in Return of The Obra Dinn.

This time, here’s the algorithm:

- sample surrounding pixels
- combine depth and normal values to form a color
- compare the values of the new combined color in surrounding pixels (if values are close, it’s not an edge, else it is)

This is what it looks like:


The lines are definitely much more even (since we’re only doing one test now), but also a lot thinner.

I actually am not such a big fan of the thin lines. Some people said it looked cleaner, but I felt it looked too faded, and didn’t have a strong visual impact.

My last approach was combining the two approaches, using the first approach for lines that are closer (so that they are thicker), and using the second approach for lines farther away.

This basically just involved having both of the above algorithms together in one shader, and I would set a depth split threshold. If the depth values were below this (closer to camera), I’d use first approach, if depth values were above the depth split threshold (farther away), then we use second approach.

This is what it looks like:3

There are several problems with this. Having thick lines closer and thin lines farther doesn’t really help as much with depth perception as using color (lighter lines farther away, darker lines closer).

The width of the lines is due to sample distance, which is actually discrete, so that means the line width change is not continuous.

The farther lines are very faded, and I actually prefer how the far lines look in the uneven line version, as you can see better what’s in the distance.

Finally, and most importantly, this version looks awful in motion. You can see exactly where the algorithm change happens as the line width change is quite sudden. So it just looks like this bit of darkness that’s moving in front of you when you move.

So this whole exploration took about a week, as I was also writing a custom inspector to sort out variables (so that the inspector only showed the necessary variables for each edge detection method), and also connecting it to the MIDI controller for tweaking (which was a real pain to set up with all three methods as I had to write in ways to change the setup of the MIDI controller as well).

In the end, it turned out the method I had before was the best one. This was a bit of a disappointment, but such is game development.

Anyway, while doing all this shader stuff, I thought I might as well try to tackle the edge detection shader artifact issue that had been bugging me for the past 2 years.

Continuing in Part 2.


Door Segment Animation

Decided to take a look at the door segment scaling issue. The way I hide away the door segments into the frame of the door is by scaling it in the direction of movement.

Here’s what it looks like slowed down:


Here it is without scaling:


I could just leave it this way without scaling and simply hide the door segments in the surround walls, but the problem is sometimes the surrounding walls are orthogonal to the door, like this:


Not to mention the floor…

Anyway, I looked into seeing if I could resolve this by using a shader that makes the parts of the door that protrude transparent.

However, this ended up being much more complicated because of the edge detection shader.

If the object is rendered in the transparent queue, it’s rendered after post-processing, but this means you can see the edges of other objects through parts of the object that are opaque:


In the above image, it’s supposed to be stripes of transparency – the white parts should be opaque and you shouldn’t be able to see black edges through it.

But if I render the object not as part of the transparent queue, but before post-processing, the outline of the object itself gets drawn. Notice that an edge is drawn along the part that is supposed to be transparent:

Obviously, I don’t want to see the outline of the door segment that’s protruding.

Another approach that I thought would work would be the replace the door segment with a single box with a texture applied on top, and then offset the texture based on how much scaling was done.

However, that had the same issue with the edges. The door segment with the texture (float in front) looks very different than the actual door:


Some people on twitter suggested simply modeling and animating the door in a 3D software, and have parts that nest into each other as the door goes into the frame. This would work.

However, I thought about it some more, and I don’t think it’s really an issue I need to worry about. The scaling seemed very obvious in the gifs I posted mostly because time was slowed down quite a bit. In the game, the door opening happens in 0.3 seconds. It’s literally a fraction of a second, and given that players are usually standing some distance away when the door opens (because they’re pressing a button or putting a box on a switch), the scaling effect would be hardly noticeable.

You can see the door in action in the gif below, in which I demonstrate cubefruit regrow and destruction (that happens behind the camera), working with the traveling light path and the door switch in a perfect loop:


The door opening is so quick I don’t think it’s anything to worry about at least for now. If it really proves to be a problem later on, I’ll just get a 3D artist to animate it with the suggestion above.




Fruit Growing on Tree Basics

Got the basics of fruit growing on tree working.


In the above gif, the animation looks pretty dead. It’s just a linear growth rate.

I then switched to using an animation curve to control the growth rate, and made it so the box actually is larger than normal at a certain point to add some juiciness to the animation.

Found this really funny glitch while playing around:


Anyway, you can pick the box off of the tree and then use that box to open a door:


It was a really cool moment when I finally saw this. I’ve had this idea in my head for so long…. Since Tokyo Game Show in October I believe. To finally see it in action felt very weird, but incredibly satisfying.
Like I knew exactly what to expect since I wrote it, but to actually see something with my eyes that I had been picturing in my head for so long… I couldn’t stop laughing.





Traveling Light on Switch and Doors!

Got the light on the line to work together with the switch:


I think it works better for the entire line to go off when the box is removed, instead of having the light slowly retreat, as that would look pretty weird just to have the light on but not the box there. Also feels more snappy that way.

Also worked on door animation: door-opening

I think I’d like to draw up some more designs of doors. I like the way this opens (feels very equal on all sides, which is necessary for the mechanic), but I wonder if the actual design can be improved.

Calling the door animation from script was a bit more tricky as Unity 5 has changed the API for animation, and it’s quite confusing coming from Unity 4.

Basically, it seems like one has to use animator now instead of animation component, and it’s animation states instead of animation clips.

Here’s how the script looked to get the call the animation from within a script:

using UnityEngine;
using System.Collections;public class DoorController : MonoBehaviour {
void Start () {


// Update is called once per frame
void Update () {

if (Input.GetKeyDown(KeyCode.T)) {


if (Input.GetKeyDown(KeyCode.Y)) {



Once that was done, it was just a matter of connecting the door to the switch: switch-door-light-almost

It feels quite satisfying, but I’m going to make it so that the door doesn’t open until the light travels across the path and hits it. Right now the door opens right when the box gets placed on the square, and I think it feels a little bit off.