Edge Detection Shader Deep Dive! Part 1 – Even or Thinner Edges?

It has been a while since the last update. I think this may actually be the longest gap between updates on this devlog in quite some time….

I spent the last week and a half deep diving into shaders, writing multiple edge detection algorithms, connecting them to MIDI controllers for better tweaking, so my mind was down that rabbit hole. I feel like I’ve finally emerged and am seeing sunglight for the first time in a while.

Anyway, I’m going to talk about what I did during this past week, and the different approaches I tried. What worked, and what didn’t.

Basic Shader Knowledge

Edge detection in Unity relies on using the depthNormal texture from the camera.

I’ve written about this before, so recommend checking it out just to get a basic understanding:

Unity Shaders – Depth and Normal Textures – Part 1

Even / Thinner Edges

The edge detection shader I had been using for most of the past year, relied on this algorithm:

- Sample surrounding pixels
- Compare the normal values of opposite surrounding pixels (if values are close, it’s not an edge, else, it is)
- Compare the depth values of opposite surrounding pixels (if values are close, it’s not an edge, else it is)

So basically we test for both normals and depth. The pixel only has to pass one of the tests to be considered an edge. Some edges the depth test miss, some the normal test miss.

This is what it looks like:

1

I describe this version as using uneven lines, as you can see that some of the lines are slightly thicker than the others. These are always at the place where there is both normal and depth difference, so it’ just that the edge there is being drawn by both the normal and depth test.

I decided to try another approach, inspired by Lucas Pope’s approach for the edge detection shader in Return of The Obra Dinn.

This time, here’s the algorithm:

- sample surrounding pixels
- combine depth and normal values to form a color
- compare the values of the new combined color in surrounding pixels (if values are close, it’s not an edge, else it is)

This is what it looks like:

2

The lines are definitely much more even (since we’re only doing one test now), but also a lot thinner.

I actually am not such a big fan of the thin lines. Some people said it looked cleaner, but I felt it looked too faded, and didn’t have a strong visual impact.

My last approach was combining the two approaches, using the first approach for lines that are closer (so that they are thicker), and using the second approach for lines farther away.

This basically just involved having both of the above algorithms together in one shader, and I would set a depth split threshold. If the depth values were below this (closer to camera), I’d use first approach, if depth values were above the depth split threshold (farther away), then we use second approach.

This is what it looks like:3

There are several problems with this. Having thick lines closer and thin lines farther doesn’t really help as much with depth perception as using color (lighter lines farther away, darker lines closer).

The width of the lines is due to sample distance, which is actually discrete, so that means the line width change is not continuous.

The farther lines are very faded, and I actually prefer how the far lines look in the uneven line version, as you can see better what’s in the distance.

Finally, and most importantly, this version looks awful in motion. You can see exactly where the algorithm change happens as the line width change is quite sudden. So it just looks like this bit of darkness that’s moving in front of you when you move.

So this whole exploration took about a week, as I was also writing a custom inspector to sort out variables (so that the inspector only showed the necessary variables for each edge detection method), and also connecting it to the MIDI controller for tweaking (which was a real pain to set up with all three methods as I had to write in ways to change the setup of the MIDI controller as well).

In the end, it turned out the method I had before was the best one. This was a bit of a disappointment, but such is game development.

Anyway, while doing all this shader stuff, I thought I might as well try to tackle the edge detection shader artifact issue that had been bugging me for the past 2 years.

Continuing in Part 2.

 

Interlockable Design

The last few days have been spent trying to work out the design of the interlockables.

It was decided early on to go with the gradient over the stripe patterns. I myself much preferred gradient, and a quick poll on twitter showed that most people agreed:

I posted a few more iterations on twitter to figure out what was essential to the design, and got a lot of really good feedback. I thought I’d post everything here to show you the process of arriving at the design, and also to talk about the constraints that are involved.

How do the interlockables behave?

The interlockables are giant polynominos that cannot be picked up, but which respond to the player’s gravity.

Interlockables are always bi-directional, and respond to gravity on either ends of an axis. So they are red-blue, yellow-green, or purple-orange.

The Interlockables cannot be picked up.

The Interlockables sometimes have lines on them, which can connect to other lines and carry power (from a switch) to an object that can be powered on (like a door).

An interlockable can have more than one line. The lines don’t necessarily have to follow the entire shape of the interlockable ie the line can stop halfway up the side of the interlockable.

Version 1.0

This is what the interlockable originally looked like

As you can see, there are lots of problems with this.

The biggest complaint I got from playtesters was that you couldn’t tell which direction an interlockable was going to fall in when it was inactive.

Usually, it was along the axis of the longest piece, but that wasn’t always the case.

Version 2.0

First major revision of the interlockable. Basically using the double gravity cube texture and making it really big.

the problem is that it’s super ugly, and also the stripes don’t help communicate direction. As in, only the the very top and very bottom stripe show direction. The middle stripes don’t really do much.

Plus, it’s hard to see the line on the interlockable.

Version 2.1

Added white center stripe so that the line shows up better. Using gradient to signal direction.

Not a big fan of the diamond in the corners and after asking on twitter, a lot of people agreed.

Plus, with diamonds, it’s a bit hard to tell direction. You could be looking at the diamond from an angle, and it isn’t always clear which direction is the longer one.

eddriofer suggested this:

The problem here is that the center stripe doesn’t always align with the shape of the interlockable. So sometimes the center stripe would have to be orthogonal to the shapes of the diamonds then.

Version 2.2

Moving the diamonds to the gradient. Still has same problem of diamonds not being super clear with indicating direction.

Also, as main_gi pointed out, it looks too much like a playing card:

Version 2.3

Gavin McCarthy suggested using chevrons, which I thought was a good idea.

A rough draft:

Cleaned up the chevrons, but realized I didn’t know what to do w/ the chevrons on the bottom of the LL

Juliano had this suggestion:

However, it made me realize that having chevrons right next to each other made it hard to tell direction, because where they were touching could be interpreted as where the chevron was pointing.

Mark’s suggestion made this pretty clear.

Version 2.4

Here I didn’t like the way the center stripe was blocking the bottom chevrons.

Alan suggested this:

This actually looks really good. However, the chevrons here are part of the center stripe, and the center stripe isn’t always present or pointing in direction of interlockable gravity, so this doesn’t work functionally.

Version 2.5

This is what I’m going to go with for now. It’s the most consistent so far. There is some spacing issues that can be tweaked (alignment, etc), but I think the general idea is there.

One detail that Daniel pointed out is that the chevrons should be compressed a bit vertically so that spacing between the top and bottom is equal to spacing on sides.

Anyway, that’s where the interlockable designs are at now.

There’ll definitely be more art passes eventually. Right now, it’s enough to start playtesting and see if it’s getting the right information across to players.