Leaf Shader

Leaf Shader

I spent all day yesterday trying to debug this annoying problem with the shader for the leaves in the game. It was one of those debug sessions where you try all these crazy complicated solutions and go down many rabbit holes, and after many many fruitless hours, just when you’re about to give up, you find that the solution by commenting out two lines of code.

Yes, yesterday I fought the shader dragon, and it turned out to be a dragonfly. But I won (!), so I guess it’s all good.

Anyway, gather around my friends, and I will recount the tale of my majestic (mis)adventure!

The Problem

Here’s what the leaf in the game looks like, when Unity is in the PC platform.

Notice how in certain angles, you can see the back face of the block, which makes for a slightly darker alpha area. I really like this effect, because it reminds of when you’re doing watercolor, and you color over an area that’s already been colored. It also adds a nice variation to the color, preventing it from looking too uniform, and also adds a nice sense of 3D.

Also, what’s great is that it responds to changes in lighting:

However, when getting it to run on PS4, I end up with the nightmarish pink that’s all too familiar to anyone who has done graphics programming:

Approach No. 1

My initial thinking was that it was because this shader is a surface shader, and I can’t used fixed-function pipeline shader, so I decided to try to convert it to a vertex fragment shader.

For surface shaders in Unity, if you add the line

Then in the inspector, when you hit “show generated code”, it’ll show you the vertex fragment version of the surface shader:

The script that was generated though, was over 1,000 lines long and I really couldn’t figure out the logic of what was going wrong.

Approach No. 2

I decided to try to just write a vertex fragment shader from scratch that had the same effect.

Here’s the progress of it:

I was able to get both front and back faces rendered transparently. In the second picture above, you can see that for the red box on the left, you can see the grids on both the front and back faces.

However, I wasn’t able to get different alpha values on the different faces (ie. using the normals), and also couldn’t get the coloring to respond to lighting.

I soon realized the reason why it’s so great to work with surface shaders is because it takes care of all the lighting issues that you’d have to write yourself in a vertex fragment shader, which is very complicated.

Just FYI, I was using this shader on multiple lighting and this one on translucent bodies as references from the Cg programming wikibooks. They’re both great examples, but I wasn’t able get the effect I wanted.

Approach No. 3

I gave up on writing a vertex fragment shader and decided to try to use Unity 5’s new standard shader model.

The problem is that it seems that the rendering order has changed. Now, it renders transparency before post-processing, so you can see the line is drawn on top of the transparent object:

Whereas in the previous shader model (still available in Unity 5 as legacy shaders), render transparency after post-processing:

In any case, I couldn’t figure out a way to directly edit the standard model shader script, so wasn’t able to add back face rendering.

Approach No. 4

It was around 4 AM in the morning, and I was pretty much ready to give up. I thought I’d just go with the legacy shader transparent material. It didn’t have the nice back face rendering that the original shader had, but it was close enough.

As a last resort, I decided to comment out the second pass of the original shader, and all of a sudden it worked. So it seemed like the problem was just due to the second pass in the script. I then started to do a trial and error approach, commenting out different sections of the code until eventually, I commented out this:

And boom! Lo and behold, it worked! This is a screenshot taken from the PS4 devkit:

Check out that sweet back face rendering and layered alpha!