Weird “invalid serialized file header” Bug

I’ve recently fallen into a particular routine (this always happens right before a big convention, and PSX is right around the corner).

Basically, I wake up at around 10am. Check in with David about about updates to the game, what needs to get worked on, bugs that need to get fixed, etc. From then until 4pm, I’m basically working on admin stuff – updating devlog, writing emails, doing any writing that needs to be done (applications, game submissions, press interviews, etc), managing calendar, finances, etc, etc. I’ll often have phone calls, skype meetings, or (god forbid) real life physical meetings during this time as well.

This stuff, which I call “non-gamedev gamedev” takes up a surprisingly large chunk of my time now, and I do it closer to when I first get up, because my brain takes a while to absorb caffeine and be fully awake (yes, it takes my brain around 6 hours to wake up).

Anyway, from 6pm to 4am is my prime coding hours. There’s usually no activities scheduled, not much happening online (not in my time zone anyway), and the world is just much quieter. This is when I like to tackle the tough problems, go through code and track down bugs, etc.

Last night, around 8:30pm, I decided to stream for a bit to help me focus as I hacked away at more bugs. I went live, pulled the latest version of the build from Perforce, and then Unity just stalled at the project opening. I could see the window outline, but the project just wasn’t loading. This went on for way longer than normal. Ctrl Alt Delete to shut down Unity, but still couldn’t get the project to open.

I ended up canceling the stream, figuring that watching me open Unity is really not that interesting, plus I needed to focus on this.

I spent all night trying to get the project open. Deleting the Library and Temp folders, starting a brand new folder and pulling from Perforce. Nothing worked.

It was incredibly infuriating. It’s 4 days before I leave for PSX, I was supposed to fix a bunch of bugs and test the build on the devkit, and here I am, unable to even open up the project.

I spent until 4am trying to get this to work, with no progress. Ended up going to bed super frustrated and annoyed.

This morning, talked to David, and the problem turned out to be due to this one material (David actually had this problem before).

The material is the Default_Prototype material that comes with ProBuilder (though I don’t think it’s the issue with ProBuilder here, but a way the asset is getting serialized by Unity for version control).

serialization_bug_001

Our solution thus far is to rely on an older revision of the material:

serialization_bug_002

What is messed up though, is when we try to commit a newer revision of the material, that’s exactly like version 7, that still causes problems!

Doing a google search on “invalid serialized file header”, I found this thread: http://forum.unity3d.com/threads/some-assets-are-giving-an-invalid-serialized-file-header-error.328777/

serialization_bug_003

The issue seems to be that P4 server is modifying the end of the line for the meta file?

Given all the symptoms, a problem with version control system messing with something would seem to make the most sense here.

I’ll have to talk to Mike who runs the P4 server to make these changes, and will then post the update here.

Anyway, pretty frustrating to have lost a whole work day as a result of some weird and obscure back.

I do have things are back on track now, in that I can at least open the project and keep working.

Such is game development, I guess.

Tools Programming – Staircase Generator

There are A LOT of staircases in Manifold Garden.

In fact, some levels consist entirely of staircases:

ManifoldGarden_Stairs

It took way too long for me to finally get around to writing a staircase generator, but I’m really glad I did.

Now, ProBuilder does have a staircase generator, but the stairs in Manifold Garden have to be built in a very specific way.

Thin Stairs

0.5 is the smallest unit for the geometry in Manifold Garden. All the floors, walls, etc, are at least 0.5 units thick.

Since you can change gravity and walk on any surface in the game, every staircase needs to work on both sides. The player can walk on the top surface of the stairs, as well as the bottom surface of the stairs.

If the floor is 0.5 units thick, the stairs must be built like this:

ManifoldGarden_StairsBuildingProBuilder

Here’s how I would build it:

stairsbuilding_oldmethod

The individual steps are ProBuilder cubes, which I have to manually put in place. Once there are few, you can speed up the process by duplicating multiple steps at a time. It’s still quite slow though.

Then I have to make the “ghost steps”. These are the small invisible steps that the player actually walks on. The visible steps are actually too tall for the player to climb, since the player doesn’t actually have legs and is a just a capsule.

The ghost steps are made with ProBuilder staircase tool. However, it still takes time to figure out the size of the ghost steps, the number of steps, and also to put it in the correct position.

Anyway, for a long time, due to the 0.5 thickness requirement, I actually thought that all floors had to be 0.5 units thick and no more.

Thick Stairs

Eventually, I realized that there was actually a way to build stairs to allow for thicker floors. Thicker floors are actually quite nice because this means that the player can walk on the sides of the staircase as well, instead of just the top and the bottom.

The key was to stack the steps horizontally instead of vertically, like so:

manifoldgarden_thickstairs

As you can see in the image above, this allows for the floor to be 2 units thick while still making the staircase work from both top and bottom surfaces.

The build process is pretty much the same as that for thin steps (here I copied the ghost steps from another staircase):

stairbuilding_thick_old_method

Staircase Generator

Anyway, finally decided to write a tool to automate this process:

manifoldgarden_staircasegenerator

The tool automatically decides whether to stack horizontally or vertically depending on the step height assigned, and places the ghost steps accordingly. It also puts everything inside a parent object and names it with the width, height, and number of steps. It also assigns the correct tags and layers to the objects.

Here it is in action:

stairbuilding_new

The tool took about a weekend to make, and now a step that was rather tedious for me, and took minutes to do, is now done 3, 4 button clicks in the span of a few seconds.

 

 

E3 Prep / More Fractals

Just hit a major milestone for E3 prep. Feeling really great.

The latest build of RELATIVITY is definitely the best one by far. Great framerate, and the feel of movement has improved dramatically.

Did a stream last night making a bunch of crazy fractal levels. They turned out to be even better than I had expected.

Some of the visuals are due to a glitch with Unity’s mesh merging function. I believe there is upper limit to that and I hit it, so it created a bunch of angled pieces.

Below are some of my favorite screenshots from the session. Some of them really look like ink drawings straight out of a manga!
relativity_fractal1 relativity_fractal2 relativity_fractal3 relativity_fractal4 relativity_fractal5 relativity_fractal6 relativity_fractal7 relativity_fractal8 relativity_fractal9 relativity_fractal10

Fractal Generating with ProBuilder

Playing around with using a fractal generating script in Unity with the ProBuilder API.

The reason for using the ProBuilder API is so that I can edit the objects afterwards if necessary. I’m thinking of using the generation script primary as a foundation for creating interesting level design.

(If you’re interesting in playing with the ProBuilder API, inside the ProCore folder, there is actually an example script that shows you on to instantiate ProBuilder shapes, apply vertex color, and set the size, etc).

Also, the PS4 version of the game is coming along well. Really happy with the developer tools that we’ve added to the game. Being able to switch on NoClip has been absolutely invaluable.

Here are some of the coolest screenshots from the experiments with fractal generating.

fractal_01 fractal_02 fractal_03 fractal_04 fractal_05

 

Edge-Detection Shader Artifacts Fixed!

Back in May of last year, I wrote this devlog post about a particular issue I was having with my edge-detection shader. Specifically, it was generating these tiny black dots that appeared along the seam where two objects touched.

From what I could tell, it was due to the normals of the faces. It seemed like there was a floating point error and even though you can have two boxes lined up against one another, there was still a tiny gap that was invisible to the naked eye, but which the shader could pick up. At least this is what I determined.

My friend Devon, who was the graphics programmer on Octodad helped me tweak the shader a bit, and he did make improvements, but the artifact wasn’t completely removed.

Finally, during GDC last week, I stopped by the Unity booth on the expo floor, where ProBuilder had a kiosk, and spoke with Karl from the ProBuilder team about this issue.

Karl had two suggestions:

1) Merge the boxes together and then weld the overlapping vertices. Welding puts both vertices at the same point, so should in theory close up the seam in between the faces.

2) Delete the faces that are facing each other in between the boxes.

I decided to try these methods when I got back. Here’s a video comparing the different methods and their results:

Separate Boxes

This is the default set up of the situation. In this case, I have a wall that’s made up of a separate individual ProBuilder boxes. The boxes are vertex-colored to show that they are different. At runtime, a material is applied to all of the boxes, so that together they look like one wall.

Below you can see that there are artifacts (note that this doesn’t show the problem as well as the video) since it’s just a still. However, it shows that these artifacts exist. You can click on the images to see the larger version.

Merged

In this method, I merged all the boxes into one mesh. As you can see, the problem persists.

Merged with Welded Vertices

I actually forgot to include footage from this method in the video, which sort of defeats the point a bit…

But anyway, it looks just liked the previous two methods, without any improvements.

No Draw

In this method, I set the top and bottom face of each box to a ‘NoDraw’ material. If using ProBuilder, then during run-time, these faces get deleted.

It reduced the number of artifacts significantly when viewing the wall at a distance, but when you’re up close and looking at the wall at an angle, there are still artifacts.

Also, one thing that is very interesting is that the artifacts actually look different than the others. The others are a black X, but these artifacts seem to be a white square.

No Draw and Merged

Taking all the boxes with no draw faces on them and merging them into one mesh.

For some reason, the mesh texture just gets completely meshed up. I think ProBuilder doesn’t intend for you to apply NoDraw faces and then merge?

Clearly, this method isn’t an option.

Deleted Faces

Delete the top and bottom face of each box. As to be expected, this looks just like the case when I’m using separate boxes with NoDraw faces (since NoDraw faces get deleted at runtime).

There are much fewer artifacts when viewed at a distance, but close up, there are still a lot.

Deleted Faces and Merged

Take all the boxes with top and bottom faces deleted, and merge them into one mesh.

This actually fixes the problem! There are no artifacts! NO ARTIFACTS!!!!

Check out how clean that wall looks both up close and from a distance:

It’s so beautiful!  Tears of Joy

DevLog Update – Lightmapping solution, project management, portal rendering

Whoo! First devlog post of the new year.

Hope you all had a fun new year’s celebration. For me, New Year’s Eve was pretty great. The weather in Chicago was pretty awful that night, but I attended a college buddy’s wedding and got to ring in the new year with some old friends I hadn’t seen in a few years. It was nice to get out and socialize for a bit after working alone on the game for so long.

Anyway, some updates on Relativity:

Lightmapping Issue Resolved
As you know, I was having a lot of problems with lightmapping in the previous update, with geometry coming out looking charred for some reason. It turns out the issue had something to do with ProBuilder, specifically the UV2 maps. ProBuilder has a button that allows you to adjust UV2 generation settings.

probuilder_uv2_param_gen

Somehow, by changing the value for “Angle Error” from the default 8 to 10, it solved the problem I was having with ‘charred geometry’. I am not sure why this works, or what angle error is, as it’s not discussed in the ProBuilder documents. However, it somehow fixed my problem. My guess is that the hierarchy in which game objects were organized (with the ghost layer on top of the real layer) interfered with something behind the scenes.

Anyway, here you can see the scene with the lightmapping issue resolved:

Relativity_Game_Screenshot-2014-01-02_13-44-19

I will send an email to Gabriel at SixBySeven Studio (maker of ProBuilder) to see if he knows what’s going on here.

First Stage Redesign
I am continuing to redesign the first stage of the game. It’s more or less the first 20 – 30 minutes of the game, which is arguably the most important part, as this is where most people will get their first impression. I’m doing lots of iterations, making small improvements here and there, and it’s slowly coming together.

Here are some more work-in-progress shots:

Relativity_Game_Screenshot-2014-01-01_16-10-04

Relativity_Game_Screenshot-2013-12-31_01-05-09

editor_scene

Project Planning
I’m taking a step back from development to do some clean up and long-term project planning. The game project is starting to get pretty big and complicated enough to warrant a new approach to organizing everything.

These are priority:

  • Version Control – I’m a little embarrassed to admit I don’t have a really good version control system going (aside from zipping the project folder every other day). I don’t really have an excuse except that there was a period when I kept rewriting the game from scratch and found it a little tedious to set up github each time, so just kept putting it off. Also, I had kind of a bad experience with github at a previous job, and haven’t quite gotten over it yet. But seriously, I really need to set this up now. Seems pretty universal that this is more or less mandatory.
  • Asset Management – I’m learning that the system of organizing assets I used when making small prototypes in Unity is really not working well for a project that’s about 2 GB in size. Just as an example, I have about 15 different stair prefabs, and 20 different window prefabs. I can no longer remember the difference between “windowA_02″ and “windowB_04″. Also, I need a way to separate the latest versions of assets from the ones used for prototyping.

Portals & Non-Euclidean Geometry
I’ve been playing around a bit with portal rendering in Unity, and I think they’d make a nice addition to Relativity. From a practical standpoint, it would actually be very helpful with regards to connecting different spaces.

However, it does seem like a pretty major game mechanic to introduce, and it would be pretty time-consuming to integrate it. It would definitely add to the tripiness and crazy physics that are already there, but I’m wondering if it might be too much?

I think I will take a few days next week to experiment and get some prototypes working. If it seems promising I’ll draw up some quick levels with this and get people to playtest it, before committing to using it all over the game.

DevLog Update – Lightmapping Problems Continue

Despite everything, it seems that I still don’t fully understand lightmapping in Unity3D. For some reason, I keep getting pieces of geometry that end up looking charred. It’s as if they’re being ignored by the lightmapping process. This is what it looks like from the game view:

Relativity_Game_Screenshot-2013-12-31_10-58-48

As you can see, the platforms on the upper left and bottom right corners look really dark, as if they weren’t lightmapped.

Here’s what it looks like from the scene view, first without lightmapping applied, and then with lightmapping:

scene_no_lightmapping

scene_with_lightmapping

You can see that the same thing happened to the stairs on the left.

I honestly have no idea what’s going on here. This has happened to me several times already. Sometimes, I’ll select the problem geometry, and choose “Bake Selected”, and this will fix the problem. Sometimes, renaming objects can do the trick. However, there hasn’t been a consistent fix that has allowed me to pinpoint the problem.

In any case, here are a few tutorials on Unity lightmapping that I watched this morning while trying to find a solution. None of them helped me solve my problem, but they’re still good for getting a basic overview of lightmapping In Unity:

Lightmapping in Unity

Beast Lightmapping [UnityQuickTips]

ProBuilder: Super-Optimize Your Scene! (This one is specific to ProBuilder users)