MIDI Controller for Tweaking Variables

IMG_9205Background

For the past 2 months, I’ve been focusing on game feel in RELATIVITY, trying to get the motion of falling, carrying boxes, and switching walls to feel good. To this end, I’ve implemented a Quake-style debug console to help me monitor variables and to change them from within the game, which is incredibly useful when I’m playtesting the game on my laptop.

Right before GDC, my friend Thew mentioned in a twitter conversation that he uses a MIDI controller for tweaking variables.

thew twitter conversation

This had actually never occurred to me before, and it was kind of mind blowing how brilliant of an idea it is. No more fiddling with values in the inspector during gameplay (which is awkward for a FPS game because it requires the use of the mouse, which is synced to the player head movement). And even with a debug console, I had to set the values explicitly, and with knobs and sliders, it gives much smoother control over the values.

MIDI Controller

The MIDI Controller I bought is the KORG nanoKontrol2. It sells on Amazon right now for about $40. When I bought it, it was $30, but there was no free shipping (it was $10), so I guess they’ve just absorbed the shipping cost into the retail price.

Korg_NanoKontrol2

Once you get the controller, you’ll also want to download the KORG KONTROL Editor Software, which is available for free on the Korg website.

The software allows you to change various settings such as the channel numbers for the different buttons, sliders, and knobs.

Below is a view of the Kontrol Editor, and what the default channel numbers are:

Korg editor kontrol

Setting it up with Unity

I’m using Unity 5 on Windows 7.

I found this wonderful plugin by Keijiro which lets Unity communicate with external MIDI Controllers.

I add the folders in the test branch to my Unity project and was able to get the demos running.

The demo scene ‘CC’ lets you move the different knobs and sliders to control rectangles on the screen.

Nothing happened with me for ‘Notes’. I think my controller didn’t have the right buttons/keys?

Anyway, figuring out how to get the value for the buttons was enough for me, and CC allowed me to do that.

One thing, though, I do get this error in my console:

console_error

Everything still runs, so I’m not really sure what the problem is.

I believe it has to do with these two files both being named ‘UnityMidiReceiver’ in the plugins folder:

unitymidireceiver

However, if I try to delete one of them, or change their settings or name, I get way more errors as a result.

If anyone knows what the issue here is, please let me know.

Code

The plugin is pretty sweet, and also gives you smoothing functions for all the dials.

To get the value for a knob or slider, you use a line like this:

1
value = MidiInput.GetKnob(0, filter);

The first parameter is the channel number, and ‘filter’ is the input filter, which can be ‘RealTime’, ‘Fast’, or ‘Slow’. RealTime means that the value of the slider moves with the slider itself, while fast and slow have smoothing functions, so the value trails behind the actual slider a bit.

Value Mapping Problem

The value for each channel goes from 0 to 1. For the sliders and knobs, its continuous. For the buttons, it’s binary.

When I first set up the controller, this was a bit of a problem, because the knobs and sliders weren’t always in the exact position that you wanted your values to start at.

Say for example you have a value for ‘Resistance’ that you want to tweak. In the inspector, you have ‘Resistance’ set to 0.5, but on the controller, the slider could be in any position, and not exactly halfway through.

As soon as you touch the slider, then the value gets set off. Likewise, if you set all the knobs and sliders to 0 when you start, then all your variables end up being 0, which is also annoying. This means you have to start from scratch everytime you tweak the values, instead of getting it close in the inspector, and then using the controller to finetune.

Another issue is that then your values are limited to 0 and 1, which is not always the case with variables. You can remap the values, but if you make it go from -10 to 10, the slider’s physical distance is still the same, which then makes it less precise.

The solution was suggested by Ezra, which is to make the knobs and sliders infinite:

ezra conversation

I made it so that if you’re holding down the ‘Stop’ button, then none of the changes to the sliders or knobs get added to the actual value of the variable. So if the slider has reached the top and you want to keep increasing the value, hold the ‘Stop’ button, bring the slider down, then continue from there.

Below is some example code where I’m reading value from one slider:

1
2
3
4
5
6
7
8
9
10
11
12
stopButton = MidiInput.GetKnob(42, filter); // this is the stop button

sliderInput = MidiInput.GetKnob(0, filter);

//determine difference w/ previous value to decide how much to add to or subtract from the actual value
float difference = sliderInput - previousSliderInput;

if (inputStopButton == 0) { // if stop button is not pressed
value += difference;
}

previousSliderInput = sliderInput;

In the above code, ‘value’ is the actual value of the variable we’re changing.

Results

Anyway, that’s the basics of getting the MIDI controller set up. Once that’s done, you can use it to pretty much tweak any variable that’s a float, from physics values to light intensity.

So far, I’ve just connected the sliders and knobs to variables in different shaders I’m using. What’s great is that you can also just fool around by randomly moving sliders and turning knobs to see what happens. Here are some results of the experiment:

I hope this is helpful! I’m really looking forward to tweaking values in the game and fine tuning everything from acceleration speed to the vignette blur distance.

Relativity_Game_Screenshot-2015-03-14_22-08-42 Relativity_Game_Screenshot-2015-03-14_22-13-13 Relativity_Game_Screenshot-2015-03-14_22-24-42

Final Tips

During GDC, I met Matt Boch, who is a creative director at Harmonix, and has worked on games like Fantasia, as well as the hardware for Dance Central and Rock Band. I told about getting a MIDI controller to tweak variables and he had a few tips:

1. Often times, we choose values that seem to ‘look good’. For example, when I look at a lot of the values I had in my code, they were often ‘clean’ numbers like 4, 3.5, or 80. In reality, what actually feels good may be something like 7.828312. Using a controller to tweak can really help you get the right numbers based on feel.

2. I told Matt about how because I worked with placeholder game feel for a long time and got used to it, it’s really hard for me to get a sense of what feels good now. He suggested setting the values to something that I know for sure is bad for a week, to “shock” myself out of the game feel, and then go back and tweak the values to get it right.

Development Update – Infinite Library

Started to cut together a video as part of my submission for the Experimental Gameplay Workshop at GDC. I submitted last year, but didn’t get in. However, the game has improved quite a bit, so I think my chances of getting in are much better this year.

Keeping my fingers crossed!

New Level – Infinite Library

Started working on a new level. Calling it the “Infinite Library” for now. It’s inspired by Borge’s short story “The Library of Babel”, and also the Wan Shi Tong’s Library from “Avatar: The Last Airbender”. If I had to guess, I’m guessing the writers for that Avatar episode got the inspiration from Borge’s story.

Anyway, here are some screenshots of the level. It’s all work-in-progress, hence the weird color scheme (it’s there to help me see the different componenents):

Relativity_Infinite_library_001 Relativity_Infinite_library_002 Relativity_Infinite_library_003

This is what it looks like to fall through the middle part of it:

Infinite_library_relativity

And finally, today’s Perfect Loop:

PerfectLoop_2015-01-03_Tumblr

Development Update – World 2

World 2

World 2 (working title for the level) is now feature-complete. It’s pretty buggy, but it is playable. Let the playtesting commence!

It’s probably going to go through at least 10 iterations before I figure out what the right pacing and puzzle sequence is. If I had to guess, it’s probably way too difficult and tedious at the moment. We will see. The important thing is now I have something concrete to work with and can start tweaking the design.

Here’s a bunch of screenshots of the level at the moment:

relativity_world2A

relativity_world2B

relativity_world2C

relativity_world2D

Like World 1, there is also a mico/macro relationship between different puzzles and layers of mechanics. I’m really interested in seeing if players are able to pick up on it.

Infinite Staircase World

After working on this issue on and off for quite some time, I finally figured out the correct spacing between each repetition instance in the infinite staircase world.

infinite_staircase_world

The secret is in the L-shaped platforms. They help to even out the spacing and makes it much easier to do then relying on the staircases alone.

Perfect Loop

Finally, here’s today’s perfect loop. Been keeping it up for 4 days so far!

PerfectLoop_2015-01-02_HR

Development Update – Water, Game Design, and House of Stairs

It’s been a while since I’ve last posted here. A lot has been going on. PlayStation Experience will be my last showing in quite some time. I’m back to prototyping and working on tech/design problems of the game.

Water

water still

I went back to working and refining the water mechanic. On the surface level, it doesn’t look any different from what I had before, but the code is much cleaner. Previously, the code was full of nested if/else statements that made it incredibly difficult to read and debug, especially as there were all these different cases to account for.

I finally sat down and really organized the code, moving a lot of things to separate functions which could then be called and reused regularly.

The water mechanic is now working quite well. It uses an object pool, so each “water block” is recycled (disabled and enabled again), instead of being instantiated during run time.

The boxes are basically the trigger areas. I made them visible for the purpose of debugging. There were a bunch of issues when I was using them at a larger size (1x1x1), so I made them smaller, and that fixed many of the issues.

So now, the water responds dynamically when player places a “redirection cube” in front of the stream (or removes it), and also responds when the ground beneath it moves away, or moves back into place.

3xZxqiV

6RBrDi0

Having played around with the mechanic a little more, I think it won’t actually be quite what I had originally imagined. I don’t think I will give player total freedom to redirect water streams with the “redirection cubes”. It’s just too unpredictable, causes too many design problems, and I don’t think it actually adds that much to the game.

Instead, what I’m going to do is limit the places where water can redirected, such as by having these stations in which player can change the direction of the water.

Game Design

Recently, it has been dawning on me just how massive this project is. This was especially evident as I was writing the code for water, and realizing all the possibility spaces this mechanic brings up, and all the design challenges that come with it.

I have a lot of thoughts about the design of the game’s world structure. I will sit down and write about this some more in detail next week when I’m home for the holidays and have some free time.

That being said, I think I’ve been making design decisions out of fear of negative reviews. I keep trying to make sure that the player is not lost, and can figure things out, and that moments of confusion are to be avoided at all cost. I’m starting to think I should not adhere to that philosophy so closely. Again, more about this in another post.

House of Stairs

Going to start playing with crazy architecture some more.

Here’s a space I’m trying to make which is just made of staircases everywhere.

I love the visuals, the it’s really pushing the performance of Unity to the limit. I need to figure out a way to optimize both the creation process as well as when it’s running in game.

stairs_01

stairs_02

stairs_03

stairs_04

Regarding the last two images, the player isn’t actually supposed to see that. There’s a bug which causes you to fall out of the game world. I love the look of it though. Might be interesting to see if this can worked into the game somehow.