VEDA MIDI with Tidal

testing is science…

Part of my latest evolution in art projects has been the desire to generate and control visuals.  Previously I’ve been using Resolume Arena for doing some video transformations, and eventually using it’s FFGL plugins to kind of generate visuals based on MIDI.  This was great, but the resources used by Resolume are not light, and really it seemed a bit overkill for what I’m wanting to do.

Through the Lurk chat I stumbled on VEDA which GLSL runtime environment for the Atom editor.  This kind of became a something I really wanted to check out since I’m using Atom for all of my Tidal patterns.  Once I found out that VEDA had MIDI capabilities I started digging in more

Most of what came about here was from help in the Lurk channel by the developer Takayosi Amagi.  It would have taken me quite some time to troubleshoot things without his help and I’m extremely thankful to him.  ありがとう

First things first, you need to install VEDA.  This is pretty painless and the instructions are on the VEDA page, it’s pretty much a brew, an apm, and then answer yes to the things that Atom want’s to install afterwards.  Installation issues can likely be dealt with at the Lurk #veda channel.

Once you’ve finished installation you should open up a new project in Atom, and paste in some of the example code, save the file with the .frag extension and toggle VEDA.  Once toggled you might have to hit ctrl+enter in the window to get things started.  The output should look like

Pretty cool right?  It’s like mushrooms without the stomach ache and stinky bedroom.  Although if you stare at this long enough, you might puke.

This is probably the simplest fragment shader you could run (maybe) and I’m not going to dissect the code here.  That will be for some future posts as I dive into the GLSL language.

Next, I would suggest trying to run the example vertex shader.  Same process as above except it needs a .vert extension.  Once you get that going you should see something like this

So if you have things looking like the above examples, then you are ready to rock and roll with GLSL.

Going back to the original topic here, I wanted to be able to control VEDA using Tidal.  In a previous post I discussed how wonderfully stable SuperDirt MIDI has been using the 1.0-Dev branch.  So assuming we are up to speed with that, the first piece of code to test out is something that Amagi-San posted up on Gist in order to help me out.

I tried running the code and still came up with nothing happening, so through monitoring MIDI messages using the MIDI web console, we were able to match up numbers and resolutions to get things working.

The working code is here:

/*{ "vertexCount": 100, "midi": true }*/

precision mediump float;
attribute float vertexId;
uniform float vertexCount;
uniform float time;
uniform vec2 resolution;
varying vec4 v_color;
uniform sampler2D midi;
uniform sampler2D note;

void main() {
  float i = vertexId + time * 0.01;

  float k = (
    texture2D(note, vec2(120. / 128.)).x * 1. +
    texture2D(note, vec2(122. / 128.)).x * 2. +
    texture2D(note, vec2(124. / 128.)).x * 3. +
    texture2D(note, vec2(125. / 128.)).x
  ) * 20.;

   float m = (
     texture2D(midi, vec2(176. / 256., 119. / 128.)).x * 1. +
     texture2D(midi, vec2(176. / 256., 118. / 128.)).x * 2. +
     texture2D(midi, vec2(176. / 256., 117. / 128.)).x * 3. +
     texture2D(midi, vec2(176. / 256., 116. / 128.)).x
  ) * 20.;


  vec3 pos = vec3(cos(i * k), sin(i * 9.), cos(i * 2.));
  gl_Position = vec4(pos.x, pos.y, pos.z, 1);
  v_color = vec4(fract(vertexId / 1.), 1,1,1);
}

The float k and float m have two different uses for our tests.  ‘k’ is used when sending MIDI notes and ‘m’ is for sending CC values.  There are a couple things to point out here.  In the ‘k’ block we have called out ‘note’ and the note numbers are being used.  If you take a look at the Tidal code below, you’ll see that we are using note numbers “60 62 64 65”.  This is one area where watching the MIDI monitor helped out quite a bit.  I suspect this has to do with how the numbers in Tidal correspond to notes which have different values in MIDI, I think I need to read up on MIDI more to find the answer to this.  Regardless, you will need to put the values from the monitor in.  The other thing to note is the ‘/256 and /128’.  I’m not entirely sure what this significance of this is but Amagi-San said “midi` texture is sized 256×128″.  I think reading more into GLSL will make this more clear.  One thing I have not been able to get working is specifying the channel to use with the MIDI notes.  Because of this, I think I will be avoiding MIDI notes to VEDA until I figure it out.

In the ‘m’ block we are using CC values.  The ‘176’ value is corresponding to channel 1 (or 0 in Tidal cause we count from 0).  This can be viewed using the MIDI monitor.  After that the CC values themselves are a direct mapping (THANK GOD SOMETHING IS STRAIGHTFORWARD).

The corresponding Tidal code to test with is

d1 $ midinote "60*4 62*4 64*2 65*4" # midichan 0 # s "midi"
d1 silence
d2 $ stack [
  control (_discretise 128 $ slow 4 (scale 32 127 $ (sine))) # ctlNum 119,
  control (_discretise 128 $ slow 32 (scale 64 127 $ (tri))) # ctlNum 118,
  control (_discretise 128 $ slow 64 (scale 16 127 $ (sine))) # ctlNum 117
] # midichan 0 # s "midi" # midicmd "control"

I have the code sent through two Tidal channels so that you can just swap variables around in VEDA and see the difference.  This example video shows what happens when just sending the CC values out.