After trying to find a simple to use 3D application to see how I can use the WiiChuck to control servos for a robotic arm, I settled on 3D Studio Max (3DS) because it’s the one I know best.

But how could I transfer the datas from the arduino to 3DS ?
Well, 3D Studio Max have a nice SDK and a lot of stuff, but it’s too hard for me to code a plugin using the serial port. So I stuck with Processing. Maybe I’ll try to check into MaxScript to see if it can communicate with the serial port.

A good way to control objects in 3DS is to use the track view to assign different type of controllers to the various properties of the object you want to control. For instance, if you want to have small vibrations on the tip of a bird’s wing, you can use a Noise Controller. For this setup, I used the Motion Capture controller.

3DS can use different inputs to allow motion capture : Mouse, Keyboard, Joystick or MIDI. I wanted to use some kind of “translation” in processing to emulate a HID device, but it seems to be impossible… So I used MIDI.

Here’s my setup under windows :
- Arduino using a WiiChuck adapter from todbot (thanks kurt ^^) and the WiiChuck library from Tim Hirzel
- Data sent to Processing via serial connection and translated to MIDI CC messages using the proMIDI library by Christian Riekoff
- MIDI output from processing sent to midiYoke
- midiYoke sends this data to Ableton Live
- Ableton re-sends the CC messages to midiYoke
- Using Float Motion Capture controllers on 3D Studio Max to rotate the objects according to the pitch and roll of the wiichuck…

I know it’s a little bit complicated but, I haven’t yet figured why, 3DS doesn’t want to read directly from midiYoke without the Ableton Live gate…

The drawback here is that CC messages are 8 bits only (128 values), so I still have to figure out some way to have finer resolution and handling negative values.

Anyway, I think it’s really interesting because you can use the Arduino with any kind of sensor to output your data directly into 3D Studio Max to make motion capture, for exemple, by putting tiny accelerometers on each tip of your finger, or whatever.

Here are the test codes, nothing fancy or cleaned up :

Arduino
(the WiiChuck.h library is a bit modified, to return 1 while buttons are pressed, not only on the button press)

#include "Wire.h"
#include "WiiChuck.h"

WiiChuck chuck = WiiChuck();
int zPress = 0;
int aX, aY, aZ;

// Arrays to store the values and smooth out the output
int rollA[8],pitchA[8],xA[8],yA[8],zA[8];
int i=0;

void setup() {
  Serial.begin(115200);
  chuck.begin();
  chuck.update();
  chuck.calibrateJoy();
}

void loop() {
  chuck.update();
  int roll, pitch, x, y, z, bZ, bC, joyX, joyY;
  // Cycle through arrays and store values
  rollA[i] = (int)chuck.readRoll();
  pitchA[i] = (int)chuck.readPitch();
  xA[i] = (int)chuck.readAccelX();
  yA[i] = (int)chuck.readAccelY();
  zA[i] = (int)chuck.readAccelZ();

  // Smooth values
  for (int j=0;j<8;j++) {
    roll+=rollA[j];
    pitch+=pitchA[j];
    x+=xA[j];
    y+=yA[j];
    z+=zA[j];
  }
  roll=roll/8;
  pitch=pitch/8;
  x=x/8;
  y=y/8;
  z=z/8;

  // Send the values via serial only if Z button is pressed
  if (chuck.zPressed() == 1) {
    if (zPress == 0) {
      zPress = 1;
    }
    Serial.print(roll);
    Serial.print(",");
    Serial.print(pitch);
    Serial.print(",");
    Serial.print(x);
    Serial.print(",");
    Serial.print(y);
    Serial.print(",");
    Serial.print(z);
    Serial.println();
  } else {
    zPress = 0;
  }

  // Increment position in Arrays
  i++;
  if(i==8) {
    i=0;
  }

  delay(20);
}

Processing

int sensorCount = 5;                        // number of values to expect

import processing.serial.*;
Serial myPort;                // The serial port

import promidi.*;
MidiIO midiIO;
MidiOut midiOut;

int BAUDRATE = 115200;
char DELIM = ','; // the delimeter for parsing incoming data

void setup()
{
  size(100,100);
  background(0);
  myPort = new Serial(this, Serial.list()[0], BAUDRATE);
  myPort.clear();

  //get an instance of MidiIO
  midiIO = MidiIO.getInstance(this);
  midiIO.printDevices();
  //open an midiout using the fourth device and the fifth channel
  midiOut = midiIO.getMidiOut(5,4);
}

void draw()
{
  // NOTHING
}
float[] sensorValues = new float[sensorCount];  // array to hold the incoming values

void serialEvent(Serial myPort) {
  // read incoming data until you get a newline:
  String serialString = myPort.readStringUntil('\n');
  // if the read data is a real string, parse it:

  if (serialString != null) {
    // split it into substrings on the DELIM character:
    String[] numbers = split(serialString, DELIM);
    // convert each subastring into an int
    if (numbers.length == sensorCount) {
      for (int i = 0; i < numbers.length; i++) {
        // make sure you're only reading as many numbers as
        // you can fit in the array:
        if (i <= sensorCount) {
          // trim off any whitespace from the substring:
          numbers[i] = trim(numbers[i]);
          sensorValues[i] =  float(numbers[i]);
        }
        // Put the value between 0 & 127
        int val = Math.min(127,Math.abs((int)sensorValues[0]));
        // Sends the value as Controller 10 on the MIDI channel we opened
        midiOut.sendController(
          new Controller(10,val)
        );
        // Problem with the pitch, so -1700, really not accurate ^^
        int val2 = Math.min(127,Math.abs((int)sensorValues[1]-1700));
        midiOut.sendController(
          new Controller(11,val2)
        );
      }
    }
  }
}