iPad to Computer Comms
Posted on Wednesday 28 November 2012
|
1 Comment
Further research into the game's communication systems led to the discovery of the OSC (Open Sound Control) protocol. This is often used instead of MIDI in electronic musical instruments to allow communication and control between different devices.
Much of the inspiration for this section of the project came from Pete-O's excellent work on OSC and the Arduino which can be found here.
For this project we used TouchOSC on the iPad, as there are two iPads available for use within the group (freeing up iPhones and iPod touches for possible camera use). This app costs £3, and includes a free interface designer for Windows. Using this designer, an application was developed to send robot control signals from the iPad, this interface can be seen below.
Once run, the app was configured with host IP address and port of the Processing application which is discussed later.
Processing is an IDE and language that builds on Java, but with a simpler graphics programming model. This is ideal for us, as it works very well alongside the Arduino development environment, which was developed on the same interface. This makes it relatively easy, and well documented online to get an Arduino talking to the Processing sketch. In addition, there is an OSC library available, allowing us to parse and process OSC messages received over a local network. The OSC library has been developed by Andreas Schlegel, and is available here.
Much of the inspiration for this section of the project came from Pete-O's excellent work on OSC and the Arduino which can be found here.
For this project we used TouchOSC on the iPad, as there are two iPads available for use within the group (freeing up iPhones and iPod touches for possible camera use). This app costs £3, and includes a free interface designer for Windows. Using this designer, an application was developed to send robot control signals from the iPad, this interface can be seen below.
 |
| iPad interface for control - note space at the top for future game messages |
Processing is an IDE and language that builds on Java, but with a simpler graphics programming model. This is ideal for us, as it works very well alongside the Arduino development environment, which was developed on the same interface. This makes it relatively easy, and well documented online to get an Arduino talking to the Processing sketch. In addition, there is an OSC library available, allowing us to parse and process OSC messages received over a local network. The OSC library has been developed by Andreas Schlegel, and is available here.
Packet
For the Arduino to effectively pick up and parse the signals being sent concurrently, the design decision was made to use a defined interface packet that would carry every command at once. Once the Arduino received a packet, it would act out all of the signals, even if they had not changed. A packet was only to be sent if a single control signal had changed. This packet was to start with the character '<' and finish with '>'.
The format for this prototype is shown below:
< - Indicates start of packet
F or f - This sets whether the laser is on or off.
XX - These two characters can be read together to show the X value of a servo
XX - Shows the Y value of a servo
> - Indicates end of packet.
It is very easy to add more information to this packet as required in the future, for example motor control and which robot this message is for (as the communications are likely to be based on broadcast topology).
It is very easy to add more information to this packet as required in the future, for example motor control and which robot this message is for (as the communications are likely to be based on broadcast topology).
Parsing
In order to interpret the OSC messages correctly, it was necessary to print out the messages to the Processing console to see the exact values that were being received. The following code was used.
import oscP5.*;
OscP5 oscP5;
void setup() {
size(400,400);
frameRate(25);
/* start oscP5, listening for incoming messages at port 8000 */
oscP5 = new OscP5(this,8000);
}
void draw() {
background(0);
}
void oscEvent(OscMessage theOscMessage) {
/* print the address pattern and the typetag of the received OscMessage */
print("### received an osc message.");
print(" addrpattern: "+theOscMessage.addrPattern());
println(" typetag: "+theOscMessage.typetag());
println("value " + theOscMessage.get(1).floatValue());
}
Using this Processing sketch it was possible to experiment with the different values being sent and received, and learn more about how the OSC protocol works.
Sending to Arduino
As the Arduino will be connected to a serial port (via an Xbee network) it can be dealt with as with any other serial device. We find the Arduino in the list of serial devices using the following line:
arduinoPort = new Serial(this, Serial.list()[0], 9600);
In my laptop it shows up as the first (index 0), others may vary. It is possible to print this list to figure it out.
The OSC message is parsed and the appropriate variables are updated. Once the program drops into the constantly running draw() method, the message is only written to the serial port if the message has changed to avoid flooding of the port.
The Processing code is shown below. This program takes in OSC values, and converts them to the defined packet for controlling the laser firing, and the servo positions.
The OSC message is parsed and the appropriate variables are updated. Once the program drops into the constantly running draw() method, the message is only written to the serial port if the message has changed to avoid flooding of the port.
The Processing code is shown below. This program takes in OSC values, and converts them to the defined packet for controlling the laser firing, and the servo positions.
import oscP5.*; // Load OSC P5 library
import netP5.*; // Load net P5 library
import processing.serial.*; // Load serial library
Serial arduinoPort; // Set arduinoPort as serial connection
OscP5 oscP5; // Set oscP5 as OSC connection
//initialised to first two digit number to facilitate easier parsing on Arduino
int xValue = 10;
int yValue = 10;
int laserFire =0;
String oldMessage = "";
String newMessage = "";
void setup() {
size(100,100); // Processing screen size, dummy GUI for prototyping
noStroke(); // We don’t want an outline or Stroke on our graphics
// Start oscP5, listening for incoming messages at port 8000
oscP5 = new OscP5(this,8000);
// Set arduino to 9600 baud
arduinoPort = new Serial(this, Serial.list()[0], 9600);
}
void oscEvent(OscMessage theOscMessage) {
// This runs whenever there is a new OSC message
String addr = theOscMessage.addrPattern();
// Creates a string out of the OSC message
if(addr.equals("/1/fireButton")){ // Filters out firebutton on iOS
laserFire = int(theOscMessage.get(0).floatValue());
}
if(addr.equals("/1/xy1")){ // Filters out XY pad on iOS
// x and y values sent 1 after the other
xValue = int(theOscMessage.get(0).floatValue());
yValue = int(theOscMessage.get(1).floatValue());
}
}
void draw() {
background(50);
if(laserFire == 0){ // if laser is set to zero set the below message
newMessage = "<f"+xValue+yValue+">";
}else{ // if not, send capital F
newMessage = "<F"+xValue+yValue+">";
}
if(!oldMessage.equals(newMessage)){ // as this procedure runs all the time,
// only send if message has changed
arduinoPort.write(0); // FLUSH SERIAL PORT BEFORE SENDING TO AVOID CONFUSION
arduinoPort.write(newMessage); // send message
print(newMessage); // print to console
oldMessage = newMessage;
}
delay(50); //very short delay to prevent overloading
}
void serialEvent(Serial arduinoPort)
//overrides serial event, so that Arduino can echo received messages
// (serial monitor in Arduino IDE not available as processing is using serial port)
{
String input = arduinoPort.readStringUntil('\n');
if (input != null) {
println("Receiving:" + input);
}
}
The last method, serialEvent allows the Arduino to send messages back to Processing, which can then print them out. This means that we can have the Arduino echo any message that it receives, which is useful for debugging, when we can't open the Arduino serial monitor.
Top stuff, great upload!! Taught me how to read xy pads and other multi inputs in touch osc.
ReplyDelete