JDroidLib
 
 

The source code of all examples is included in the JDroidLib distribution.

You must add the following three lines as second to last lines in AndroidManifest.xml
(the line before </manifest>):

<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_ADMIN" />
<uses-permission android:name="android.permission.INTERNET" />


(or check the "Use Bluetooth" and "Use Internet" checkboxes in ProjectBuilder)

 

Combining TCP/IP and Bluetooth

In a more general scenario, the commander has no direct visual contact with the robot. He/she may even be thousands of kilometers away from the robot and communicate with the robot over TCP via a mobile phone connection (GSM, GPRS, EDGE, WLAN). Because the NXT robot does not support TCP, we use a cheap Android smartphone hooked on NXT brick. This agent phone is double linked to the internet via TCP and to the NXT via Bluetooth.

Sensing

legoagent


In this example the challenge is to move the NXT robot whose initial position and orientation is unknown to the controller out of the white area delimited by the black border to the green exit. The robot reports what it "sees" by measuring the luminosity of the light reflected from the ground. It uses a standard Lego light sensor mounted in front of the NXT with its head close to the floor. (It would be nice to transmit a video image in real-time from the NXT over the Bluethooth/TCP channel back to the controller, but this requires a high data bandwidth. But some less greedy information could be returned, e.g. the current position/orientation.)

The controller takes his decision how to move the robot from the reported luminosity and the current orientation and relative position of the NXT. In this simple scenario the position/orientation is not measured and transmitted by the robot, so the controller has only a rough estimation of the position/orientation.

The implementation of the classes NxtController and NxtAgent is rather simple when using the JDroidLib library with its addons for TCP-Communication, Bluetooth and NXT robotics. Because the TCP link should work even on a wide area network where only port 80 is transparent, TCP tunneling is necessary using the concept of TcpJLib (for further information consult http://www.aplu.ch/tcpjlib).

 

NxtAgent does not need a real GUI, so the simple text window is appropriate. For debugging purposes (and fun) we display the commands received from the controller and emit a beep. The agent and the commander act as TcpNodes connected to the TcpRelay. They are automatically informed about connection/disconnection of the partner and we do not have to worry, who starts the application first.

When the NxtAgent starts it first initializes the screen and connects by Bluetooth to the NXT brick. The helper class NxtRover handles the connection, sends steering commands to the NXT and reads the intensity value from the attached light sensor. It communicates with the NXT in direct mode (no program download to the brick) using the ch.aplu.android.nxt package integrated in the JDroidLib framework (see the NxtJLib documentation for more information). The constructor of NxtRobot initiates the Bluetooth connection by requesting the Bluetooth friendly name of the NXT. If the connection is successful, a new TcpNode is created and a TcpNodeListener registered that reports status infos and messages from the TCP connection (see the TcpJLib documentation for more information). The main thread then polls the light sensor to get the luminosity value at regular intervals. The data is sent as string to the Commander via TCP using the sendMessage() method.

The statusReceived() callback handles the connection/disconnection of the remote NxtCommander. If the commander disconnects from the relay, the rover stops for security reasons but continues to read the luminosity. If the commander reconnects later on, the rover is thus still ready to move.

Commands from the commander are received by the messageReceived() callback. For simplicity the commands are coded as letters: 'S' for stop, 'F' for forward, 'B' for back, 'L' for left, 'R' for right.

import ch.aplu.android.*;
import ch.aplu.android.nxt.*;
import ch.aplu.tcp.*;
import ch.aplu.util.Monitor;

public class NxtAgent extends GameGrid
  implements NxtConnectionListener, TcpNodeListener
{
  private final String VERSION = "1.0";
  private NxtRover rover;
  private TcpNode node;
  private final String nickname = "NxtAgent";
  private static String sessionID = "777";
  protected GGTextField[] tf = new GGTextField[8];
  private volatile boolean isRunning = true;

  public NxtAgent()
  {
    super(WHITE);
  }

  public void main()
  {
    getBg().clear(BLUE);
    for (int i = 0; i < tf.length; i++)
    {
      tf[i] = new GGTextField(new Location(520 + 20 * i)true);
      tf[i].setFontSize(10);
      tf[i].show();
    }

    tf[0].setText("NxtAgent Version " + VERSION);
    tf[1].setText("Connecting Bluetooth...");

    rover = new NxtRover(this);
    rover.addNxtConnectionListener(this);
    if (rover.connect() != NxtRobot.ConnectState.CONNECTED)
    {
      tf[2].setText("Bluetooth connection failed");
      return;
    }
    tf[2].setText("Bluetooth OK. Connecting TCP...");
    node = new TcpNode();
    node.addTcpNodeListener(this);
    node.connect(sessionID, nickname);
    while (isRunning)
    {
      int value = rover.getIntensity();
      tf[6].setText("Luminosity value = " + value);
      node.sendMessage("" + value);
      delay(100);
    }
    tf[6].setText("Connection to NXT broken");
  }

  public void onPause()
  {
    if (rover != null)
      rover.exit();
    super.onPause();
  }

  public void notifyConnection(boolean connected)
  {
    if (!connected)
      isRunning = false;
  }

  public void nodeStateChanged(TcpNodeState state)
  {
    if (state == TcpNodeState.CONNECTED)
      tf[3].setText("Connected to relay");
    if (state == TcpNodeState.DISCONNECTED)
      tf[3].setText("Disconnected from relay");
  }

  public void statusReceived(String text)
  {
    L.i("statusReceived: " + text);
    if (text.contains("NxtCommander") && !text.contains("Disconnected"))
    {
      announce(1200);
      tf[5].setText("NxtCommander connected");
      Monitor.wakeUp();
    }
    if (text.contains("Disconnected"))
    {
      announce(800);
      tf[5].setText("NxtCommander disconnected");
      rover.stop();   // For security
    }
  }

  private void announce(int freq)
  {
    playTone(freq, 100);
    delay(600);
    playTone(freq, 100);
  }

  public void messageReceived(String sender, String text)
  {
    playTone(1000100);
    switch (text.charAt(0))
    {
      case 'S':
        rover.stop();
        break;
      case 'F':
        rover.forward();
        break;
      case 'B':
        rover.backward();
        break;
      case 'L':
        rover.left();
        break;
      case 'R':
        rover.right();
        break;
    }
  }
}

 

The helper class NxtRover is-a NxtRobot that uses the conventions of the NxtJLib framework.

package ch.aplu.nxtagent;

import ch.aplu.android.*;
import ch.aplu.android.nxt.*;

public class NxtRover extends NxtRobot
{
  private NxtAgent gg;
  private Motor motorA = new Motor(MotorPort.A);
  private Motor motorB = new Motor(MotorPort.B);
  private LightSensor ls = new LightSensor();
  private final int linearSpeed = 15;
  private final int angularSpeed = 3
  
  public NxtRover(NxtAgent gg)
  {
    super(gg);
    this.gg = gg;
    addPart(motorA);
    addPart(motorB);
    addPart(ls);
  }

  public int getIntensity()
  {
    return ls.getValue();
  }

  public void stop()
  {
    gg.tf[7].setText("Last command: stop");
    motorA.stop();
    motorB.stop();
  }

  public void forward()
  {
    gg.tf[7].setText("Last command: forward");
    motorA.setSpeed(linearSpeed);
    motorB.setSpeed(linearSpeed);
    motorA.forward();
    motorB.forward();
  }

  public void backward()
  {
    ...
  }

  public void left()
  {
    ...
  }

  public void right()
  {
    ...
  }
}

 

Download Android NxtAgent app for installation on a smartphone

Create QR code to download Android app to your smartphone.

Download sources (NxtAgent.zip).

The NxtCommander app provides a simple graphics interface, where the current position and direction of a NXT image is shown. The reported luminosity is transformed into a colored spot. Four buttons forward/backward/left/right are provided to command the robot. The Menu button is used to stop the robot and the Back button resets NXT image to its starting position/orientation. When the application is terminated by pressing the Home button or when the TCP link is broken, the NxtAgent is informed and stops the NXT robot for obvious security reasons.

We don't show the complete code because most of the principles are similar to the NxtAgent app. You may download the code and study the details.

One thing is worth to mention. The program uses three luminosity levels corresponding to the white, black and green color. These values depend on the color of the background where the agent moves and the ambient light. They must be determined by a calibration process. We use the Android preferences framework to store them, so they can be reused at each application invocation. GGPreferences is a small wrapper that simplifies the use of Android's SharedPreferences.xxx

private void selectLuminosity()
{
  GGPreferences prefs = new GGPreferences(this);
  vWhite = prefs.retrieveInt("whiteLevel");
  vBlack = prefs.retrieveInt("blackLevel");
  vGreen = prefs.retrieveInt("greenLevel");

  if (vWhite == null)  // preferences do not exist, set defaults
  {
    vWhite = 600;
    vBlack = 300;
    vGreen = 400;
  }

  vWhite = requestInt("Luminosity Calibration""White Level", vWhite);
  vBlack = requestInt("Luminosity Calibration""Black Level", vBlack);
  vGreen = requestInt("Luminosity Calibration""Green Level", vGreen);

  prefs.storeInt("whiteLevel", vWhite);
  prefs.storeInt("blackLevel", vBlack);
  prefs.storeInt("greenLevel", vGreen);
}

 

Download Android NxtCommander app for installation on a smartphone

Create QR code to download Android app to your smartphone.

Download sources (NxtCommander.zip).

legocommander

We like this project because it demonstrates by a typical scenario how several CPUs work gently together to provide a flawless two-way information link. Three systems are visible to us: Commander smartphone - Agent smartphone - NXT brick. But there are hundreds of hidden processors executing code when the commander presses a key. Think of all Bluetooth controllers, sensor controllers, internet and GSM routers, disk and interface controllers, etc.