CyclopsNXT

Greetings

It has been a long time since I posted any of my creations. This is because for the past few days, I had been working on possibly my most entertaining project to date: A NXT cyclops that can track faces! Before I get started, here are some pics.

Front View, base and cam

Side view, cam holder in detail

Gearing to turn the camera

I see you!

So to make this creation, I used the OpenCV wrapper for Java (JavaCV), Lejos as well as Java SE. This creations consists of two motors. One motor drives a set of gears which can tilt the camera up or down. Another set of gears rotates a geared base CW or CCW, which in turn rotates the camera. The camera itself is connected to a terminal, which is the only wired connection to an external terminal in the whole project. You can use Bluetooth webcams and do away with wires. The geared base, rotating motor and the NXT brick all sit on a pretty sturdy base, which rests on four wheels. The whole project is about 20x15x20 cm. So basically the camera has four degrees of freedom. So what does this project achieve and how does it do that? I wanted for a long time to be able to build a robot that could track objects in real time and this robot does just that. The camera gets input from the environment and sends it to the computer for processing. A Java/JavaCV program running on the computer processes this input video and draws a rectangle on all detected faces (program works with only one face though).
       Now a method in the program computes the distance of the edges of the box around the face to the edges of the video frame. If the distance from the left edge of the box to the left edge of the frame is different by more than 80px compared to the distance from the right edge of the box to the right edge of the frame, the JavaCV program on the computer sends a command to the NXT Brick to either turn left or right, so as to position the face in the center of the video frame. The same works in the y axis, where the distances from the top and bottom edges of the box around the face to the top and bottom edges of the video frame are compared. Again if the difference is more than 60px, the JavaCV program sends a command to the NXT to tilt the camera up or down depending on the scenario.
       So that is pretty much how the program works. I have posted links to the code below, from both the NXT and the Computer. It is annotated and licensed under the MIT OSI License. Feel free to go over the coed and edit it. Hopefully it is educational. If you are having problems building the robot itself, send me a message, and I can help you out.  I have posted pics above that can guide you. Oh and a huge thanks to Nashruddin whose blog taught me how to program face recognition in OpenCV! And finally here is a video showing the robot in action.

*LINKS*
Code on the NXT
Code on the computer
Classifier to detect faces, must be in the folder from which comp.class is running.

This is not the end...

The NXT step

Greetings

For a long time, I wanted to use my programming abilities in robotics. I wanted to create a robot and program it myself to respond to stimuli, unlike the pre-programmed ones that were available in the market. This meant that I had to buy motors, PC boards, micro-controllers, switches and sensors independently, assemble them and then program them.  While I searched the net to buy these parts (A lot of companies did not ship to the Middle East, where I used to live), I started getting disheartened because of the nonavailability of the most basic micro-controllers and motors.
                Fortunately for me, Lego and MIT released the Lego Mindstorms Kit. While the NXT-G programming language was sufficient, Visual Programming lacked a certain something that had drawn me to textual programming languages. When the good guys at LeJOS released LeJOS v0.85, I immediately flashed my NXT to that version(The latest version at the time of writing is 0.90, refer to this page to see how to flash your NXT and get the LeJOS environment running on your computer) and began coding in sweet, sweet Java. I had always been fascinated by remote controlled cars, so the first thing I set out to do was to create a program to control my NXT vehicle via input from the keyboard. To do this, I created a small GUI based Java program(send.java) which acted as the control center. The program had a command log, connect, disconnect, shutdown and options(The options button has no functionality currently) buttons, and a text area to receive the input form the keyboard. The program sent commands from the keyboard to the NXT(running rec.java) via Bluetooth. The NXT acts as a slave(The main thread is pretty much stopped from executing, until a Bluetooth connection opens) and waits for input from the computer and acts accordingly. My program works in a simple manner. When I press the top arrow key, the key-code of that key specified by Java is sent to the NXT and the NXT knows to move forward, and so on. The idea is simple in concept, but it does have a few problems. One of the problems is that in this model, the sending of two commands simultaneously is a bit of a hassle. Right now, the robot can turn or move, but not at the same time. One possible solution would be to multiply the key-codes of the two arrows and  return the product as input for the NXT because for the key-codes 37, 38, 39, 40, the product of any two numbers is different from the product of any two other numbers in the set(note that the same cannot be said for addition.) In any case, the program does the work of successfully sending commands to the NXT, albeit not with a lot of options. This is just a test program, and I am working on another one with greater functionality(one that has a working options button!)
                 In any case I will put the files that run on the NXT and on the computer up for download. Please note that to run these files, you should have LeJOS installed on both the NXT and the Computer. The files to be run on the NXT will be called rec.java and the file to run on the computer will be called send.java.Here is what you have to do. Open up command prompt and and navigate to the folder where you have placed the two files. Then type in the following commands:

nxjc rec.java              (compile the rec.java file)
nxj rec                    (upload the rec.class file to the NXT)
Now go the NXT and execute the rec.nxj file. Go back to command prompt.
nxjpcc send.java           (compile send.java)
nxjpc send                 (run send.class, AFTER starting rec.nxj on the NXT)

I have once again annotated the code, explaining how and why it works. Please edit the code, add your own functionality and experiment with it. I am once again releasing the code under the MIT license.  You can use any NXT car creation you want to test out the program, but the model has to turn via a difference in the speeds of the different sets of wheels, like in a tank. I used the 30 minute model from Lego Mindstorms for testing purposes, here is a link to the instructions to build a similar model. You can replace the tracks with wheels if you want. Also, here is a video of the program and the NXT in action.

Here are the links for the two.java files. I have not created any JARs as they are really not needed.

rec.java (On the NXT)
send.java (On the computer)

This is not the end.......

Onto the NXT project

Greetings

Until the NXT time it is! If you picked up from the last post, you would have guessed that the NeXT post is going to something about Mindstorms NXT. And it is. In this post I am briefly going to talk about a technology which has united my two greatest passions: Programming and Robotics! Mindstorms, by Lego, is a Robotics Kit which one use to build and program fully or partially autonomous programmable robots (Robots including, but  not limited to bipedal humanoids). Lego Mindstorms was released following a collaboration between Lego and MIT's Media Lab. The Lego Mindstorms started out as Mindstorms v1.0 and the latest version now is the Mindstorms NXT v2.0, a huge leap from its predecessors in terms of processing power by the Mindstorms Brick and availability of different kinds of sensors. Lego Mindstorms NXT is programmable through the NXT-G programming software(simple programs can be built on the Brick itself), which supports Visual Programming. I, however owning to my hacker nature, decided to go one step forward and installed the LeJOS firmware on my NXT. LeJOS allows the NXT to execute special byte-code from LeJOS specific java programs! This means that you can code all you NXT programs in leJOS Java(similar to Java ME), convert it to special byte-code with the help of LeJOS NXJ on your computer(similar to the java  JDK) and transfer the byte-code to the NXT and execute it using the LeJOS Virtual Machine. As one can see, the ability to use such a powerful language greatly increases the possibilities of the Mindstorms NXT. Also LeJOS can be used to communicate(2 way data streams) with other Java programs, whether they reside in a computer, a smart phone or even another NXT! Using you cellphone to control your NXT or developing a Client-Server relationship between the NXT and the computer can be achieved through LeJOS. The methods of communication are BlueTooth and the USB specifications. LeJOS is the not the only custom firmware for the NXT; ROBOTC and BrickOS can be used to run c/c++ programs on the NXT. For now however,  I will be sticking to LeJOS. I will soon be posting a page on how exactly to get LeJOS on your NXT, as well as many clever, innovate programs for LeJOS NXT.  Until then, take a look at a list of useful sites that I have posted down below!

Never The End.

******************
**Some Useful Sites**
******************

The LeJOS API
The LeJOS PC API
Great NXT tutorials
A NXT LeJOS forum

Hello, World!

 Greetings

And here is the first post. *Drum roll*. This blog has finally taken off after...............3 weeks. I have finally found some time to post some material to this blog, thanks to the long weekend. Anyway now that things have been set in motion(perpetual, I hope), I will soon be posting links to my own programs and source code, and describe how they work, and why they work. You can download them, edit them, redistribute them(feedback them?).......the possibilities are endless. Most of my programs will be in c++ or Java, and for those out there with Lego Mindstorms NXT, I will be posting programs for the LeJOS OS on the NXT. I also always have a ton of doubts and if you are a person who is more knowledgeable than me in the field of programming(you only have to be ahead by a class), please help me out!

This is not the end...........

Aravind Rao