Programming Solutions for the LEGO Mindstorms NXT - Which approach is best for you? Author Name: Dick Swan

by Dick Swan

Three different programming environments are available from LEGO for the NXT– NXT-G, ROBOLAB and ROBOTC, and they all make it quite easy to program Mindstorms robots. In addition, the open source community has developed other alternative programming solutions for the NXT. This article provides an introduction to the rich selection of programming approaches available today. This online expanded version of the Winter 2007 article features an extended comparison table and a lot more technical details behind this article.

NXT-G

NXT-G is a graphical programming environment developed by National Instruments for LEGO. Writing an NXT-G program is very much like creating a flowchart. You “write” a program by dragging icons (“code blocks”) that describe different behaviors, e.g., turn motor A on at 75 percent of full power, and connect them with lines to describe the program behavior. Using a variety of code blocks, you can control motors, introduce delays, play sounds and direct the flow of your code according to the state of sensors and timers, etc.

The diagram shows an NXT-G implementation for a “classical” line-following robot. The program looks at the value of a light sensor positioned above the line, and depending on which side of the line edge it is on, i.e., light or dark, turns one rear wheel on and the other rear wheel off. This is implemented as a loop that is repeated forever. Inside the loop, the light-sensor value is checked and the program branches to “true” or “false” code based on the value. NXT-G is targeted at children and adults with no programming experience, and for this reason, it is very easy to use.

NXT-G graphical programming screen for a line-following robot.

ROBOLAB

ROBOLAB was originally developed by Tufts University for LEGO for the first generation LEGO Mindstorms RCX microprocessor “brick.” It was extensively enhanced and revised to support both the RCX and the second-generation NXT.

ROBOLAB is another graphical environment, although it is not as intuitive as the NXT-G language. If you already know ROBOLAB or you need to program on both the RCX and NXT, ROBOLAB is a good choice for you. If you’re just starting and want a graphical programming language for the NXT, the NXT-G is the better choice.

ROBOLAB program screen for line following.

ROBOLAB was written using the LabVIEW system from National Instruments. LabVIEW is also the underlying core technology used to write NXT-G. Other useful ROBOLAB features include support of both integer and floating point calculations, and ROBOLAB has a comprehensive data-logging solution. Graphical interfaces such as NXT-G and ROBOLAB are very intuitive but can become tedious as you become more experienced with programming.

ROBOTC

The ROBOTC solution allows the NXT to be programmed using the industry-standard C language. It was developed by the Robotics Academy at Carnegie Mellon University and can be obtained from the LEGO Education Group or directly from the Robotics Academy at www.robotc.net. Both of the graphical programming solutions had drag-and-drop capabilities for the “code blocks.” ROBOTC has a similar capability, but with it, you drag and drop text. The left window in the picture contains the “dictionary” of ROBOTC’s built-in robotics control capabilities. ROBOTC is targeted at novices and advanced users. ROBOTC has “basic” and “expert” modes; in the “basic” mode, a lot of the advanced functionality and menus are hidden.

ROBOTC screen showing classic line-following program.

ROBOTC has a powerful interactive real-time debugger that significantly reduces the time it takes to debug programs. So far, over 2,000 students have been taught ROBOTC in the classroom; at the end of the first class, they were programming and running their first ROBOTC programs for the NXT. Carnegie Mellon University’s Robotics Academy has developed step-by-step instructional videos that enable new users to become competent programmers quickly. ROBOTC supports both the NXT and RCX as well as products from Innovation FIRST (VEX and the FIRST Robotics Competition).

NXC ... NOT EXACTLY C

NXC (Not eXactly C) is a C-like language for the NXT. NXC programs are developed using the Bricx development environment. Bricx was originally developed for LEGO’s earlier RCX robotics product and has been enhanced to support the NXT.

NXC uses the same firmware as NXT-G. This is very convenient for users who want to program in both a graphical and a text environment because they don’t have to reload and change the firmware every time they switch the environment type. You can store both NXT-G and NXC programs simultaneously in the same brick. NXC has the same limitations as the NXT-G solution. It has integer but not floating-point variables. It doesn’t have more powerful LCD text formatting.

NXC line-drawing program screen.

The NXC programming language syntax and semantics are similar to C’s, but they are also different. For example, NXC character strings start with a double quote and end with a single quote; C uses double quotes for both the start and the end. NXC and ROBOTC are the only solutions that have run-time debuggers.

PBLUA Lua is a relatively new text-based language that has gained a lot of attention and interest in the academic arena. pbLua is a full-featured version of Lua for the NXT. The pbLua implementation of the line- following program is very similar to the ROBOTC and NXC implementation.

pBLua implementation of line following.

NXJ

NXJ is a JAVA implementation for the NXT. It is standard JAVA but with a much smaller Class library. The standard Class library is far too large for the total 256K bytes of memory on the NXT. NXJ programs are written and compiled on the PC. The compiled programs are then transferred to the NXT where they can be executed.

NXJ line-following programming.

SUMMARY OF PROGRAMMING LANGUAGES AND IDEs

FEATURE NXT-G ROBOLAB

ROBOTC

NXC pbLua NXJ

LANGUAGE Graphical Graphical

C C-like Lua JAVA

PRICE $0 (retail) $50(education)

$50

$30 web $50 – CD

Open Source

Open Source

Open Source

PLATFORMS Windows, MAC

Windows,

Windows

Windows,

Windows,

Windows

OSX MAC OSX

MAC OSX, LINUX

MAC OSX

END USER FIT Novice to Intermediate

Novice to Intermediate

Novice to Advanced

Intermediate to Advanced

Intermediate or higher

Intermediate or higher

ONE BUTTON INSTALLATION

Y Y Y Y No Partial

EASE OF USE (1 to 10. 10 is best)

10 7 8 6 4 4

RELATIVE EXECUTION SPEED

1X 70X 130X (fastest)

25X 35X n/a

INTEGRATED DEVELOPMENT ENVIRONMENT

Y Y Y Y Soon (via Eclipse)

Partial (via Eclipse)

REAL TIME DEVELOPER N N Full Featured

Partial N N

SOUND PLAYBACK CAPABILITIES

Tones + WAV

Tones + WAV

Tones + WAV

Tones + WAV

Tones Tones + WAV

LCD DRAWING CAPABILITIES

Text Basic

Enhanced++

Enhanced++

Basic Enhanced

Enhanced

Graphics Basic

Enhanced

Enhanced

Basic N Enhanced

Icons Y Y Y Y N N

LANGUAGE FEATURES

Variables

Hard Hard Y Y Y Y

Floating Point

N Y Y N Y Y

Trigonometry Functions

N Y Y N Y? Y?

‘switch’ construct

Y N Y ? N Y

Arrays N N Y Y Y Y

BLUETOOTH CONNECTIVITY

Connectivity to other devices

PC, NXT None

PC, NXT, GPS, others

PC, NXT

PC, NXT, GPS, others

NXT, Others

Fantom Y Y Y N N

Protocol Supported

Duplex (speed)

Half

Full Half Full Full

ON-BRICK FILE SYSTEM Y Y Y Y N N

ON-BRICK PROGRAMMING SYSTEM

Y Y Y Y N N

INTEGRATED DATA LOGGING

N Y Y N N N

LEGO RCX COMPATIBLE N Y Y Partial N Partial

The above table was created September 2007. Many of the solutions continue to evolve and, over time, may improve their entries on an individual item. A large amount of data was condensed and simplified to create the above table. The following provides additional expansion on the table items.

Sound Playback Capabilities

FUNCTIONALITY FEATURE DESCRIPTION

TONES Play a single tone on the NXT speed at a specified frequency and for a specified duration.

TONES + WAV Play “WAV” (i.e. sound) files stored on the NXT. The sound files contain a list of sound “samples” that are played back at a fast rate.

LCD Drawing Capabilities

ATTRIBUTE FUNCTIONALITY FEATURE DESCRIPTION

TEXT

Basic • Draw a text string at specified coordinates

on the LCD screen.

Enhanced

• “Print” format controls on text string and numbers. For example, “%6d” to format a number with exactly six characters padded with blanks on left.

Enhanced++

• Multiple display fonts. ROBOTC and ROBOLAB have both the 6x8 size used in other systems and a 12x16 double size font.

• Optionally erase individual text lines before

drawing to eliminate artifacts. For example, if text line currently contains “012345” and you draw “999” you’ll get “999345” displayed without the optional erase.

• Drawing routines optimized for minimal real time.

GRAPHICS

Basic • Draw lines, rectangles, circles and single

points on LCD

Enhanced

• Erase and fill functions for rectangles and circles for expanded drawing capability.

• Draw ovals as well as circles.

ICONS

• Ability to draw ICON files to the screen.

Language Features

ATTRIBUTE CAPABILITIES

VARIABLES

• All solutions support user defined variables. • The mechanism for defining variables in the NXT-G language

is cumbersome. • Variables in graphical languages have global scope. You

cannot define variables with local scope. • Variables in text languages are letter-case sensitive. ROBOTC

has a nice feature that will generate a warning message rather than a fatal error message if you mistype the letter case; e.g. if variable is defined as “Value” and is subsequently referred to as “value” other compilers will generate a fatal “undefined variable” error whereas ROBOTC will (optionally) generate a warning only.

FLOATING POINT

• All solutions support “integer” or whole number calculations. Expression results are always rounded to integer value; e.g. “4 / 5 results in “0” and not “0.8”.

• Many text languages all support fractions or “floating point” numbers.

TRIG FUNCTIONS

• Support for trigonometry functions is useful for programs performing “dead reckoning” based on wheel encoder results.

ARRAYS

• Arrays are a useful programming concept. For example, to keep the last 10 sensor readings. Or to store a list of waypoints that robot should move to.

Bluetooth Connectivity

ATTRIBUTE CAPABILITIES

OTHER DEVICES

• Standard NXT-G firmware supports Bluetooth (BT) communications between PC and NXT and between NXTs. This includes use of the LEGO defined “Fantom” message protocol for this communication.

• There are requirements to connect a BT device that does not support the “Fantom” protocol. A good example is connecting to BT enabled GPS receivers which utilize a different messaging protocol.

DUPLEX (SPEED)

• The Fantom protocol uses the BT link in a half-duplex fashion. o The master device of BT link can send messages to a

slave device at any time. o The slave device only sends messages to master in

response to a ‘poll’ request from the master.

This results in inefficient use of the BT link and a long latency in sending messages from the slave to the master. When a slave wants to send a message, it is first buffered in an internal queue waiting for a polling request from the master to “get top message from the queue and send it back to the master”.

• Some solutions can utilize the BT link in a full-duplex fashion. The slave can immediately transfer messages to the BT transmission hardware without waiting for a poll request from the master. This significantly reduces transmission latency.

PROGRAM DEBUGGING

It often takes longer to debug a program, i.e., to find and correct errors, than it initially took to write it. ROBOTC and NXC are the only solutions that have an interactive, PC-based, real-time debugger which dramatically reduces the debugging time.

ROBOTC debugger in action. Single stepping of a program is under way and is about to execute line 132 in the source file. The “Program Debug” window contains the commands to step through the program. The “NXT Remote Screen” is similar to Windows’ remote desktop. It “remotes” the NXT LCD onto a PC window and allows you to simulate key presses with the PC mouse. And it works great over the Bluetooth wireless network.

Traditionally it is difficult to debug (i.e. find and correct errors) programs for an embedded system. The development process is a repeated cycle of:

1. Modify program source code on the PC. 2. Compile and link your program on the PC. 3. Download the linked or executable version of your program to the embedded system

(i.e. the NXT). 4. Run or execute your program and hope it works. 5. Upon failure, determine what went wrong and return to step 1.

It’s difficult to examine the state of your program as it is executing because there’s little or no visibility from the PC into the operation of the NXT CPU which is executing millions of

instructions per second.

In an enterprise environment you would typically utilize an add-on hardware debugger that attaches between the NXT’s circuit board and your PC. This hardware debugger provides low-level control and access over the CPU and memory. A typical device is shown in the picture to the right.

The average hobbyist will not use a hardware debugger! They won’t want to cut open the plastic case of the NXT and solder the connector for the hardware debugger to the circuit board. Nor will they purchase the hardware debugger which, in the above diagram, is an industry standard JTAG debugger from IAR systems.

So, for basic debugging, the hobbyist must resort to simple techniques.

• To determine if your program is executing the intended logic you insert code in your program to play different tones/sounds as different sections of code are executed. You listen to the sounds that are played to give you some indication of what parts of your program are being executed.

• Alternatively, you modify your program to display progress and debugging information on the NXT LCD screen and watch the output to determine how your

program is executing. However, watching the LCD screen can be challenging if your robot is moving position as it is running.

• If you guess wrong on where to modify your program then you have redo to examine different sections of your program code.

• When complete, you then remove the modifications for debugging.

The development systems for the NXT generate instructions for an intermediate virtual machine (VM) rather than direct instructions for the NXT CPU. When you run a program on the NXT, a VM interpreter application within the NXT firmware begins “interpreting” the intermediate instructions of your program. Adding some extra capabilities in the VM – e.g. suspend and resume the interpretation – can provide debugging functionality without the need for an external hardware module. Each programming solution for the NXT has different definition of its VM. ROBOTC is the only solution that has debugging functionality fully integrated into the VM interpreter.

The NXC programming environment also has a software based debugger. It uses a different implementation approach. Debugging functionality is not integrated into the VM. Instead the NXC compiler, transparently to the end user, generates additional debug related instructions for each line of source code. Programs execute slower with the debugger active.

The ROBOTC solution is full featured and provides capabilities comparable to the professional hardware debuggers. An integrated debugger eliminates the need to manually add code for debugging to your program. A debugger provides better functionality to the techniques discussed above without modifying your source code!

An effective PC-based real-time debugging capability has the potential to significantly reduce the time it takes to find and correct errors in your programs. With a good debugger for the NXT you can:

• Use either USB or Bluetooth (BT) for communications with the NXT. The wireless BT link is preferred because it can be pretty tough to test in real-time a moving robot if it is tethered to the PC via a USB cable!

• Start and stop your robot’s program execution from the PC.

• Suspend and resume execution in the middle of your program execution. • “Single step” through your program executing one line of code at a time and

examining the results (i.e. the values of variables) and the flow of execution. • Define “breakpoints” in your code that will cause program execution to be suspended

when they are reached during program execution • Read and write the values any of the variables defined in your program • Read and write the “values” of motors and sensors. • Be in-obtrusive. The debugger should not impact the performance (i.e. speed of

execution) of your program

The following picture illustrates some of the ROBOTC debugging windows during an interactive debugging session.

Note: To minimize the size of the picture, the two debugger windows in the above picture were positioned on top of the main ROBOTC window. This is not a requirement. They can be positioned anywhere on the PC desktop.

Four debugger capabilities are shown in the above picture. Each of these is highlighted in the following picture.

ITEM DESCRIPTION

Debugger Controls This is the main debugger window for controlling user program execution

Current Execution Point

The yellow left arrow indicates the next line of source code to be executed in the program. Yellow is used to indicate that program execution is currently suspended. Green is used when the program is “free running”.

User Defined Breakpoint

The red “stop sign” indicates a user defined break point. Program execution will be suspended if this line of code is reached during program execution. Breakpoints are easily defined by left-clicking in the “gray” column corresponding to the desired breakpoint.

Variable Display

This is a display of all the variables that are defined in your program. The values are periodically retrieved from your robot and displayed here. If desired you can select a variable and rewrite its value.

Debugger Controls

This is the main debugger window for controlling user program execution. The buttons on this window allow you to start/stop, suspend/resume program execution. Other buttons allow you to control the rate at which information from your robot is refreshed.

The names of the buttons change based on the state of your program execution. For example “Start” is shown if no program is running and “Stop” is displayed when a program is executing.

Each of the buttons is described in the following table.

BUTTON ACTION PERFORMED

Start / Stop Starts or stops the execution of a program

Suspend / Resume Temporarily suspends (or resumes) the execution of your program.

Step

Causes a suspended program to execute the next line of code in your program.

Note: Some “complicated” lines of code, especially code lines that cause a break in sequential program execution like a “for” or “while” clause may only be partially executed by a single “Step” command.

Once Will trigger a single refresh of the debugger windows. It also stops continuous refresh of the debugger windows.

Continuous / Stop Starts or stops continuous refresh of the debugger windows.

User Defined Breakpoints

“Break points” are locations in your source code where program execution can be interrupted for manual intervention.

The red “stop sign” indicates a user defined break point. Program execution will be suspended if this line of code is reached during program execution. Breakpoints are easily defined by left-clicking in the “gray” column on the source code line corresponding to the desired breakpoint.

ROBOTC breakpoints do not impact the speed with which your program executes. The

breakpoint capability is built-into the ROBOTC firmware and does not require insertion of extra instructions in your compiled code.

You can define (virtually) unlimited breakpoints within your program. There are no restrictions on the number of breakpoints within a single function or task.

Changing Flow of Program Execution

Sometime during debugging you may want to alter the normal flow of program execution. If you right-click instead of left-clicking with the mouse a pop-up menu appears. One of the commands is “Set Next Instruction” which allows you to change the currently executing point of your program to this selected line! The pop-up menu is shown in the picture below.

Debugger Displays

There are several different debugger windows that can be used when the ROBOTC debugger is operational. These windows give access to user program and built-in variables. Some of the windows are infrequently used but can be useful when needed. Debugging windows can be opened from the sub-menu shown in the following picture.

Other robot platforms may have a different list of windows customized to the robot controller

This picture shows the Debugger Windows available for the NXT platform in ROBOTC.

hardware. The number of available windows is also impacted by the “Menu Level” setting found on the “Window” sub-menu. In the “basic” mode, some of the more advanced and less-frequently used windows are not available.

Several of these windows are shown below.

This window contains a table of all the variables declared in your program along with their current values. If needed, you can select and change the value of any variables.

ROBOTC has several built-in variables that customize the performance of your robot. The “System Parameters” window provides PC access to these variables. This example is for the NXT robot controller.

Of course, all of these variables, above-right, are directly accessible within your program code.

Although not usually needed, the advanced user may find a few of these fields particularly interesting. For example:

• ‘avgBackgroundTime’ is the overhead time spent in the device driver code. The remaining CPU time is available for user program execution. This

example shows background overhead at 13%.

• ‘bNoPowerDownOnACAdaptor’ is a neat variable to prevent the NXT from automatically powering down if a rechargeable battery is being used and it is connected to AC power.

This is the “devices” debugging window for the NXT robot controller platform. Different versions of the screen are available for other platforms supported by ROBOTC.

The above-right screen provides access to current values for the motors and sensors on the NXT.

The “More” button expands the window to provide controls that enable you to “write” to the sensors and motors to setup their initial configuration.

There are no restrictions on the number of debugger windows that can be simultaneously opened. However, each window does require the PC to “message” with the robot controller to refresh its data. The more windows that are open the more data transmission required for the refreshing activity.

ROBOTC has been optimized to minimize refresh times. Typically only one or two messages are required to refresh each window; this is valuable on the robot controllers that have a “slow” communications channel between the robot and the PC.

For example, Bluetooth messaging on the NXT platform is slow, taking about 25 milliseconds per message. Out of the box, NXT Bluetooth messages are restricted to 58 bytes of data and 13 messages are required to refresh the 800 byte NXT LCD image. The ROBOTC enhanced firmware performs only requires a single message.

This debugging window is only available on the NXT robot platform.

The remote screen allows you to control your NXT from your PC. It provides an image of the NXT LCD screen. A mouse press over one of the four buttons will simulate a key press; or you can use one of the four arrow keys on your PC keyboard.

Three key capabilities differentiate ROBOTC’s debugger. Single step mode is built into the firmware and does not require the extra CPU cycles required of solutions that use extra instructions automatically inserted by the compiler to implement single stepping. A debugger requires high bandwidth on the PC-NXT link to provide fast user response; ROBOTC’s debugger is integrated with the IDE so that any or all debugging screens can be open at one time.

CONCLUSION

NXT-G is likely a good choice for a graphical programming solution. ROBOTC is a good, text-based programming solution. If you don’t need the full power of ROBOTC or want an open source solution and already have programming experience, NXC is another choice for you. pbLua and NXJ show promise but are not currently mature enough or sufficiently full-featured for most users.

Author Dick Swan is an avid LEGO robotics fan. He is also one of the principal developers of the ROBOTC system and co-developer of the ROBOLAB extensions for the NXT.

Links

LEGO Mindstorms retail,

mindstorms.lego.com LEGO Education, www.lego.com/eng/education/mindstorms/default.asp National Instruments NXT-G Programming, www.ni.com/academic/mindstorms NXJ JAVA for the NXT, lejos.sourceforge.net NXC Programming Language, bricxcc.sourceforge.net/nbc pbLUA for the NXT, www.hempeldesigngroup.com/lego/pbLua ROBOLAB graphical programming, 130.64.87.22/robolabatceeo or www.lego.com/eng/education/mindstorms/home.asp? pagename=robolab Robotics Academy ROBOTC, www.robotc.net