This article is my second explaining the fundamentals of coding with the Beaglebone. In the first article I explained some of the mysteries of pin muxing, and gave an example of a very simple usage of a digital pin. This time, I’ll use an analog sensor and serial I/O (just O, actually), to create a time and temperature LCD display.

I have to admit that I’m far from an expert – I’m basically writing about this stuff as I figure it out.

The Beaglebone will hopefully prove to be a ground-breaking product, introducing a lot of electronics hobbyists to embedded Linux programming. Unfortunately, at this point there isn’t much in the way of embedded Linux development tools or tutorials that is geared to newcomers. This should change by the summer of 2012, as bonescript expands and more Beaglebone-based projects pop up on blogs and sites like Instructables. Until then, it will be one baby step at a time…

Analog Input

The first rule of analog input with the Beaglebone is:
NOTE: Maximum voltage is 1.8V. Do not exceed this voltage. Voltage dividers
should be used for voltages higher than 1.8V.

So sayeth the Beaglebone System Reference Manual (Emphasis theirs. )

I haven’t experimented with what happens if you go above 1.8V, but since they’ve made the warning red and underlined, I’m pretty sure it would not go well.
This limitation is a problem, since:

• There is no pin on the Beaglebone that provides a 1.8V source
  [Correction Jan 25 - as pointed out in the comments, there is a pin that provides 1.8V, Port 9 pin 32 (labelled VDD_ADC in the Beaglebone System Reference Manual).    So you can ignore the stuff about voltage dividers and level converters below if you are using an analog sensor that can operate on 1.8V, like a potentiometer.]
• Many analog sensors require a minimum voltage greater than 1.8V

So, until someone has a better idea, voltage dividers will be an important part of any Beaglebone analog circuit.

The second most important thing to know about the Beaglebone’s analog input is that the software support seems to be a work-in-progress. The approach I’m taken is based on a brief blog post by one of the maintainers of the Beaglebone Angstrom demo images. His remarks would suggest that they have a better plan that is under development. As far as I know, the approach taken here is an Angstrom kernel modification that won’t work at all on a Ubuntu image with the vanilla Ubuntu kernel.

Back to the issue of 1.8 max voltage. You have a couple of options in your design:

• Added Jan 25: Use Port 9 Pin 32 as your voltage source, if your analog sensor can work on 1.8V
• Use a couple of high precision resistors that can knock 3.3V or 5V down to exactly 1.8V
• Use a couple of garden variety resistors to knock the voltage below 1.8V, then modify your software to adjust for the difference

Being lazy, I’m going with the latter approach. Being very lazy, I’m not even bothering to hook up the resistors – as shown in the screenshot below, I’m using a tiny breakout board to do the work.

Specifically, this is a Sparkfun Logic Level Converter – BOB-08745. You’ll see this part recommended quite often in Beagleboard/Beaglebone circles, since it is an inexpensive solution that can handle a range of voltages, and supports 2-way conversion (necessary for I2C).

For the purpose of analog input I’m not using the board as intended – I’m taking advantage of a semi-documented hack. The RX pins on this board just use a voltage divider: the voltage going into the “HV” pins comes out as half the amount from the LV pins. The circuit for this, taken from the BOB-08745 schematic, is shown below.

If you don’t have a BOB-08745, you can just use a couple of 10K resistors.

Before actually using this type of circuit, measure what voltage you get from the LV end of the voltage divider if the HV end is connected directly to 3.3V. If it isn’t below 1.8V, figure out what’s wrong before you hook it up to the Beaglebone. (Red and underlined, remember?) Don’t worry if the reading isn’t exactly 50% – we will calibrate the software to handle the variance.

The 2 analog sensors I’m using are a 10K potentiometer and an analog thermistor (temperature sensor), the Microchip MCP9700A.

As you can see from the photos, the connections are:

- Potentiometer and MCP9700A:

• Input pin to 3.3V (which connects to pins 3 or 4 of Port 9 on the Beaglebone),
• Ground pin to ground (connects to pins 1 or 2 of Port 9 of the Beaglebone),
• Output pin to one of the RXI pins on the Sparkfun level converter (or to the HV pin on the voltage divider circuit)
• Added Jan 25: An alternative for the potentiometer is to connect the input pin to Port 9 Pin 32 (VDD_ADC)., and the output pin directly to the Beaglebone AIN0 pin (pin 39 on Port 9)

- Sparkfun level converter:

• HV to 3.3V
• HV GND and LV GND to Ground
• LV RXO for the potentiometer to the Beaglebone AIN0 pin (pin 39 on Port 9, or the outside pin of the 4th row from the “bottom” – the end opposite the Ethernet port)
• LV RXO for the MCP9700A to the Beaglebone AIN1 pin (pin 40 on Port 9, or the inside pin of the 4th row)

Note that the LV pin on the level converter isn’t used, since we are just interested in the voltage divider.

The definitive guide to the Beaglebone pin connections is the System Reference Manual. For this project we are just using Port 9, so you can refer to the table below:

Testing Analog Input

Remember all that stuff about pin muxing from the first article? You can forget about it for analog in.

As you can see by looking at Table 12 of the Beaglebone System Reference Manual (part of which is shown below), the mux table for all of the analog in pins (AIN1 to AIN6) is empty: the pins can only be used for analog in.

There actually are entries for these pins in the file system, /sys/kernel/debug/omap_mux, but they have the correct setting by default, and as far as I can tell the Mode setting has no effect on the pins. (The Python code I wrote sets them anyway, but that’s admittedly more out of ignorance than caution.)

So, we can proceed directly to using the pins.

Let’s start with the potentiometer on analog 0 (pin 39 of Port 9). To get the current reading from the pot, enter the following (yes, it’s ain1 for analog 0 – go figure):

root@beaglebone:~# cat /sys/devices/platform/tsc/ain1
 1807

A number should be displayed. Turn the pot all the way up and repeat. You’ll see the maximum value.

For me, it’s 3779. Or 3775. Or 3776. It drifts around a little. Go figure.

The analog input pins on the Beaglebone are 12-bit, so the maximum possible value is 4096. This represents a value of 1.8V. At 3779, it’s reading 3779/4096 * 1.8V, or 1.66V, which is about what you would expect with a 10K/10K voltage divider with 3.3V input.

Now, check the MCP7200A thermometer on analog 1 (again, the analog index is off by 1, so it’s ain2):

root@beaglebone:~# cat /sys/devices/platform/tsc/ain2
 826

You should get a relatively low value: if it’s above 1000, either you are in hell or your thermistor isn’t hooked up correctly.

The formula for converting this reading to a temperature is:

(milliVolts – 500) / 10

The 826 reading I got is 826/4096 * 1.8V, or 362 mV.

Is it cold in here, or is it just me?

It’s just me. The voltage divider knocked the reading from the MCP7200A down by about half, or the actual reading is 362 * 2, or 724 mV. The temperature reading is therefore (724 – 500) / 10, or 22.4. (No, that isn’t cold, it’s just metric).

Actually, neither the MCP9700A nor this approach is very accurate. The MCP7200A’s datasheet says it’ typical accuracy is to within 1 degree C . There isn’t anything we can do to improve that, but we can make our calculation more accurate by allowing for slight variances of the resistors in our voltage divider.

To see what effect the voltage divider is having, temporarily hook the input to 3.3V instead of the MCP9700A’s output pin. Then check the value of /sys/devices/platform/tsc/ain2 again.

And again. And again.

I get 3765. And 3762. And 3759.

Given that each 10mV is a degree, that jumping about is a real problem. We’ll plug the average (call it 3762) into the code, then average a bunch of readings to get the temperature, approximately.

Serial I/O

The Beaglebone has 6 serial UARTs. One of those, UART0, is connected to the USB port, but that leaves us with 5 to play with. This is quite a step up from the Arduino’s single UART, or the Netduino’s 2 UARTs.

All of these are 3.3V UARTs. This is a problem if you’re communicating with a 5V device, but only when receiving data. Some 3.3 microcontrollers, like the Netduino’s, have “5V tolerant” pins, so they can communicate directly with 5V serial devices. But, for the Beaglebone:

NOTE: Do not connect 5V logic level signals to these pins or the board will be
damaged.

So sayeth the Beaglebone System Reference Manual.

I assume this means that connecting an UART RX pin directly to the TX pin of a 5V serial device is a bad idea. I did it for awhile, and my Beaglebone lived to tell the tale, but maybe I got lucky.

Were we receiving data from the Serial LCD, then a level converter would be required, like the Sparkfun BOB-08745 described earlier. (Actually a second level converter would be needed, since we would be going between 3.3Vand 5, not 3.3 and 1.8ish.)

However, it will be serial TX only for this project, and all 5V serial devices that I’ve come across handle 3.3V on the RX pin without problems.

In the demo Angstrom image, the pins are not enabled for UART by default. Yup, time for the black art of pin muxing again.

Working with UART1’s mux mode is relatively straightforward. UART1 is the Mode 0 usage for the pin, and as you may recall from my first article, the pin name used in the file system are taken from the Mode 0 usage. To check the current settings:

root@beaglebone:~# cat /sys/kernel/debug/omap_mux/uart1_rxd
 name: uart1_rxd.(null) (0x44e10980/0x980 = 0x0037), b NA, t NA
 mode: OMAP_PIN_OUTPUT | OMAP_MUX_MODE7
 signals: uart1_rxd | mmc1_sdwp | d_can1_tx | NA | NA | NA | NA | NA

 root@beaglebone:~# cat /sys/kernel/debug/omap_mux/uart1_txd
 name: uart1_txd.(null) (0x44e10984/0x984 = 0x0037), b NA, t NA
 mode: OMAP_PIN_OUTPUT | OMAP_MUX_MODE7
 signals: uart1_txd | mmc2_sdwp | d_can1_rx | NA | NA | NA | NA | NA

As you can see, both are set to 0×37 by default in the Angstrom demo image. This setting means they are in Mode 7 (as the 2nd line of the output confirms) and that the Receive feature is enabled for both pins.

Let’s briefly take a closer look at the meaning of these mux pins. In my last article I explained the Mode settings, but not the other bits. The full list of the bit settings can be found in Table 9-58 of the AM335x Technical Reference Manual – you can find a link to it here. I’ll save you the trouble of searching for the table in that 4500-page behemoth:

The 2 UART pins currently have a mux setting of 0×37, or 0011 0111. So:

• The slow slew rate is selected – this might concern me if I knew what it meant.
• The Receiver is enabled – we want that for the RX pin, not so much for the TX pin
• The pullup/pulldown is set to pullup – not a concern for serial communication
• The pullup/pulldown is enabled
• The 3 mode bits are all on, to indicate mode 7

We want to set both pins to Mode 0, and we also want the receiver disabled on the TX pin:

root@beaglebone:~# echo 20 > /sys/kernel/debug/omap_mux/uart1_rxd

 root@beaglebone:~# cat /sys/kernel/debug/omap_mux/uart1_rxd
 name: uart1_rxd.uart1_rxd (0x44e10980/0x980 = 0x0020), b NA, t NA
 mode: OMAP_PIN_OUTPUT | OMAP_MUX_MODE0
 signals: uart1_rxd | mmc1_sdwp | d_can1_tx | NA | NA | NA | NA | NA

 root@beaglebone:~# echo 0 > /sys/kernel/debug/omap_mux/uart1_txd

 root@beaglebone:~# cat /sys/kernel/debug/omap_mux/uart1_txd
 name: uart1_txd.uart1_txd (0x44e10984/0x984 = 0x0000), b NA, t NA
 mode: OMAP_PIN_OUTPUT | OMAP_MUX_MODE0
 signals: uart1_txd | mmc2_sdwp | d_can1_rx | NA | NA | NA | NA | NA

Having said all that, I’m actually going to use UART2 for this project.

UART2 is a little trickier, since its pins have other names in the file system. Looking at Table 11 of the Beaglebone System Reference Manual, we see that UART2_TXD is pin 21, and UART2_RXD is pin 22. Table 12 (below) tells us that pin 21’s Mode 0 usage (and therefore the name used in the file system) is spi0_d0, and that UART2_RXD is the Mode 1 usage. For Pin 22, it’s spi0_sclk, and we also want to change it to Mode 1.

Here are the commands for checking the current settings, and changing the settings to use UART2:

root@beaglebone:~# cat /sys/kernel/debug/omap_mux/spi0_d0
 name: spi0_d0.(null) (0x44e10954/0x954 = 0x0037), b NA, t NA
 mode: OMAP_PIN_OUTPUT | OMAP_MUX_MODE7
 signals: spi0_d0 | NA | NA | NA | NA | NA | NA | NA

 root@beaglebone:~# echo 1 > /sys/kernel/debug/omap_mux/spi0_d0

 root@beaglebone:~# cat /sys/kernel/debug/omap_mux/spi0_d0
 name: spi0_d0.(null) (0x44e10954/0x954 = 0x0001), b NA, t NA
 mode: OMAP_PIN_OUTPUT | OMAP_MUX_MODE1
 signals: spi0_d0 | NA | NA | NA | NA | NA | NA | NA

 root@beaglebone:~# cat /sys/kernel/debug/omap_mux/spi0_sclk
 name: spi0_sclk.(null) (0x44e10950/0x950 = 0x0037), b NA, t NA
 mode: OMAP_PIN_OUTPUT | OMAP_MUX_MODE7
 signals: spi0_sclk | NA | NA | NA | NA | NA | NA | NA

 root@beaglebone:~# echo 21 > /sys/kernel/debug/omap_mux/spi0_sclk

 root@beaglebone:~# cat /sys/kernel/debug/omap_mux/spi0_sclk
 name: spi0_sclk.(null) (0x44e10950/0x950 = 0x0021), b NA, t NA
 mode: OMAP_PIN_OUTPUT | OMAP_MUX_MODE1
 signals: spi0_sclk | NA | NA | NA | NA | NA | NA | NA

A Python Time and Temperature Display for the Beaglebone

I’ve written some Python code to try out the analog and serial pins of the Beaglebone. It gets the temperature from the MCP9700A, and uses the potentiometer to set the backlight of the LCD.

You can download the Python program from my Google Code project page: it’s just a single file, serdisplay.py.

From the Beaglebone command line, you can download the Python program as follows:

root@beaglebone:~# wget http://gigamega-micro.googlecode.com/files/gpiotester.py

I’ve tested the code with the Sparkfun Serial Enabled LCD Kit, the Sparkfun Serial Enabled Backpack. I’ve also coded support for Matrix Orbital serial LCDs, but haven’t tested that yet.

The code uses one Python library that isn’t included in the Angstrom demo image, python-serial. To install it:

root@beaglebone:~# opkg install python-serial

There is no configuration file or command line parameters yet, so any changes to the default settings (the Sparkfun Serial LCD Backpack and a 4-row 20-column LCD) should be made to the code near the top of the program:

# -------------- configurable settings ---------
# settings for UART1
 #DISPLAYPORT = '/dev/ttyO1'
 #RX_MUX = 'uart1_rxd'
 #TX_MUX = 'uart1_txd'
 #MUX_MODE = 0
# settings for UART2
 DISPLAYPORT = '/dev/ttyO2'
 RX_MUX = 'spi0_sclk'
 TX_MUX = 'spi0_d0'
 MUX_MODE = 1
# settings for Serial LCD
 DISPLAY_TYPE = 'SPARKFUN_KIT'
 #DISPLAY_TYPE = 'MATRIX_ORBITAL'
 #DISPLAY_TYPE = 'SPARKFUN'
# settings for analog voltage conversion
 MAX_ANALOG = 3762 # approx 4096 * (1.65/1.8), since voltage divider gives me max of 1.65
 # -- actual values of 3762 based on measurements
# settings for display updates
 TIME_UPDATE_INTERVAL = 1 # every second
 TIME_DISPLAY_ROW = 1
 TEMPERATURE_DISPLAY_ROW = 2
 LCD_NUM_ROWS = 4
 LCD_NUM_COLS = 20
# determines whether debug info is written to the console
 Debug = True
# ---------------------------------------------------------

Then, just run the program as root:

root@beaglebone:~# python serdisplay.py

To stop the program, press Ctrl-Z to put it in the background, then kill the background job:

root@beaglebone:~# kill %1

If you want to keep the program running all the time, I’d recommend using the GNU Screen utility:

root@beaglebone:~# screen
<Press enter when prompted>
root@beaglebone:~# python serdisplay.py

To return to the main command command prompt (leaving the Python program running in the background) press Ctrl-A D.  To go back to the Screen session at any time in the future:

root@beaglebone:~# screen –r –d

For troubleshooting, any error messages are written to serdisplay.log in the same directory as serdisplay.py.

Programmer’s Show And Tell

Since the focus of this article is the Beaglebone, not Python, I’ll just point out some Beaglebone-specific parts of the code.

The code which initializes the UART for the serial display is:

DISPLAYPORT = '/dev/ttyO2'
 RX_MUX = 'spi0_sclk'
 TX_MUX = 'spi0_d0'
 MUX_MODE = 1
 BAUDRATE = 9600
 TIMEOUT = 3 # serial port timeout is 3 seconds - only used when reading from display
# MUX settings
 RECEIVE_ENABLE = 32
. . .
open('/sys/kernel/debug/omap_mux/' + RX_MUX, 'wb').write("%X" % (RECEIVE_ENABLE + MUX_MODE))
 # set the TX pin for Mode 0
 open('/sys/kernel/debug/omap_mux/' + TX_MUX, 'wb').write("%X" % MUX_MODE)
 serDisplay = serial.Serial(DISPLAYPORT, BAUDRATE, timeout=TIMEOUT)

There are Linux character devices assigned to the BeagleBone UARTs: UART1 is /dev/ttyO1, UART2 is /dev/ttyO2, and so on.

From the point of view of the Python program, the Beaglebone’s UART behaves like any other serial port. The code I wrote would work unchanged (aside from the /dev name) when communicating with an XBee, or through a USB-based virtual COM port.

Analog input is a little more finicky. The code which reads the analog value is:

def readAnalog(pinIndex):
 try:
    # add 1 to pin index to get analog pin sys filename
    reading = open("/sys/devices/platform/tsc/ain" + str(pinIndex + 1), "r").read()

    # sometimes string has trailing nulls - delete them
    val = int(re.sub(r'[^\d]+', '', reading))
    return val
 except:
    log.exception('Error in readAnalog')

As indicated by the comments, I found that the value read through the file system was sometimes corrupted: it contained nulls, usually after the value, but occasionally imbedded within the value. I’m not sure if this is a Python-specific thing, or other code would also encounter the problem. In Python, you can strip out the null (and any other non-numeric) values using a regular expression:

val = int(re.sub(r'[^\d]+', '', reading))

From time to time, the attempt to read the analog value will throw an exception.  So, it’s best to use a try/except handler like the one above.

One other problem is that the analog value tends to jump around by about 5 points from one reading to the next, even when it should be constant (e.g. from the potentiometer). I adjusted for this by using an average reading for the temperature, and by ignoring potentiometer readings (i.e. LCD brightness settings) that are within 20 of the last setting.

Don’t forget that the analog value is from 0-4096 – 16 times more sensitive than on the Arduino —  so allow for a greater range of readings.

Wrapping Up

Based on my testing of the code in this project, I’ve found that the Beaglebone’s UARTs are quite reliable and pretty easy to use.  Serial communications should be a relatively easy and trouble-free addition to any Beaglebone project.

Analog in, on the other hand,  has room for improvement.

The approach I used for reading analog input in is good enough for things that don’t require precise accuracy, like the potentiometer or a light meter.  The readings returned by the MCP9700A thermistor are not as stable as they should be.

Fortunately, analog is not the only game in town when it comes to temperature readings, so I plan to experiment with some I2C sensors in the future.

In November, Texas Instruments, Digi-Key and the other members of Beagleboard.org, the Beaglebone.  This is a simpler, more hobbyist-friendly little brother to the Beagleboard (which I’ve written about in past articles).

Compared to the Beagleboards, the Beaglebone is considerably less expensive ($90), provides access to all of its pins, is pin-compatible with 3.3V sensors and devices (aside from analog in, which is still 1.8V), and provides a more friendly “out of the box” experience.  As with the Arduino, the only thing you need to get started is a USB cable.

As a developer, my most pleasant surprise with the Beaglebone was the inclusion of an entry-level IDE and scripting language: the Cloud9 IDE configured to run node.js and bonescript.

With the Beagleboards, there was no “out of the box” IDE.  There were plenty of options for developing with the Beagleboard – Texas Instrument’s Code Composer Studio, Eclipse, or the command-line GCC tools being the most prominent – but all had a fairly steep learning curve just to do something basic, like blink an LED.

One of the most slippery patches on that learning curve is figuring out how to convince the Linux kernel to let your code access the pins.  It’s nowhere near as simple as Arduino’s “digitalwrite(13, HIGH)” – or, at least, it wasn’t until bonescript came along.  Although bonescript is still in its very early stages, there is enough meat on it to give the embedded Linux newbie a friendly introduction to coding.

The Circuit

The circuit is your garden-variety LED configuration:

• an LED
• a resistor (680R or thereabouts) connected to the negative pin of the LED
• a jumper from the positive pin of the LED to the digital pin
• a jumper from the resistor to ground

The Beaglebone actually has 65 GPIO pins to choose from, but we’ll go with the one used by the blinkled.js program that is included with bonescript.  This is Pin 3 on Header 8, referred to by the Beaglebone System Reference Manual as  GPIO1_6 on Expansion B, but known to Linux as gpmc_ad6.

Say what?

Well, the “Pin 3 on Header 8″ part is simple enough. If you look at the Beaglebone, Port 8 is one of the 2 double-rows of female headers,  labeled P8 on the circuit board.  (In the figure below, taken from the System Reference Manual, Port 8 is labeled “Expansion B).

Beaglebone, showing pin P8_3 and Ground

At either end of the headers you’ll find the rows marked 1 and 2, and 45 and 46.  So, Pin 3 is the 2^nd row from the top, left column, as indicated in the figure.

The other names for the pin, along with the locations of the ground pins, can be found in the System Reference Manual.  If you look at Table 8 – “Expansion Header P8 Pinout” (a portion of which is shown below), you’ll find the pins labeled with their default usages in the Angstrom Linux demo image (the one that comes packaged with the beaglebone on a microSD card).

Default pin settings in Anstrom Linux

Clearly, Ground is the top row of Port 8, so that’s the last bit of information we need to know to hook up the circuit.  But we’ll need to understand that part about pins having “default usages” – more on that later.

Cloud9 IDE and bonescript

When you boot up the Beaglebone with the microSD card from the box, the Cloud9 IDE is automatically started.  Once you have an  IP address assigned to the Beaglebone (either through its Ethernet port connection or through USB networking), you can load the Cloud9 IDE on your PC browser at http:<beaglebone address>:3000

(I haven’t tried to use USB networking myself, but the procedure for doing so is explained in Getting started with your new BeagleBone.)

The Cloud9 IDE interface is pretty intuitive.  You select a source code file in the left pane, edit it in a tab in the right pane, and select Debug or Run from the toolbar to execute the code.

Cloud9 IDE

If you are coming at this from the Arduino, you will notice 3 significant differences from the Arduino IDE:

• You don’t upload your code to the board.  The Beaglebone is more like a PC than an Arduino – the code is stored on its file system, and you just run it.
• You can debug your code.  Not Arduino’s form of debugging – print statements to the console (though you can do that too) — but real debugging, as in breakpoints, watch variables, step-by-step execution.
• The coding language is Javascript, not C.  Specifically, it’s node.js, which is Javascript optimized for running on a server, rather than in a browser, by way of some extra libraries.  The “server” in this case is the little old Beaglebone.  As you might imagine, node.js is not the fastest environment for running code on the Beaglebone, but for LED blinking and many other types of prototyping, it’s fast enough.

Despite the differences, Arduino coders should find the transition to Cloud9 and bonescript to be quite easy.  The blinkled.js code looks very much like Arduino code.  That’s no coincidence: the README for the bonescript project, which you can find on its github page here,  says that the goal is “to have something that provides most of the Arduino functions and is generally usable by Summer 2012″.

var ledPin = bone.P8_3;
var ledPin2 = bone.USR3;

setup = function() {
    pinMode(ledPin, OUTPUT);
    pinMode(ledPin2, OUTPUT);
};

loop = function() {
    digitalWrite(ledPin, HIGH);
    digitalWrite(ledPin2, HIGH);
    delay(1000);
    digitalWrite(ledPin, LOW);
    digitalWrite(ledPin2, LOW);
    delay(1000);
};

This code from blinkled.js does the following:

• defines 2 pins, P8_3 (Port 8, pin 3) and USR3 (which is one of the built-in LEDS on the board, next to the Ethernet port)
• sets these pins for Output
• loops, turning these 2 LEDs on and off, once per second, ad infinitum

If you’ve attached an LED to pin 3, and you run blinkled.js in Cloud9, you should find it blinking happily away. (There is a dropdown button next to the Run and Debug toolbar icons in Cloud9 to let you choose the file).

It’s that simple.

I lie.

There is actually considerable complexity behind the code which creates those 2 pin objects, bone.P8_3 and bone.USR3.  The complicated code is in one of the 2 “library” files in the bonescript folder, index.js.

Pins and Muxing

Before we dive into the index.js code, let’s return to the Beaglebone System Reference Manual.  As you’ll recall, I earlier referred to the “default usage” on Port 8 Pin 3.  It has other usages, and the process of telling the board how you want to use that pin is called muxing (short for multiplexing).

The available usages of the pin, and all of the others on the Port 8, are listed in Tables 9 (the first part of which is shown below) and 10 of the System Reference Manual.  Each pin can have up to 8 uses, “mux mode” 0 through 7.  The default setting in the Angstrom Image is generally, but not always, mode 7.  Table 8 is the definitive guide to Angstrom’s default mux setting for each pin.

Pin Mux settings from the System Reference Manual

(Incidentally, other Linux distributions for the Beaglebone, such as Ubuntu, will have their own default mux settings.  The board doesn’t control the mux settings, the software does.  The developers of the Angstrom Linux image on the SD card selected their defaults to be the most commonly used ones by hobbyists, providing a good out-of-the-box experience.)

If you Google “Beagleboard mux”, you’ll find lots of pages explaining the various ways in which you can change the mux setting.  Many of them refer to U-Boot configuration and kernel modules, which are complicated and, fortunately, no longer necessary.  There has been a lot of work done recently at making the mux settings accessible to “userland” code by way of the Linux file system.  If you (or your code) wants to read or change the mux setting, it just needs to read or write to a file.

Great.  So which file?

This information used to be hard to come by, scattering amongst various forum and blog posts, or hidden inside code in U-Boot or the linux kernel.  However, if you’re learning to code with the Beaglebone, I think the best way to learn the mux file system is to look at how bonescript handles it.

Muxing the bonescript way

So, back to the Cloud9 IDE.  This time, open up the index.js file in the bonescript directory.  (Note that this part of bonescript is in active development and is likely to change by the time you read this, but the underlying file system will still be the same)

A little ways into the code (line 39 in the current release, bonescript 1.0-r10), you’ll find the following:

var gpio0 = 0;
var gpio1 = gpio0+32;
var gpio2 = gpio1+32;
var gpio3 = gpio2+32;

bone = exports.bone =
{
    P8_1: { name: "DGND" },
    P8_2: { name: "DGND" },
    P8_3: { name: "GPIO1_6", gpio: gpio1+6, mux: "gpmc_ad6" },
    P8_4: { name: "GPIO1_7", gpio: gpio1+7, mux: "gpmc_ad7" },
    P8_5: { name: "GPIO1_2", gpio: gpio1+2, mux: "gpmc_ad2" },
    P8_6: { name: "GPIO1_3", gpio: gpio1+3, mux: "gpmc_ad3" },
    P8_7: { name: "TIMER4", gpio: gpio2+2, mux: "gpmc_advn_ale" },
    P8_8: { name: "TIMER7", gpio: gpio2+3, mux: "gpmc_oen_ren" },
. . .
    USR0: { name: "USR0", gpio: gpio1+21, led: "usr0", mux: "gpmc_a5" },
    USR1: { name: "USR1", gpio: gpio1+22, led: "usr1", mux: "gpmc_a6" },
    USR2: { name: "USR2", gpio: gpio1+23, led: "usr2", mux: "gpmc_a7" },
    USR3: { name: "USR3", gpio: gpio1+24, led: "usr3", mux: "gpmc_a8" }

This table contains 3 important pieces of information:

1. the label that bonescript uses to identify the pins (P8_1, P8_2, etc)
2. the GPIO pin number (P8_3 is gpio1+6, or 38)
3. the mux label (P8_3 is gpmc_ad6)

The GPIO pin number tells you the directory name you will use to read from or write to the pin.  The mux label tells you the filename you will use to read or write the mux setting.

Let’s start with the mux setting.  This is handled by some code just below the exports.bone table:

if(pin.mux) {
 try {
    var muxfile = fs.openSync(
      "/sys/kernel/debug/omap_mux/" + pin.mux, "w"
      );
    fs.writeSync(muxfile, "7", null);
 } catch(ex3) {
    console.log("" + ex3);
    console.log("Unable to configure pinmux for: " + pin.name +
      " (" + pin.mux + ")");
. . .

There is more to the error handling, but the important part is the reference to “/sys/kernel/debug/omap_mux/” + pin.mux.  This is the file used to read and write the mux setting.  In the case of Port 8 Pin 3, the full path is /sys/kernel/debug/omap_mux/gpmc_ad6.

This is the part that is newcomers to pin muxing often trip over: if you want to set a pin to be a GPIO pin, your code needs to reference a filename that appears to be intended for a different usage of the pin (gpmc_ad6).  The mux filename isn’t taken from its intended usage, it’s taken from its mode 0 usage.  The mode 0 usage for Port 8 Pin 3 is gpmc_ad6 – we know this from the table in the bonescript code, but the definitive reference for the Beaglebone pin mux modes are Tables 9 and 11 of the Beaglebone System Reference Manual.

Having opened the file (fs.openSync), the bonescript code writes a 7 to it (fs.writeSync).  This, as you probably guessed, sets the mux mode to 7, the GPIO mode for that pin.  The process of setting mux modes isn’t quite as simple as “write the  mode to a file”:  the mux mode setting also affects other aspects of the pin, such as whether it can be used for input.  The bonescript code shown above is actually correct only for setting pins to be GPIO Output pins.  The bonescript project will soon support other mux mode settings.

Blinking LEDs the bonescript way

Having set the pin to be in digital output mode, the next task is to set it high or low.  You’ll recall that this was handled in blinkled.js by calling a function called digitalWrite, same as the Arduino does.

The source code for digitalWrite can be found further down in index.js:

digitalWrite = exports.digitalWrite = function(pin, value)
{
    fs.writeFileSync(gpio[pin.gpio].path, "" + value);
};

OK, so it’s writing a value to a file, either HIGH (defined as 1) or LOW (0).  But which file?

The code to set that “path” property also in index.js, just below where the pin mux setting was done:

try {
    try {
        fs.writeFileSync("/sys/class/gpio/export", "" + n);
    } catch(ex2) {
        // TODO: If the file is already exported, can we know who did
        // did it so that we aren't opening it twice?  In general, this
        // shouldn't be an error until we have some better resource
        // management.
        //console.log(ex2);
        //console.log("Unable to export gpio: " + n);
    }
    fs.writeFileSync("/sys/class/gpio/gpio" + n + "/direction",
        mode);
    gpio[n].path = "/sys/class/gpio/gpio" + n + "/value";
    return(true);
} catch(ex) {
    // Perhaps we couldn't open it because it was allocated as an LED
    if(pin.led) {
        fs.writeFileSync(
            "/sys/class/leds/beaglebone::" + pin.led + "/trigger",
            "gpio");
        if(mode == OUTPUT) {
            gpio[n].path =
                "/sys/class/leds/beaglebone::" + pin.led +
                "/brightness";
        } else {
            gpio[n].path =
                "/sys/class/leds/beaglebone::" + pin.led +
                "/gpio";
        }
        return(true);
    }
}

The first thing it does is try to write to a file named “export”, specifically , “/sys/class/gpio/export”.  The value written is the GPIO pin number, which you’ll recall was specified in the table earlier in index.js.  For Port 8 Pin 3, it was defined as gpio1+6, or 38.

The export file is a funky weird-ass file: when you write a pin number to it, it (well, the kernel process which monitors that file) creates a directory and a bunch of files that provide interfaces for accessing the pin.  The directory is created at /sys/class/gpio/export/gpioNN. In the case of Port 8 Pin 3, that’s /sys/class/gpio/export/gpio38.

For the purpose of using a GPIO pin, the two most important files in the gpioNN directory are named direction and value.

The direction file determines whether the pin is going to be used for reading or writing.  As mentioned earlier, you can’t use a pin for reading without setting flipping a bit in the mux mode setting, so our only choice at this point is output.   The bonescript code writes the value “out” to the direction file (i.e. to /sys/class/gpio/export/gpio38/direction).

The code then saves the path to the value file (i.e. /sys/class/gpio/export/gpio38/) in the pin’s path property.  This, then, explains what the code in the digitalWrite function is doing:

fs.writeFileSync(gpio[pin.gpio].path, "" + value);

It writes a 1 to the value file (i.e. /sys/class/gpio/export/gpio38/value) to set the pin high (and the turn the LED on) and a 0 to set it low (and the LED off).

But wait, there’s more!

The bonescript code shown above also has an error handler.

Writing to the export file also tells the kernel that you intend to use that pin.  If the pin is already being used by something more important than your dinky userland application (i.e. the kernel or one of its pals), it will helpfully toss an error at you.

Generally, when your attempt to export a pin fails, you have no option but to go away and sulk.  Something is using that pin, and it might be a hardware driver that won’t be giving it up any time soon (such as the SD card socket, or an LCD).  There are ways of tracking down what’s using it, but I won’t cover them in this post.

One possibility is that the GPIO pin is being used by one of the built-in “user LEDs”, USR0 – USR3.  As you can see from the final entries in the bonescript mux table, those LEDs are connected to GPIOs 53-56.  So, the bonescript code checks to see if you’re trying to write to those GPIOs – if so, it assumes you want to access the user LEDs, and redirects you to their address.

These user LEDs are accessed through a different part of the file system than the GPIOs – technically, they are part of the Beaglebone board, not part of the CPU.  So, they are accessed through the /sys/class/leds/beaglebone directory, not /sys/class/gpio.

The Beagleboard (the ‘bone’s big brother) also has user LEDs, and they are accessed in basically the same way there, so you can find a good explanation of how to write to them in some of the articles written for the Beagleboard.  The command line interface is explained in this article on eLinux.  For C language code that interfaces to the user LEDs, see this sample from Jan Axelson’s USB Embedded Hosts: The Developer’s Guide.  (This, by the way, is one of the few books currently available that specifically covers Beagleboard programming).

Blinking LEDs the Python Way

The directories and paths used by the bonescript code can also be used from other programming languages: pretty much anything that runs on Linux can read and write from the file system.

Here is my own humble contribution, a snippet of Python code which blinks the USR2, USR3 and Port 8 Pin 3 ten times.  It has the advantage of distilling the bonescript code down to the bare bones needed to access that particular pin.

[Updated Jan 22 - added support for User LEDs 2 and 3, handle case where pin already exported, and simplified  the file I/O syntax:]

import time

# put Port 8 Pin 3 into mode 7 (GPIO)
open('/sys/kernel/debug/omap_mux/gpmc_ad6', 'wb').write("%X" % 7)

try:
   # check to see if the pin is already exported
   open('/sys/class/gpio/gpio38/direction').read()
except:
   # it isn't, so export it
   print("exporting GPIO 38")
   open('/sys/class/gpio/export', 'w').write('38')

# set Port 8 Pin 3 for output
open('/sys/class/gpio/gpio38/direction', 'w').write('out')
# we will assume that USR1 and USR 2 are already configured as LEDs

for i in range(10):
   # turn on USR1 and external LED
   open('/sys/class/gpio/gpio38/value', 'w').write("1")
   open("/sys/devices/platform/leds-gpio/leds/beaglebone::usr1/brightness", 'w').write("1")
   # turn off USR2
   open("/sys/devices/platform/leds-gpio/leds/beaglebone::usr2/brightness", 'w').write("0")

   time.sleep(1)

   # turn off USR1 and external LED
   open('/sys/class/gpio/gpio38/value', 'w').write("0")
   open("/sys/devices/platform/leds-gpio/leds/beaglebone::usr1/brightness", 'w').write("0")
   # turn on USR2
   open("/sys/devices/platform/leds-gpio/leds/beaglebone::usr2/brightness", 'w').write("1")

   time.sleep(1)

# cleanup - remove GPIO38 folder from file system
open('/sys/class/gpio/unexport', 'w').write('38')

The Python code also demonstrates something that the bonescript code doesn’t handle.  To be a nice little userland app and clean up after yourself, you can write the GPIO pin number to /sys/class/gpio/unexport.  This doesn’t “release” the GPIO to other apps – you never had it locked for your exclusive use in the first place – so unexport’ing is just a convention to make the gpio file system is a little more manageable.

Since python is already installed on the Angstrom image, you can download the file and run it as follows:

wget http://gigamega-micro.googlecode.com/files/gpiotester.py
python gpiotester.py

Wrapping Up

If you think that’s a lot of work in order to blink an LED, you should have tried it a few years ago, when the Beagleboard was first released.

Back in the day, to do anything with a GPIO pin you pretty much had to be an embedded Linux guru, or beg for crumbs of knowledge from the guru’s table.  (Oh, and good luck getting your LED to light up on 1.8V.  You kids have it easy these days!)

The Beaglebone arrives at a time when the embedded Linux device market is growing by leaps and bounds, both in terms or hardware and development tools.  As a hobbyist (or a budding professional), this is a good time to hop on board the platform.

Incidentally, while the Beaglebone platform is geared for hobbyists and prototypers, the chip it uses is very much targeted at professionals.  The Texas Instruments Sitara AM3359 is a descendent of (and largely code compatible with) the storied OMAP chip family, which powers a lot of consumer gadgets: many of Nokia’s high-end smartphones like the E90, Motorola’s Droid line of Android smartphones, the Nook Color and the Kindle Fire.

If you can get to a root command line on one of those devices, you can probably set a GPIO high and low using the same code as above (and if you’re lucky, cause an inglorious crash).  How cool is that?

Nimbits itself was recently updated to version 3.3, and the its web site has been reorganized to make all of the documentation easy to find.  If you are new to Nimbits, or haven’t seen it for a while, check it out!

In this article, I’d like to explore parts of the Javascript code in the gadget, and describe how they interface with the Pachube API.

Viewing the Source Code

As with any iGoogle Gadget, the source code can be viewed by clicking the “View source” link in the bottom right of the gadget’s iGoogle Gadget Directory page – my gadget’s page is here.

However, this will only show you the XML definition for the gadget, since I’ve put the Javascript in a separate file.  My gadget has 2 different views defined in the XML file, normal and “canvas” (i.e. maximized), as you can see at the bottom of the XML file.  Using a separate Javascript file allows the 2 views to use the same code.  Here are direct links to the XML and Javascript files.

• XML: http://www.gigamegablog.com/gadgets/gm-pachube.xml
• Javascript: http://www.gigamegablog.com/gadgets/gm-pachube.js.  (It’s OK to click on it.   Since this Javascript can’t execute outside of Google’s gadget container, your browser will just display the source code instead of running it.  Or, you can just right-click the link and save it as a file, you big baby. )

The Pachube API

Pachube’s API documentation is quite good overall, but it can be a little difficult to find the details for a particular feature.

The graphing API I’m using, for example, is in the “Read datastream – GET  v2/feeds/<feed_id>/datastreams/<datastream_id>” section.  It’s just barely there, actually – a passing reference to the all-important PNG parameter, along with a table of parameters you can pass to the API.  (A better way to learn the graphing API is to use Pachube’s Graph Builder – see the next section).

There are a lot of options for setting the time range of a graph, including some that my gadget doesn’t support, such as a specific end date.  These options are described in the “Historical Queries” section tucked away at the end of that documentation page.

To get a list of datastreams in a feed, I use the “Read feed” API, described here.  As you can see from the JSON example in that section, this is a very handy API that returns a lot of information about the feed and each datastream.

An API key must be passed with most API calls, even when accessing Public feeds.     The mechanism for passing the key is described on a separate page, the API Overview.  That page also describes the timezone parameter.  You’ll find more information on the API key and time zones later in this post.

Pachube Graph Builder

The best way to learn the API syntax for generating a graph is Pachube’s Graph Builder.  To display it, go to your Feed’s page on Pachube’s web site, then click on the gear icon in the lower right of the graph for a datastream, as shown below.

Launching the Graph Builder

The Graph Builder page allows you to interactively try out various graph parameters, then copy a URL that calls the API to generate that graph.

Pachube Graph Builder

Pachube Graph API

The URL that you’ll get from the Graph Builder will look something like this:

https://api.pachube.com/v2/feeds/37080/datastreams/JadePlant.png?
    width=500&height=300&colour=%238024f1&duration=3hours 
   &title=Jade Plant Moisture&show_axis_labels=true &detailed_grid=true
   &timezone=Eastern Time (US & Canada)

All Pachube API URLs have the same basic layout, following the REST standard:

• https://api.pachube.com/v2 – All API URLs start with this.  (You can use the non-secure http:// prefix if your coding platform doesn’t support https, such as an Arduino.)
  The V2 refers to Version 2 of the API.  Version 1, which uses a different syntax, is “deprecated” (i.e. still supported, but use is discouraged).  Some of the examples you’ll find on the Web use the V1 API: if there is one /v2 in the URL, it’s a version 1 call.
• feeds/37080  - All API URLs for a specific feed will identify the feed # this way.
• Datastreams/JadePlant.png – All API URLs which deal with a specific datastream will identify it using its ID field (in this case, “JadePlant”).  The “.png” part is what tells the API that you want a graph.  If you replace that with “.xml”, you’ll get back XML data about your datastream instead.
• ?<parms> – The parameters that follow the question mark are specific to the API call you are making.
  Pachube’s API is pretty easy-going about these parameters: if you pass ones that don’t apply to the API call, or even ones that don’t exist at all, the API will just ignore them.
  If you’re familiar with URLs, you might have noticed that the above URL example (copied from the Graph Builder) contains an invalid parameter: the &timezone parameter contains spaces and an ampersand that need to be properly “escaped” when including in a URL.  The API doesn’t tell you this, it just ignores the timezone parameter.  (And, as a result, the X axis of the graph shows UTC times, the default, rather than EST times).

API Keys

I have to admit that I don’t have a firm grasp on which APIs and feeds require authentication.  I’m tempted to just write “always pass an API key – it can’t hurt”.  But that’s way too simple for this blog, so here’s my best guess explanation of how it works…

If you tried pasting URLs from the Graph Builder into your browser, you probably noticed that no login was required, and no API key is being passed.  As far as I know, an API key is never required for a graph of a public datastream.

However, the other API used by the gadget, which retrieves a list of datastreams for a specified feed, does require authentication, even if the feed is public.  If you have a neglected browser from which you’ve never logged into Pachube (that blue E icon will probably do), try accessing  the following URL

http://api.pachube.com/v2/feeds/37080.json? 
   timezone=Eastern%20Time%20%28US%20%26%20Canada%29

You should get a pop-up authentication prompt.

Private feeds always require authentication, even for a graph.  For example, the following should definitely display an authentication prompt, unless you are a nefarious hacker.

http://api.pachube.com/v2/feeds/39460.json? 
   timezone=Eastern%20Time%20%28US%20%26%20Canada%29

If your application were to try doing the same thing, it would get a “404 – Authentication Error”.  When trying to graph a private datastream without an API key, the application gets back a nice “Server error” graphic, instead

To avoid that, the code needs to pass an API key.  The gadget doesn’t ask you to fill in an API key of your own: instead, I’ve hard-coded one that has read access to public feeds, but update access to nothing (including my own feeds).

The key is passed in an HTML header field, as specified in the Pachube API documentation.  Various programming languages have various methods for setting the contents of the header field – the Google Gadgets approach is described in their documentation, and shown below:

var params = {};
params[gadgets.io.RequestParameters.HEADERS] = {
    "X-PachubeApiKey": "<your API key>"
};

Incidentally, if you’re using the gadget to graph datastreams in a private feed, you might wonder how my API key has access to your private feed.  Relax, it doesn’t.  Through some kind of under-the-covers-cached-browser-authentication-cookie-machination (my words, not Pachube’s), the gadget is allowed to access your private feed on a browser in which you’ve previously logged into Pachube.  I’ve noticed intermittent “authentication failed” errors when using the gadget with my own private feed, presumably when my cached authentication expires (or the cookie-machinations go awry).

Time Zones

According to the API documentation on time zones :

There are two places where you can specify the time zone: in your user profile on the website and via a parameter in an API request.

I think the intention is that your user profile’s timezone setting is automatically used by the API unless overridden, but it currently doesn’t work that way for graphs.  Perhaps that’s a bug that will be fixed at some point, but for now you have to pass the timezone parameter to the graph API call, or the X axis will display UTC times.

I had originally planned to have a “UTC offset” field in the gadget settings, where you would fill in the # of hours difference between your time zone and UTC: for example, -5 for Eastern Standard Time.

However, when testing I noticed something odd: timezone=-5 gave me UTC minus 5 hours, as expected, but timezone=-4 gave me UTC minus 3 hours.

As described in this post in the Pachube Community forum, if you set the timezone parameter to a numeric value, the API will select the first matching entry in Pachube’s (rather eclectic) list of geographic time zone settings.

For example, timezone=-4 is the same as specifying “Atlantic Time (Canada), and timezone=-5 is the same as specifying “Bogota”.  Rather than being dependent on the local customs of Atlanteans and Bogatians, I gave in and included the full list of geographic timezones in the the gadget’s Edit Settings page.  (If you have no idea which one on those time zones you are in, ask Pachube, not me.)

Yes, Virginia, there is a Nuku'alofa

As shown in the Graph Builder section above , another tricky thing to consider when working with timezones is proper encoding of non-URL characters, like spaces and ampersands.  Javascript makes this easy: the gadget’s code for handling the conversion is simply:

// replace And with & (Google's Gadget API hates ampersands)
timeZone = timeZone.replace(" and ", " & ");
// use escape function to encode spaces and ampersands in URL
urlSuffix = "timezone=" + escape(timeZone);

One last pothole to avoid: Google’s Gadget API seems to have a particular distaste for ampersands, however they’re encoded: the dropdown list of timezones wouldn’t display until I took them out.

Getting the List of Datastreams in a Feed

Pachube makes it easy to get the list of datastreams in a feed: just one API call returns a variety of properties, including the timestamp and value of the last datapoint received by each datastream.

The Javascript code for doing this is shown below, most of which is from the gadget’s getDatastreamStatus function:

var urlPrefix = "https://api.pachube.com/v2/";
var urlSuffix = "timezone=" + escape(timeZone);
var url = urlPrefix + "feeds/" + feedID + ".json" + "?" + urlSuffix;
var params = {};

params[gadgets.io.RequestParameters.CONTENT_TYPE] = gadgets.io.ContentType.JSON;
// NOTE - this API key has READ access only
params[gadgets.io.RequestParameters.HEADERS] = {
    "X-PachubeApiKey": "nBGUIT63Onke5-sfOoARTvKvkwl45b85wjz5PKDIZkc"

// refresh data if longer than 5 minutes since last request
makeCachedRequest(url, getDatastreamsResponse, params, 300);

Yes, that’s an actual, genuine API key that I’ve published – my own, in fact.  As mentioned earlier, this one is defanged, limited to read-only access to public feeds.

The “.json” in the URL specifies that the data is to be returned in JSON format: other options are XML and CSV.  Your choice of format depends primarily on the coding platform you’re using.  Google’s Gadget API supports both XML and JSON, but I went with JSON because it is more light-weight (i.e. faster), a consideration if you’re working with a lot of datastreams.  The CSV option is intended for platforms that don’t support XML or JSON, such as an Arduino.

The code for makeCachedRequest is taken directly from Google’s API documentation.  The code allows you to override Google’s default caching algorithm for gadget data, which retrieves updated data from the source (i.e. Pachube in this case) every 60 minutes, regardless of how often the user clicks the browser refresh button.  I’m overriding the interval to be every 5 minutes.

Note that the gadget doesn’t automatically update every 5 minutes: the update only occurs when the user first opens the web page containing the gadget, or clicks the refresh button, or when iGoogle refreshes the gadget.  (iGoogle refreshes each gadget once an hour, if the iGoogle page is in the foreground tab).  As far as I know, your gadget code has no way of forcing a refresh to occur.

The response to the API request is returned to the callback function passed as a parameter to makeCachedRequest, a simplified version of which is:

function getDatastreamsResponse(obj)
{
  if (obj.data === null || typeof obj.data === "undefined")
  {
    // something’s wrong – check the other obj properties, like obj.error and obj.rc
    . . .
  }

  jsonData = obj.data;

  if (!jsonData.datastreams)
  {
    displayErrorMsg("No datastreams found in feed");
    return;
  }

  datastreamList = jsonData.datastreams;
  feedTitle = jsonData.title;
  feedStatus = jsonData.status;
  feedDesc = jsonData.description;

The meat of the response is contained in the .data property of the parm passed to the callback function.  If that property is empty, then an error occurred.  Generally the error information is found in obj.error and/or obj.rc.  The former contains an error message as a raw JSON text string, and the latter contains one of the standard HTTP return codes as listed in Pachube’s documentation.

The data object is bristling with useful information about the feed and datastream.  As shown in the code, I’m getting the feed’s title, status and description from there, as well as a list of datastreams, each of which is itself a JSON object with properties.  The screenshot below shows some of the other properties in this data object as seen in the Google Chrome Javascript console.  For a full list of all of the available properties, see the example in Pachube’s Get Feeds documentation.

Getting the Datastream Graphs

The Javascript which handles the calls to the Graph API, displayGraphs(), basically just pieces together the URL for the Graph API,  as explained in the “Pachube Graph API” section above.

There are a couple of issues handled here that I didn’t describe earlier.

Unbeknownst to the user (wouldn’t want to worry their pretty little heads, would we?), the code scales down the graph if necessary to fit within Pachube’s limit 300K pixels.

var MAX_GRAPH_SIZE = 300000;  // Pachube's maximum graph size, in pixels
. . .
var defaultWidth = 320; // in normal gadget mode, 320 is the recommended width
if (graphHeight * graphWidth > MAX_GRAPH_SIZE)
{
  // set to maximum size that will fit within Pachube's limit
  if (graphWidth > defaultWidth)
  {
    graphWidth = defaultWidth;
  }

  graphHeight = Math.floor(MAX_GRAPH_SIZE / graphWidth);
}

This is done somewhat crudely, basically yanking the graph width back to 320 pixels, then yanking the height down as needed to fit within the 300K.

Also admittedly quite crude is the support for increasing the graph size when the gadget is maximized.

// in canvas mode, double the graph size
if (gadgets.views.getCurrentView().getName() === "CANVAS")
{
  graphHeight *= 2;
  graphWidth *= 2;
  defaultWidth *= 2;
}

I’m a little surprised that most gadgets ignore the maximized setting altogether, just displaying the same content in the same cramped size.  I think the problem is one of “discoverability”, to use a Googley term: the feature is disabled by default, and Google’s documentation on the feature confusingly insists on referring to it as “canvas” mode.

As shown in the API documentation, enabling “canvas” mode is just a matter of pasting some boilerplate XML settings into your gadget, like so:

<Content type="html" view="home">
<![CDATA[
 <div id="content_div"></div>
. . .
<script src="http://www.gigamegablog.com/gadgets/gm-pachube.js" type="text/javascript"></script>
]]>
</Content>
<Content type="html" view="canvas">
<![CDATA[
 <div id="content_div"></div>
. . .
<script src="http://www.gigamegablog.com/gadgets/gm-pachube.js" type="text/javascript"></script>
]]>
</Content>

Perhaps Google would prefer that your gadget take on an entirely new form when maximized, since the XML allows for a separate block of Javascript for canvas mode (the “content_div” section).  I wasn’t that ambitious, so instead I use a script tag to feed the same Javascript code into both the normal and canvas mode XML definitions, then changed the code to check which mode it’s in, as shown in the earlier code sample.

Wrapping It Up

If you’ve read the Programmer’s Show and Tell for my Nimbits gadget, you’ll notice that this time I’ve done much  less ranting about how frustrating Google’s Gadget API is to work with.  The API hasn’t changed, but once you know where the potholes are, the drive is a lot smoother. If you intend to write your own gadget I’d suggest you read my earlier rant/article first, since it points out a lot of those potholes.

I’m still dubious about the future of Google gadgets given Google’s newfound emphasis on monetization and focusing on core products.   The iGoogle Developer Blog hasn’t had any new posts since a May post asking “Did you know we are continuously adding new features” ?  (In fairness, they were referring to the Gadget Dashboard – they stopped adding new features to the gadget API long before May).

Methinks the best browser-based platform for making use of the Pachube API lies elsewhere, perhaps a Google App Engine project like this one. .  The App Engine would give your code a lot more elbow room, and there is so much more to the Pachube API than what I’ve covered here.

I like Pachube — it’s reliable, well-designed, well-documented, and very much open to hobbyists.  They recently did away with paid plans and made all aspects of the site and service free, so I’d encourage you to give them a try. If you need help at getting your data into Pachube, see their Quick Start guide and Tutorials. My article on connecting Tweet-A-Watt to Pachube also contains some tips on getting started, and my article on the FEZ XBee Sensor shows an easy way to post data to Pachube from Python.

Pachube’s site provides a page for each of your feeds, showing a graph of the last 24 hours of readings from each of your datastreams. That’s good if you remember to go look at it, but I’d prefer that my graphs come to me, via a gadget on my iGoogle home page.

Pachube released a Google Gadget some time ago, and I’ve been using it for some time. It works well, but there are a few extra features that I really wanted:
- larger graphs, and the ability to resize the gadget container to see them
- a time axis in local time, rather than UTC
- the ability the set the timespan: 12 hours, 1 week, 1 month, etc.
- a Maximized view that shows the graphs in humongous GigaMegaSize

So, I’ve created a Google Gadget which adds these features, plus a few more.

You can grab the gadget from the Google Gadgets directory.

Settings

Here is a description of the various options available on the Edit Settings dialog.

Feed #: Numeric Pachube Feed ID

Datastreams: To graph all datastreams in the feed, leave this field blank. Otherwise, enter the datastream ID and click the Add button. For example, to add the 2 datastreams below…

…type Cactus, click Add, type CatGrass, click Add. The field should then look like this:

Time Span, Time Span Units – This determines what period of time will be graphed. As shown in the screenshot, the default is 24 hours. Time Span Units can be set to hours, days, weeks or months.

Show Status. If true, each graph will be preceded by a line of text giving the Datastream ID, the last reading and the date and time it was received, as shown in this screenshot.

Detailed Grid. If true, vertical and horizontal grid lines will be displayed, as in the 2 graphs above. If false, only a vertical grid separating each time span unit is displayed, as in the following (GigaMegaSized) graph.

Axis Labels: If true, numeric labels are added to the vertical and horizontal axis indicating the range of values. Note that, depending on the time span, these labels tend to overlap in a smaller graph (and I don’t have any control over the format of the labels). If false, the only labels are the minimum and maximum value of the vertical axis.

Graph Width and Graph Height: Determines the size of each graph, in pixels. When the gadget is maximized, both sizes are automatically doubled. These values are required, and are initially set to a width of 280 and a height of 100.

Gadget Height: Determines the height of the overall gadget, in pixels. If not specified, a default of 200 is used. (The gadget width can’t be set by the code, and varies depending on the browser type and the resolution of the monitor.)

Time zone: Your local time zone. The default is UTC (aka Greenwich Mean Time). This setting affects the timestamp in the Status field, as well as the labels on the horizontal axis of the grid. The rather eclectic collection of names in the dropdown lists was Pachube’s idea, not mine. (Look here if you don’t believe me!)

Wrapping Up

I’d welcome any bug reports or feature requests.

iGoogle’s Gadget framework does a lot of caching sleight-of-hand, so if you find that a change of settings doesn’t appear in the gadget, try refreshing the page. As for features , there are some things I can’t add because they aren’t supported by Pachube’s API yet (multiple datastreams in a graph, for example), but otherwise I’m open to suggestions.

For developers, I plan to write an article in the near future explaining the gadget’s use of the Pachube API.

Like its predecessor, the XBee Sensor is a .Net Micro Framework device, which uses an XBee wireless transceiver to post data to the Internet by way of a Python middleman.

I’m structuring the code to be more flexible, supporting a variety of sensors and logging to a variety of databases.  Personally, I’m using it to measure plant soil moisture, light levels, temperature and humidity, as described in the “Hardware Configuration” section below.

If you decide to use a different mix of sensors (as I’m sure you will), you’ll need to make a small change to the C# code, as shown in the Programmer’s Show and Tell section below.  I’m aiming for a more generic device configuration, where all  sensor devices run exactly the same code, and all sensor configuration is done through Python.

Since a new project gives me an excuse to start playing with new hardware, I’ve also switched from using the Netduino to a rival Micro Net Framework product, the FEZ Panda.

I’m not abandoning the Netduino: my own Netduino continues to perform its duties as a Plant Light Controller, and I’ll do my best to continue to support the Netduino in the XBee Sensor source code.  It’s relatively easy to switch the code between a FEZ and Netduino target.  I’ll point the code changes that are required in the Programmer’s Show and Tell section .

Also, the Python code used in this article is compatible with the Netduino Plant Light controller code from Watching the Watcher, so you can have both a Netduino Plant Light Controller and an XBee sensor reporting data to the Python code if you like, or even just the Netduino.

The FEZ Panda

The FEZ Panda is one of a line of products from GHI Electronics: Fez Mini, Panda, Panda II and Domino.  The Fez Mini is similar to the Netduino Mini — both mimic the form factor and pinout of the Basic Stamp — while the Pandas are similar to the Netduino.  The Domino is a more powerful beast, using a chipset that supports USB Host.

FEZ Panda II

Compared to the Netduino, the Pandas (both versions I and II) have a few advantages that will come in handy when acting as a remote sensor.

1. More pins: 54 digital I/Os, as opposed to the Netduino’s (and Arduino’s) 13, and 3 COM ports to the Netduino’s 2.  More analog pins would have been even better, but the Panda only has 6, same as the other platforms.

2. Native code.  This one is huge: the .Net code can invoke “unmanaged” native code, which has full access to the microprocessor’s capabilities, including microsecond-level timing.  This eliminates the Micro Net Framework’s biggest disadvantage to the Arduino.

3. Ability to store data in flash memory.  The FEZ firmware includes a class that allows .Net code read/write access to 4K of flash memory, analogous to the Arduino’s Flash library.

4. More documentation!  This one might not excite you as much as it does me (whee!), but GHI’s TinyCLR site offers a ton of documentation covering pretty much every feature of the FEZ platform and the underlying Net Micro Framework.  I’d particularly like to point out the Beginner’s Guide to .Net Micro Framework, which I feel is still the best publication of any type covering the MNF, and much of which is also applicable to the Netduino.  A Tutorials section was recently added to the TinyCLR site, which has rapidly expanded to cover a lot of very useful features, again many of them applicable to the Netduino.  (Both the Beginners Guide and the Tutorials section can be found on the TinyCLR Support page).

Back in the Netduino: A Little Help From Its Friends article, I used an Arduino as a helper to the Netduino.  It handled a couple of actions that the Netduino (and Micro Net Framework) couldn’t do: read data from the DHT11 temperature/humidity sensor and saving settings to Flash memory.  The FEZ Panda can actually do both these things itself.    Smarter than the average bear!

By the way, the difference between the Panda and Panda II is that the latter contains a micro SD card socket, and adds female headers for most of the additional digital pins, which are repositioned to be accessible even when an Arduino shield is in place.  I’m using Panda Is (currently being sold off by GHI at the ridiculously low price of $15), but everything that I’m doing in this project will also work on a Panda II.   The code will work almost unchanged – all you have to do is remove the Project Reference to FEZPanda_… and replace it with FezPandaII_…, as shown in the screenshot.

Selecting Panda II Assembly

Hardware Configuration

At this point, the XBee Sensor project is geared towards supporting analog sensors plus one particular digital sensor, the DHT11/DHT22 temperature and humidity sensor. I’m using the analog ports as moisture and light sensors, but any analog sensor that can run on 3.3V should work without code changes.

The XBee Sensors are intended to be used without displays, but when first configuring them it’s handy to connect an LCD to see what they’re doing (or failing to do). So, I added an optional “Debug LCD”, which displays the current sensor readings, as well as any Exceptions or other error messages.

I’ve written the code so that, if the LCD isn’t connected, it shrugs and keeps going.  However, if your device seems to behave erratically when the LCD is disconnected, you can disable use of the LCD by removing the DEBUG_LCD compiler flag (as shown in the Programmer’s Show And Tell section below).

You can also select the type of LCD using a compiler flag.  LCD support is basically the same as in the Netduino Plant Light Controller code, with the addition of the Adafruit i2c / SPI character LCD backpack — more details on that support are also in the Programmer’s Show And Tell section.

If you are using an LCD, you’ll probably want to add a button to D7, to toggle the backlight on or off.

Soil Moisture Sensors

The soil moisture sensors use the same approach as in the Garduino project.  I built mine from

• A pair of galvanized nails.  I think any nails will do, as long as they’re galvanized and big-assed, I’m using Tree Island Gold 4” 20d Hot Dip Galvanized nails
• 2 lengths of 20 AWG wire, long enough to go from the plant pot to the Panda
• a 10K resistor, connecting the analog in to ground

As you can see from the photo, one end of each wire is soldered to a nail, then covered with heat shrink tubing to keep it attached tight.  The other end of the wires connect to the Panda.  One wire connects directly to 3.3V, and the other other wire is split: it connects directly to an analog port, then connects through a 10K resistor to ground.

Moisture Sensor Wiring - Click to Enlarge

I found that the screw connectors on the Adafruit Wing Protoboard is ideal for this setup, since the screw connectors firmly hold the wires in place , and the protoboard layout makes it easy to add the resistor-to-ground connection.  The Adafruit board also provides 4 spare screw connectors, in the upper-right of the above photo – I used mine to provide the extra 3.3V connections required for this project.

The Garduino actually uses 5V, but 3.3V works just as well for this purpose, with similar calibration.  My oldest moisture sensor has been in service for almost 3 months now, with no apparent degradation of the nails or weird behaviour in the voltage readings.  (No electrocuted plants either – the 10K resistor ensures that the current is too low to hurt either plants or humans).

The light sensor is a garden variety (nyuk nyuk) cadium sensor that uses the exact same circuit as the moisture sensors: one wire connected to 3.3V, and the other split between an analog pin and ground through a 10K resistor.

Here is what the XBee Sensor looks like with all its accoutrements:

FEZ XBee Sensor - Up and Running

FEZ XBee Sensor - The Next Generation

Temperature/Humidity Sensor

I have the DHT11 from the A Little Help From Its Friends article connected to one of my XBee Sensor configurations, and the DHT11′s big brother, the DHT22, connected to another.

The code used to read from the 2 devices is identical – the only difference is how the .Net code parses the data to retrieve the temperature and humidity values.

As mentioned earlier, the Panda’s support for native “RLP” (Runtime Loadable Procedures) code means that it can handle the microsecond-level timing required by the DH22 and DHT11.

The RLP code that I’m using comes from this project on GHI’s TinyCLR site.  The description of the project only mentions the DHT11, but the RLP code works for the DHT22 too.

I’ve modified the code slightly to prevent an infinite loop if anything goes wrong with the communication, and added a few comments to explain why the code is doing what it’s doing. My modified version, along with a compiled .elf file and all of the files needed to build the .elf, can be downloaded from my Google Code project page.

(By the way, if you Google for the DHT11 or DHT22 datasheet, you’ll probably find a “Chinglish” version that’s awfully hard to understand.  A better datasheet can be found here: it refers to the RHT21, but that’s identical to the DHT22 as far as the communication is protocol is concerned).

The native code isn’t something that you copy and paste into Visual Studio alongside your C# code – instead, it is built using a separate piece of open source software with the charming name of Yagarto (Yet Another GNU ARM Toolchain).

An introduction to generating RLPs with Yagarto can be found in TinyCLR’s tutorial.  I’d recommend you use this more extensive tutorial instead, which covers a few small but important steps skipped by the TinyCLR tutorial.

However, if you’d rather not learn Yet Another Anything at this time, the good news is that you can just use the .elf file that is included with the source code for this article.  It should work with any DHT11 or DHT22 connected to any of the FEZ devices that use the USBizi100 chipset: Fez Mini, Panda, or Panda II.

You will have to get your own unlocking code from GHI, to paste into the Visual Studio project. It’s a quick and painless, if somewhat mystifying, process:

1. Create a MyGHI account here: http://www.tinyclr.com/register/

2. Once registered, login and click on the My Account link.

3. Click on the RLP Access Code link

4. Click the checkbox to agree to the “Technology Access Agreement” (which I assume is the whole point of this process), click Submit and your access code will be e-mailed to you.

The access code is pasted into the Visual Studio project, as shown in the Programmer’s Show and Tell section below.  The first time you run the unlock command on a FEZ device, it sets aside 10K for RLP use, then reboots.  The device remains unlocked after that, until the next time you update the firmware.

I’m not sure what exactly the unlock command does to your FEZ device, but based on my experience it’s a safe thing to try.  It doesn’t cause any instability or change the way that you debug your code.  If you decide you’d rather have the 10K back for .Net code, then you can remove RLP support by reloading the FEZ firmware.

I should point out one “gotcha” regarding the DHT11/DHT22 support, which probably applies to any RLP code that is timer-dependent.  You need to invoke the RLP subroutines from the main thread of the .Net application, not from a timer or event handler.  When I tried reading invoking the code from the timer thread that reads the other sensors, I found that the code ran slightly too slowly to keep up with the signals being sent from the DHT22 (which explains why I modified the code to break out of an infinite loop).  When invoked from the main thread, the code is rock solid – I haven’t noticed any failed readings in the weeks that I’ve been running it.

You might be wondering if RLP support is worth all the trouble.  If all you want is to support the DHT11/22, then you actually have an alternative to RLP: this TinyCLR Code project describes an alternative approach using just managed C# code and a clever hardware hack (which would probably also work with a Netduino, by the way).  However,  I really think RLP is worth learning, since it opens up support for all kinds of accessories previously off-limits to Micro Net Framework devices, such as Graphical LCDs and audio input.  For examples, do a search of “RLP” in the TinyCLR Code section.  Pretty much anything that the Arduino supports can (in theory) also be supported for Fez using RLP.

XBee

The XBee configuration at both the FEZ and Python end is the same as in the Netduino Meets World article, with one exception. The Python code will now be using the XBee in API mode, and its XBee (but not the one at the FEZ end) must therefore have its API mode enabled.

The easiest way to do this is to open a console connection to the XBee (as described in Netduino Meets World), and enter

+++
ATAP 1
ATWR

Or, if you prefer, you can use the X-CTU Windows application to turn on API mode.

X-CTU setting for API Mode

Note that the XBee address at the FEZ still needs to be hard-coded as “1”, even if you are using multiple XBee Sensors, or both XBee Sensors and a Netduino Plant Light Controller.  This isn’t a problem yet, since the FEZ/Netduino code doesn’t care what it receives, and the Python code doesn’t use the address to identify what it is receiving.  That will change in a future stage of the project

Python Middleware

The Python code has the same role as in the Watching the Watcher article: it forwards the sensor readings to a data repository on the Internet.

I’ve expanded the Python code to support 2 different repositories: Nimbits and Pachube.  I’ve also expanded the configuration file to give you a bit more control over which sensors are posted to which sites.

The Python code is compatible with the Netduino Plant Light controller that was created in my previous blog posts.

You can download the Python code and sample configuration file from my Google Code project page.  Note that I’ve renamed them to better reflect their role: SensorRelay.py.

The sample configuration file looks like this:

# Configuration file for SensorRelay.py
# ----------- General App Settings ------------
# XBEEPORT = \.COMxx <-- on some systems, Windows COM port 
#   must be specified in this format
XBEEPORT = COMxx
# USBPORT = COMyy
LOGFILENAME = SensorRelay.log
LOGDATA = False
SENDTIME = True
DEBUG = True
# -------- Sensor Settings -----------
#  - sensor line format is <Netduino Sensor ID> 
# = <Nimbits Data Point ID OR Pachube Datastream ID>
#[,Update Interval in seconds]
[Nimbits 1]
NIMBITS_SERVER = http://app.nimbits.com
NIMBITS_USERID = you@wherever.com
NIMBITS_API_KEY = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
UPDATE_INTERVAL = 300
A1 = ASensor,60
A2 = AnotherSensor
T1 = TemperatureSensor,1500
H1 = HumiditySensor
[Pachube 1]
PACHUBE_FEED_ID = nnnnn
PACHUBE_API_KEY = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
UPDATE_INTERVAL = 300
A1 = ASensor,60
A3 = AThirdSensor
T1 = TemperatureSensor
H1 = HumiditySensor

The settings are:

XBEEPORT = The serial port that the XBee is connected to.  On a Linux box, this would have a different format, like /dev/ttyUSBn.

USBPORT = The (very optional) serial port that an XBee Sensor is directly connected to.  (Technically, this would make it a FEZ USB/XBee Sensor – nice acronym!)

Note that you can use an XBEEPORT or a USBPORT, or both.  Just comment out or delete the one you aren’t using.

In the Python code in the Watching the Watcher article, I mentioned that the XBEEPORT setting could actually be set to the port of a USB-connected Netduino.  Why did I break that out to a separate USBPORT setting now?  The Python code now sends data to the XBEEPORT using the lower-level XBee API, while it still uses a generic Serial interface to a USB-connected device.

LOGFILENAME – The filename (or full path and filename) to which sensor data is logged in CSV format.  Note that, unlike the Python code in the Watching the Watcher article, errors and debug messages aren’t written to this file – they are written to a file named SensorRelayError.log instead, in the directory from which SensorRelay.py is run.

LOGDATA – This is used to enable or disable the logging of data to LOGFILENAME.   If your Python code is running on a ultra low power device like the Beagleboard, with just an SD card as its disk drive, then turning off data logging saves some wear-and-tear on the SD card.

SENDTIME – If true, the date and time will be sent to the XBeeSensor each minute.  You’ll want to leave this turned off if you’re using an XBee Sensor, but you should turn it on if you’re using the Netduino Plant Light Controller from my previous blog posts.

DEBUG – If true, debug messages will be written to both the console and the XBeeSensorError.log file.

The sensor configuration sections of the configuration file must always start with a name in square brackets, like “[Nimbits 1]”.  (The name can be whatever you like, actually, as long as you don’t use it more than once).  There must be at least 1 sensor configuration section, but you can have multiple sections if you want to send your data to multiple data repositories.

Personally, I use three: 1 for the public Nimbits app server (to test my code with the latest version of Nimbits), one for my personal Google App Server installation of Nimbits (to which I log more frequently), and one for Pachube, so that I can play around with its API too.

The Nimbits settings are identical to the ones used in the Watching the Watcher article – please see it for details.

The Pachube settings are almost the same as the Nimbits settings, but the lingo is somewhat different.  For details on getting started with Pachube, see my Tweet-A-Watt: Beyond the Twitter article

PACHUBE_FEED_ID = The numeric feed ID.  (A Pachube Feed is a group of datastreams.  A datastream is the Pachube equivalent of a Nimbits Data Point.)  If you want to send different sensor readings to different feeds, you should create multiple Pachube sections in your configuration file (e.g. [Pachube 1], [Pachube 2]).

PACHUBE_API_KEY = A Pachube API key that you’ve generated, and which has write access to your feeds.

UPDATE_INTERVAL = As with Nimbits, this determines how often the sensor data will be posted to Pachube, in seconds.  Note that Pachube has a rate limiter that depends on the type of plan you have.  Each datastream update counts as a separate API call, so if you have 5 datastreams, that is 5 API calls.

SENSOR_ID = DATASTREAM_ID[, UPDATE_INTERVAL] = Determines which sensor reading received from the XBee Sensor will be sent to which Pachube datastream, optionally overriding the default update interval.  For example

A1 = ASensor,60

…means that sensor A1 received from the XBee Sensor will be sent to Pachube Datastream ASensor, every 60 seconds.

Programmer’s Show and Tell

.Net Code

As before, you can get the .Net code from my Google Code page in either of 2 ways:

1. If you are familiar with Subversion, you use any SVN client (my favorite is TortoiseSVN) to get a copy of all the project files.  The SVN URL is https://gigamega-micro.googlecode.com/svn/trunk, and the code for this article is in the XBeeSensor article.
2. Or, download the good old .zip file from the Downloads section of the Google Code project. Here is a direct link to the code for this article.

I’m using the MicroLiquidCrystal library to add support for an i2c-connected LCD. The solution file will look for this project, and grumble if it can’t find it.  If you aren’t using an i2c LCD, you should remove the Reference to the MicroLiquidCrystal library from the project References list.  Otherwise, you’ll need to download the MicroLiquidCrystal source code and store it in the XBeeRemoteSensor solution folder..

While the .Net code is a totally separate project from the Netduino Plant Light controller, a lot of its contents will be familiar if you’ve read the Show and Tell sections of my previous blog posts.

Switching from the Netduino to the FEZ doesn’t require many changes to the code.  The underlying .Net Micro Framework codebase is 100% identical, and very little of the XBee Sensor code makes use of features that are specific to the Netduino or FEZ.  Where it does, I’ve used “#if” conditional compiler flags to include the correct code.

By the way, I’ve switched to putting compiler flags in the Project properties, as shown in the screenshot below, rather than hard-coding them with #define statements.  The default flag is “FEZ” – if you want to use a Netduino instead of a FEZ device, remove “FEZ” from the Conditional Compilation Symbols

Compiler Flags

In addition to getting the compiler flag right, the project’s References need to point to the right set of assemblies. For the FEZ Panda, use GHI’s libraries; for the Netduino, use SecretLabs’.  Your references lists should look like the one below – if you’ve got exclamation marks next to the GHIElectronics libraries, then perhaps you haven’t installed the FEZ SDK yet?  Otherwise, any problems are likely due to differences in the versions of the GHI libraries that I’m using and the ones you’re using – just delete the 3 GHI libraries from the References list, then add them back from the list on your PC.

Project References List

The compiler flags take care of the rest.  For example, the “using” statements at the top of each class ensure that the proper assemblies are being used:

#if FEZ
using GHIElectronics.NETMF.FEZ;
using GHIElectronics.NETMF.Hardware;
#if DHT22 || DHT11
using GHIElectronics.NETMF.Native;
#endif
#else
using SecretLabs.NETMF.Hardware;
using SecretLabs.NETMF.Hardware.Netduino;
#endif</pre>

Near the top of Program.cs, you’ll find the following code:

// !!!!!!!!!!!!!!!! USER MUST SET THE FOLLOWING 3 LINES!!!!!!!!!!!!!!!!!!!!!!!

const int NUM_ANALOG_SENSORS = 5; // number of analog sensors

#if FEZ

static FEZ_Pin.AnalogIn[] sensorPins = new FEZ_Pin.AnalogIn[] { FEZ_Pin.AnalogIn.An0, FEZ_Pin.AnalogIn.An1,

FEZ_Pin.AnalogIn.An2, FEZ_Pin.AnalogIn.An3, FEZ_Pin.AnalogIn.An4};

#else

static Cpu.Pin[] sensorPins = new Cpu.Pin[] { Pins.GPIO_PIN_A0, Pins.GPIO_PIN_A1, Pins.GPIO_PIN_A, Pins.GPIO_PIN_A3, Pins.GPIO_PIN_A4 };

#endif

static string[] sensorIDs = new

string[] { "M1", "M2", "M3", "M4", "L1", "T1", "H1" };

// !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

By default, this is configured for 5 analog sensors, A0 – A4, as well as a DHT22 providing temperature and humidity readings.  You’ll need to modify this to match your own configuration.  Note that, if using a DHT11 or DHT22, the 2 sensor IDs for temperature and humidity should be added to the end of the sensorIDs array.  The tags you use for sensorIDs can be anything you like, but they need to match the sensor IDs in the Python configuration file, SensorRelay.cfg.

Note that the above code uses a different syntax to reference analog pins on a FEZ device as compared to the Netduino.  This difference in syntax applies to the digital pins too.  While the pins might be in the same location on the outside, they map to very different CPU pins internally.

As mentioned earlier, I’ve added support for the Adafruit i2c/SPI Character LCD backpack.  This required very little code, since the backpack is supported by the MicroLiquidCrystal .Net library that I used in the Netduino Plant Light Controller code.

The backpack is enabled by adding MLC_I2C to the compiler flags.

There is only 1 I2C port on the FEZ devices (as with the Netduino), and the .Net Micro Framework is able to figure out which pins the i2c interface uses.  So, the i2c LCD initialization code just passes the number of rows and columns:

lcd = new clsLCD_MLC(4, 20);

Beyond that, the LCD interface is identical to the one used by parallel LCDs, as described in earlier articles about the Netduino Plant Light Controller.  One catch, though: I2C is quite slow compared to other LCD interfaces, and if you have multiple threads writing to the LCD, there is a good chance that one will interrupt another in mid-communication, causing the LCD to hang go all garbly.  To prevent this, I use the Monitor method of .Net’s Threading library:

static string lcdLock = "LockMe!";

. . .

try
{
	if (blnDisableLCD)
	{
		return;
	}

	// prevent other thread from writing to lcd at same time (mandatory for I2C LCD)
	Monitor.Enter(lcdLock);

	. . .

	for (int i = 0; i <= intLastRow; i++)
	{
		lcd.SelectLine((byte)(i + 1), true);
		lcd.WriteStringToLCD(strLCDBuffer[i]);

	}
finally
{
	Monitor.Exit(lcdLock);
}

Support for the DHT11 or DHT22 temperature/humidity sensor is enabled by adding the “DHT11″ or “DHT22″ compiler flag.  As mentioned earlier, this makes uses of the FEZ support for native RLP code.  The DHT11/DHT22 code won’t work at all if you’re using a Netduino.

All RLP code is initialized in basically the same way:

-          Call RLP.Enable

-          Call RLP.Unlock, passing it the values that you received in the e-mail.  Just copy and paste the “RLP.Unlock” statement from the e-mail into the code.

-          Load the RLP code (an ELF file) from your projects Resources section

-          Call GetProcedure for each RLP function that you want to call.

RLP.Enable();

//TODO - If your device isn't already unlocked,
//  you MUST paste YOUR ID and byte array into the code below
//RLP.Unlock();

byte[] elf_file = Resources.GetBytes(Resources.BinaryResources.dht11_rlp);
RLP.LoadELF(elf_file);

RLP.InitializeBSSRegion(elf_file);

ReadDHT = RLP.GetProcedure(elf_file, "ReadDHT");
DHTSetup = RLP.GetProcedure(elf_file, "Setup");

// We don't need this anymore
elf_file = null;
Debug.GC(true);

If the initialization succeeds, you can then call the DHTSetup function in the RLP code, passing it the pin that the DHT11 or DHT22 is connected to:

// Call setup function - use Di4 for DHT22
if (DHTSetup != null)
{
	success = DHTSetup.Invoke((int)(FEZ_Pin.Interrupt.Di4));
}
if (success == 0)
{
	Debug_WriteToLCD("DHT Setup failed!");
	DHTSetup = null; // disable use of DHT22
}
else
{
	// give DHT time to stabilize
	Thread.Sleep(1000);
}

To read the temperature and sensor data, call the ReadDHT function.  It returns a 5-byte array:

int intResult = ReadDHT.Invoke(bytArray);

Up to this point, the code used for the DHT11 and DHT22 is 100% identical.  The only difference is how the 5-byte array is processed to extract the temperature and humidity:

if ((intTotal & 255) == bytArray[4])
{

#if DHT22
	int intValues = 256 * (bytArray[2] & 0x7f) + bytArray[3];
	if ((bytArray[2] & 0x80) != 0)
	{
		intValues *= -1;
	}
	fltTemperature = intValues;
	fltTemperature /= 10;
#endif
#if DHT11
	fltTemperature = bytArray[2];
#endif

	Debug_WriteToLCD("Temp " + fltTemperature.ToString("F1"));

#if DHT22
	intValues = 256 * bytArray[0] + bytArray[1];
	fltHumidity = intValues;
	fltHumidity /= 10;
#endif
#if DHT11
	fltHumidity = bytArray[0];
#endif

	Debug_WriteToLCD("Hum " + fltHumidity.ToString("F1"));
}
else
{
	Debug_WriteToLCD("Checksum failed");
}

Python Code

The Python code is based on that described in the Watching the Watcher article, but with some notable improvements.

I’ve rewritten the logging code to follow proper Python standards.  So, instead of dumping error messages and debug information in the same file that contains the sensor data, I’m using the standard Python logging library:

logging.basicConfig(level=logging.DEBUG,

format='%(asctime)s %(levelname)s %(message)s',

filename='SensorRelayError.log')

log = logging.getLogger()

Note that you can change the level of logging, and the log file name, by changing this line of code.  For example, if you wanted to have debug messages sent just to the console, but not the log file, change logging.DEBUG to logging.ERROR.

Exception logging now follows Python standards as well, with the full stack trace being written to the log.  For example:

try:

    . . .

except:

    log.exception('Error sending to pachube, datapoint ' + dataPoint)

The resulting error message in the log includes all the fixings:

2011-09-19 17:40:06,180 ERROR Error sending to pachube,
   datapoint PlantLight1
Traceback (most recent call last):
File "SensorRelay.py", line 51, in sendToPachube
pac.put()
File "build/bdist.linux-armv7l/egg/eeml/datastream.py", line 45, in put
conn.request('PUT', self._url, self._eeml.toeeml().toxml(),
   {'X-PachubeApiKey': self._key})
File "/usr/lib/python2.6/httplib.py", line 914, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python2.6/httplib.py", line 951, in _send_request
self.endheaders()
File "/usr/lib/python2.6/httplib.py", line 908, in endheaders
self._send_output()
File "/usr/lib/python2.6/httplib.py", line 780, in _send_output
self.send(msg)
File "/usr/lib/python2.6/httplib.py", line 739, in send
self.connect()
File "/usr/lib/python2.6/httplib.py", line 720, in connect
self.timeout)
File "/usr/lib/python2.6/socket.py", line 547, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):

The Xbee is no longer being treated as a generic serial interface, as it was in the Watching the Watcher article.  Instead, I’m using the Python XBee library, which reads and writes data using the XBee’s API.

To be honest, there is no great advantage to using the XBee API at this point.  I’ve found it to be quite stable – after the Python code opens the XBee port after startup, the port stays open and keeps working regardless of what’s going on at the sensor end of the connection.  The serial port interface to the XBee was just as stable. However, the API lets me send data to a specific remote sensor device, and lets me know which remote sensor device I’m receiving data from, something which will be necessary later in the project.

The code which communicates with the XBee is similar to the serial interface described in the Show and Tell section of Watching the Watcher.  The only significant difference is that I can now define a callback to activated when data is received, rather than having to poll for incoming  data:

if XBEEPORT:
    try:
        serXBee = serial.Serial(XBEEPORT, BAUDRATE, timeout=TIMEOUT)
        serXBee.close() # workaround for known problem when running on Windows
        serXBee.open()
        xbee = XBee(serXBee, callback=readXBeeData)

The callback function receives a “frame” as its parameter.  To get to the sensor data sent by the FEZ (i.e. the same data that would be received from the XBee when it is used as a serial device), just access the frame.data property.  Note that the frame has a bunch of other potentially useful properties, as shown in the code below.

def readXBeeData(frame):
    global buf
    try:
        frame_id = frame['id']
        # source address is 2-bytes binary (e.g. x00x01)
        # this also works: source_addr = struct.unpack('>h', frame['source_addr'])
        source_addr = ord(frame['source_addr'][0]) * 256 + ord(frame['source_addr'][1])
        # rssi (signal strength) is 1-byte binary
        rssi = ord(frame['rssi'])
        data = frame['rf_data']
        if DEBUG:
            print "read_frame, id=", frame_id, ", source_addr=", source_addr, ", rssi=", rssi, ", data=", data
            log.debug("read_frame, id=" + str(frame_id) + ", source_addr=" + str(source_addr) + ", rssi=" + str(rssi) + ", data=" + data)
        if frame_id == "rx":
            processXbeeData(data)

When sending data to the FEZ, you need to package the data into a frame.  I found that the syntax for doing this is surprisingly unGooglable: the Python XBee library is more commonly used to send the XBee a command (e.g. set D2 high), not a custom data string.  The trick is to format the data in a “TX” command.  Note that I’m adding an ending null to the data – this makes the Python code compatible with the Netduino Plant Light Controller code used in previous blog posts.

        if xbee:
            strData = strPrefix + ":" + strCommand
            # HACK - append null to data to make it look like serial data to the other end
            xbee.send("tx", dest_addr = 'x00x01', data=strData + 'x00')

When sending data to Pachube from Python, I originally used the python-eeml library, as described in the Tweet-A-Watt: Beyond the Twitter article.  However, that library still uses the Pachube V1 API, which was limited to numeric datastream IDs.  Since the XML EEML format contains a bunch of optional data that I’m not using, I decided to simplify things and use Python’s standard urllib2 libraryinstead:

        # Note - the following codes uses the V2 Pachube API and CSV data format
        url = 'http://api.pachube.com/v2/feeds/' + siteconfig["PACHUBE_FEED_ID"]
           + '/datastreams/' + dataPoint + '.csv?_method=put'
        # HACK round to 3 decimal places and drop trailing zeros : technically not needed, but makes Pachube output nicer looking
        data = ("%.3f" % floatValue).rstrip('0').rstrip('.')
        headers = {'X-PachubeApiKey': siteconfig["PACHUBE_API_KEY"]}
        if DEBUG:
            print url
            print data
            print headers
        req = urllib2.Request(url, data, headers)
        try:
            response = urllib2.urlopen(req)
        except urllib2.HTTPError, e:
            log.exception('Error code ' + e.code + ' sending to pachube, datapoint ' + dataPoint)
            return False
        except urllib2.URLError, e:
            log.exception('Reason code ' + e.reason + ' sending to pachube, datapoint ' + dataPoint)
            return False

Note the code adds an  “X-PachubeApiKey” setting to the HTTP header, something that isn’t well documented for the urllib2 library.

Wrapping Up

Whew, another super-sized article.  Believe it or not, it takes even longer to write than it does to read!  The upside is that I’ve had the code running for over a month now, so I can report that the FEZ XBee Sensor is a reliable platform with no[t too many] bugs.

I’m quite impressed with the FEZ Panda.  Its support for native code and for Output Compare (another Arduino feature that isn’t supported by the .Net Micro Framework) opens up a lot of new opportunities. GHI updates the firmware quite frequently – more often than Secret Labs does with Netduino – and I suspect they will be first out of the gate with support for the newly released .Net Micro Framework 4.2

Next, though, I’m going to turn my attention to the other end of the data stream, the data repositories.

 It’s been a few months now since the Netduino in this project, with the help of its Arduino cousin, learned to measure the temperature and humidity in the room.  It’s faithfully reporting this data on its LCD… I think.  Who really knows what it’s doing when I’m not around?  It might be sleeping, or watching Fox News, or doing God-knows-what with its “cousin”.

 Clearly, this situation demands a nanny cam.  But my eyes ache at the thought of reviewing hours of footage of an LCD displaying temperature and humidity data (… I think).

Fortunately, now that the Netduino is connected to the Internet, we have a better option: online data monitoring.

 The Asocial Network: Rise of the Machines

 You’ve probably read recently of a rising trend, The Internet of Things.  It is becoming increasingly easy to hook devices and sensors up to the Internet, where they can report their status, receive instructions, play Farmville, and murder us in our sleep.

 The backbone of this system is the online data repository, which records the data and allows you to view the results..  I’ve written before of two of the early entrants in the field, Pachube and Nimbits. 

In this article, I’ll be connecting the Netduino to Nimbits via a Python middleman.  As explained in my last article, I prefer this approach to a common alternative, connecting the Netduino directly to an Ethernet connection.  If you’re interested in trying that approach instead, there’s a whole book that shows you how.

 Both Pachube and Nimbits use a REST API to receive data from sensors.  In past articles, I showed the Python code I use to send Tweet-A-Watt data to Pachube and to Nimbits.  I’m using the same API here.

 My article on using Tweet-A-Watt with Nimbits also covers the process of creating an account on the Nimbits server, getting an API key, and creating Data Points.  You’ll need to do all these things in order to receive data from the Netduino.  Your Nimbits settings are saved in the Python configuration file, explained below.

Netduino Configuration

The Netduino hardware and software is unchanged from the previous stage of the project, Netduino Meets World.  You’ll need to have XBees configured at both the Netduino and Python ends of the configuration – details are in Netduino Meets World.

Well, actually, you don’t need XBees if the Netduino is located quite close to the PC that’s running Python.  This was true for the Netduino Meets World article as well, I just forgot to mention in there.  Since we’re using the XBees as a generic serial device, a USB connection will also work with no software changes needed.  I have one of my Netduino’s connected by USB to a Beagleboard XM running Python. 

You can’t use the Netduino’s USB port for this, but you can connect the pins to TTL-to-USB adapter.  One option is the 3.3V FTDI cable that I described way back in the Netduino: Time and Weather article.  A slightly cheaper option is an FTDI breakout board – I’m using this one from Sparkfun and can confirm that it works fine with the code accompanying this article.

Configuring the Python Interface

The Python code, TimeRelay.py, builds upon the foundation of the Python program used last time, TimeServer.py.  TimeServer was configured using command line switches – since there are a lot more settings now, I’ve switched to using a configuration file.  You can download the Python code and sample configuration file from my Google Code page, here.

The configuration file contains the following settings.   The ones that you have to fill in are marked in italics.

# Configuration file for TimeRelay.py
# SERIALPORT = \.COMxx <-- Windows COM port must be specified in this format
SERIALPORT = /dev/ttyUSB0
LOGFILENAME = TimeRelay.log
DEBUG = True
NIMBITS_SERVER = http://app.nimbits.com
NIMBITS_USERID = 
NIMBITS_API_KEY = 
NIMBITS_UPDATE_INTERVAL = 300
# Sensor IDs - format is <Netduino Sensor ID> = <Nimbits Data Point ID>[,Nimbits Update Interval in seconds]
H1 = PlantRoomHumidity,1800
T1 = PlantRoomTemperature

The first 3 settings correspond to the command line parameters in the program’s predecessor, as listed in the last article:

• SERIALPORT – Specifies the serial port that the XBee is connected to.  Note the unusual format of the serial port name for Windows PCs — this is required by the PySerial library.
• LOGFILENAME – The logfile has an expanded role – in addition to any error messages, it also records the data readings received from the Netduino in a comma-delimited format.  You can disable the local logging feature by leaving this setting blank (i.e. deleting “TimeRelay.log” from that line).
• DEBUG – Turns on debug messages written to the console.  This is on by default, since it is quite useful for troubleshooting.

The Nimbits settings are taken from your Nimbits account:

• NIMBITS_SERVER -  The default setting, app.nimbits.com, is the public Nimbits server.  If you don’t want to rub shoulders with the unwashed masses, you can setup your own private Nimbits server on Google App Engine – you would then enter your GAE server name here (e.g. whatever.appspot.com).  (Both Nimbits and GAE are free when used for this purpose.  I briefly described the process of setting up a Nimbits server in the “Rolling Your Own” section of my Nimbits article)
• NIMBITS_USERID – This is the Gmail address you used when creating your Nimbits account.
• NIMBITS_API_KEY – This key is e-mailed to you when you click the “Secret Key” toolbar button in your Nimbits server console. 
• NIMBITS_UPDATE_INTERVAL – This specifies how often Nimbits will be updated with the latest sensor reading.  The default is 300 seconds (i.e. 5 minutes), but you can override this for individual sensors on the sensor lines at the bottom of hte configuration file.  Note that the Netduino will send a new measurement every 60 seconds, regardless of what you configure here – the Python app will  send Nimbits an average of all the readings since the last update.

The sensor settings are used to link the sensor IDs hard-coded in the Netduino application with the Data Point Names that you created in Nimbits.  The default settings assume Data Point names of PlantRoomTemperature and PlantRoomHumidity – if you use other names (as you might if you don’t actually have a Plant Room), then fill them in on these lines of the configuration file.

Note that the settings shown above will send a new humidity reading to Nimbits every 30 minutes, while the temperature will be sent using the default interval of 5 minutes.  Feel free to change this depending on how meteorologically active your home is.

Activate Sensors

After you’ve finished updating TimeRelay.cfg, run TimeRelay.Py and watch the console.

The program will start by listing the settings it found in the configuration file, then will open the serial port and wait for something to happen.

If a Netduino running the code from the last article is powered on,  and you left the “Debug” setting turned on in the TimeRelay.cfg file, you should see a temperature and humidity reading displayed within a minute.  A message showing the REST API call that sends the data to Nimbits should be displayed in 5 minutes (or whatever interval you used in the TimeRelay.cfg file). 

Switch to your Nimbits web page, double-click on the Data Point in the left hand pane, and the Data Channels pane should update with the newly received value.  

There are a bunch of useful settings that you can experiment with in the Data Points dialog.  I use the Idle Alarm setting to tell me when something has gone wrong with Netduino/Python configuration, and the High and Low Value alert to tell me when something has gone terribly wrong with my home.

The Nimbits web page also has some nifty built-in graphing capabilities.  These have recently changed with the release of Nimbits 3.2, and will continue to evolve, so see the Nimbits web page and Nimbits blog for the latest.

Since I have a fairly large number of data points that I like to glance at from time to time, I wrote a Google Gadget to display line charts in the iGoogle page.  I also wrote a blog post describing its development.  Check it out and let me know of any new features you’d like to see. (Or feel free to take the code and add the features yourself).

Programmer’s Show and Tell

Although the Netduino code isn’t new, I haven’t previously explained in the sensor data handling.

The upload process is run on yet another timer.  This is hardcoded to occur every 60 seconds — nothing magical about that interval, and it could certainly be changed as needed.

Since we’re using the XBee as a generic serial interface, all the Netduino needs to do is write the data to the serial port.  The sensor ID is hard-coded, and each reading is ended with a CR/LF – a “crude but effective” message format. I haven’t noticed any problems with data being garbled or truncated:

// upload sensor readings once a minute
timerUploadSensors = new Timer(new TimerCallback(uploadSensorData),
 null, 60000, 60000);

. . .
private static void uploadSensorData(object data)
{
 string strReading = "";
 if (blnHumidityUpdated)
 {
  blnHumidityUpdated = false;
  // make copy of humidity reading, since it
  //    could be overwritten at any time
  strReading += "H1:" + strHumidity + "rn";
 }
 if (blnTempUpdated)
 {
  blnTempUpdated = false;
  // make copy of temperature reading, since it
  //    could be overwritten at any time
  strReading += "T1:" + strTempDHT11 + "rn";
 }
 uart.WriteToUART(strReading);
}

The Python code is a little more interesting.

First, I’ve added a workaround for a known problem with the PySerial library when running on Windows.  When PySerial tries to open the COM port it thinks that its already open – the workaround is to always close it before opening it.  Trippy but effective, and the workaround doesn’t cause problems on other OSes:

ser = serial.Serial(SERIALPORT, BAUDRATE, timeout=TIMEOUT)
ser.close() # workaround for known problem when running on Windows
ser.open()

The code which reads TimeRelay.cfg demonstrates some of Python’s file- and string-handling power.  Five lines of code is all that is required to open a file, read its contents, parse the lines, and store them in a dictionary. 

def readConfigFile():

    #global config

    for line in open(CONFIGFILE):

        if line.strip()[0] != "#": # skip comment lines

            parts = line.split("=", 1)

            if len(parts) == 2:

                config[parts[0].strip()] = parts[1].strip()

The sensor readings received from the Netduino are stored in a dictionary.  The key is the sensor ID (e.g. “T1”) and the value is a List with format:

<# of readings since last upload>,
<total of readings since last update>,
<date/time of last upload> 

Python dictionaries are cool because they can store any object as their value, and the object type can be different for different entries in the same dictionary.  Less cool is the fact that Python throws an exception if you try to read a dictionary key that doesn’t exist yet, so be sure to check to see if an earlier reading is already stored there.

 
# add this reading to the sensor's list
if sensor in sensorValues:
    sensorValues[sensor][0] = sensorValues[sensor][0] + 1
    sensorValues[sensor][1] = sensorValues[sensor][1] + float_value
    if DEBUG:
        print "found sensorValue", sensor, sensorValues[sensor]           
else:           
    sensorValues[sensor] = [1, float_value, datetime.datetime.now()]
    if DEBUG:
        print "adding new sensorValue", sensor, sensorValues[sensor] 

Note that Python isn’t too picky about variable types except when it is.  If “sensorValues[sensor][1] ”or “float_value” weren’t both floating point values, an Exception would be thrown when they are added together.

The code that sends the data to Nimbits uses the CurrentValue REST API.  This API has a dual role of writing new values and reading the most recent value, but we’re only using it to write data here.

The code starts out with a hack, to round the value to 3 decimal places before sending it. This is done for purely asthetic reasons, so that the readings displayed in the Nimbits web page look nice and tidy.  I call it a hack because rounding a number shouldn’t require this degree of string manipulation.  (Note to Pythonistas: I’m criticizing my code, not your language. Peace out.)

 
# HACK round to 3 decimal places and drop trailing zeros
#  - technically not needed, but makes Nimbits output nicer looking
stringValue = ("%.3f" % value).rstrip('0').rstrip('.') 

try:
    LogMsg(dataPoint + "," + stringValue)
    url = (NIMBITS_SERVER + "/service/currentvalue?value=" +
        stringValue + "&point=" + dataPoint.replace(" ", "+") +
   "&email=" + NIMBITS_USERID +
        "&secret=" + NIMBITS_API_KEY)

    urllib.urlopen(url)
    return True
except IOError:
    print 'Error sending to nimbits'
    print sys.exc_info()[0]

The REST API call is a 2-liner, another example of Python’s power and simplicity compared to .Net.  (Note to Microsoft Fanboys: I’m on your side. Peace out.)  The only trick is to replace spaces in the Data Point name with plus signs, this being a URL and all.

Note that it isn’t necessary to send a timestamp with the data – the CurrentValue API assumes the value has a current timestamp unless otherwise specified.  Good thing, that, since working with time zones in Python can be a nightmare.  (Note to Pythonistas: yeah, now I’m criticizing your language).

Unfortunately the Nimbits CurrentValue API doesn’t return a value to indicate whether the API call was successful, so the Python code won’t write an error message to the console or log file if something goes wrong.  My first indication that something is wrong is generally when Nimbits’ “Idle Alarm” sends me an e-mail.

 Wrapping Up

The Netduino project has admittedly become quite unwieldy, with each stage heaping on more hardware and/or software to the previous stage.    It’s unlikely that anyone but me has all of the Netduino hardware add-ons needed to make use of all the software’s features.

In the next article in the series, I’m going to switch to a slimmed-down Netduino software image for a sensor node, that consists of only a Netduino, an XBee, and a few new analog sensors.

[Updated August 7: Added "Windows Weirdness" section with workarounds for problems with pySerial on Windows]

Note: This article is an installment in a series about building a plant light controller (and beyond) on the Netduino microcontroller platform. For previous articles in the series, click here.

When last we saw it, the Netduino project was a pretty smart plant light controller but a pretty dumb clock. The main problem is that it can’t figure out the time on its own – when it’s powered on I have to manually set the time using its menu.

C’mon, Netduino, you don’t need me for that! Haven’t you heard of the Internet?

An Inconvenient Truth

Give a hoot, don't compute!

Instead of Ethernet, I decided to use an Xbee Series 1 connection to a PC acting as an intermediary, partly because I already had that configuration in place to support Tweet-A-Watt monitoring, but also because of certain advantages:

• Wired Ethernet connections aren’t practical for most parts of the home
• XBee is cheaper than Wi-Fi, especially if you plan to use multiple Netduinos in various parts of the home (which we will, once we add sensor monitoring to the mix)
• The Netduino is a relatively constrained development platform compared to a PC. So, it helps to limit the Netduino’s role to what’s required on the “front lines”, and have much of the code running on a PC acting as “command and control”. This, too, will become a bigger advantage when there are multiple Netduinos involved.

The big downside to this approach is that you’re leaving a PC powered on 24/7. That adds up to a lot of wasted electricity, considering how low its workload is going to be. Dedicating a 120W+ PC to this project is absurdly excessive – do you really want to make Al Gore cry?  (Well, yeah, me too – but you know what I mean.)

Fortunately, there are an increasing number of ultra low power devices that are capable of acting as the PC for this purpose. As long as it has a USB port, an Internet connection and can run Python, then it’s good to go.

I’ve written about the Beagleboard in the past. When running Linux and the Python script in this project, a Beagleboard XM uses about 3W (as measured by a Kill-A-Watt meter).

By comparison, my “energy efficient” 2 year old Windows PC averages about 120W. My older PC averages about 200W. Considering that it’s able to serve as a full-fledged home server, the Beagleboard is incredible, and hopefully an indication of the future direction of home PCs.

Incidentally, the Netduino/Arduino/LCD combination used in this project burns well less than 1W, so using *duinos in your home projects is definitely an energy-friendly alternative.

TonidoPlug Computer

Like the Beagleboards I’ve found the TonidoPlug to be very reliable device, and it’s just as power efficient. It’s not easy to upgrade or replace the OS, but the built-in version of Ubuntu is more than adequate for the projects I have in mind.

If you decide to go with a Beagleboard, one of my past articles covers installing Angstrom and Python, and this eLinux page is gospel if you’re installing Ubuntu.

As for the TonidoPlug, it comes with Ubuntu 9.04 (Jaunty Jack), Python 2.5, and an SSH Server out of the box. All you need to add for this project is the Python serial library and the GNU Screen utility for running the Python app in a background terminal session.

To install them, login as root through SSH, create then directory used by the apt package manager, then install a couple of packages:

mkdir -p /var/cache/apt/archives/partial
apt-get update
apt-get install python-serial
apt-get install screen

Connecting an XBee to a PC

Point-to-point communication ain’t no fun with just one point, so we’ll need to connect an XBee to both the PC and the Netduino.

At the PC end, whether you use a Beagleboard, TonidoPlug, or a full PC, you’ll need a USB adapter for the XBee.

There are many options out there. I’m using an Adafruit XBee Adapter and a USB TTL-232 cable. However, unless you happen to have a spare cable this is a relatively expensive option, since the adapter costs $10 and the cable costs $20. Other options that I’ve used are the $20 Parallax XBee Adapter, and SeeedStudio’s versatile $20 UartSBee.

Before connecting the XBee to the Netduino, you should configure the 2 XBees so that they will talk to one another.

By default, the XBee adapters are all configured as adapter 0 in PAN (network ID) 3332, and they’ll send their data to other adapter 0s. While this works fine if one XBee only sends, and the other XBee only receives, we’ll soon have the Netduino and PC chatting merrily away to one another. So, you should assign one of the XBees to have a different adapter ID (number 1 works fine), then set the other XBee’s destination address to be that same ID.

This page at Adafruit covers the 2 options for configuring an XBee: the Windows-only X-CTU program, and entering AT commands from a serial terminal.

Whichever configuration method you use, you need to change the “MY” setting on the first XBee to 1, and the the “DL” setting on the other XBee to 1.  That’s it: the other defaults are fine.

I’d recommend using the X-CTU program if you’re getting started with XBees: it’s much more user friendly and foolproof than a serial terminal.   While you are in there, you should update the firmware of the XBees, something you can’t do from a serial terminal.

If you’re hardcore about using a TonidoPlug or Beagleboard as a PC, or want to avoid running X-CTU for other reasons, than a serial terminal is the way to go.

Personally, I’ve used minicom on a Beagleboard and TonidoPlug for this. In order to talk to the XBee you’ll need to run minicom as root and change some of the default settings. Connect an XBee in its adapter to the USB port, then:

1. At the Linux command line, enter “minicom -s”.  If you plan to save your settings in a profile (a good idea), you should run as root: “sudo minicom -s”
2. Select “Serial port setup”
3. The serial port settings should be as shown in the screenshot below.  Press “E” to change the Bps to 9600, and press “F” to set Hardware Flow Control to “No”.  You might also need to change the Serial Device setting: its generally “/dev/ttyUSB0″, but if you have other USB devices connected: it might be /dev/ttyUSB1 or /dev/ttyUSB2, etc.
4. Press Enter to return to the minicom configuration menu, then select “Screen and keyboard”
5. Press “P” to turn on the Add Linefeed feature, and press “Q” to turn on Local Echo.
6. Press Enter to return to the configuration menu again.  To save these settings, choose one of the “Save setup” options.  To display the serial terminal,  select “Exit”.
7. The XBee is initially ready to receive and send data.  You’ll need to put it in command mode instead.  To enter command mode, press the “+” key 3 times (i.e. +++) but don’t press Enter.  The XBee should display OK.

Minicom settings for XBee configuration

The correct terminal commands for the first XBee are:

+++
ATMY 1
ATWR

And for the second XBee

+++
ATDL 1
ATWR

The XBee should send back “OK” after each command.

If you aren’t able to see what you are typing, try turning on Local Echo again: press Ctrl-A then Z to display the menu, then press E.

Note that the XBee will automatically (and silently) exit configuration mode if you wait too long between commands, so if you aren’t getting the OK after an AT command, try entering +++ again.

Connecting an XBee to a Netduino

The hardware connection to the Netduino is relatively simple, because the Netduino uses the same 3.3V voltage level as the XBee and the Netduino’s serial pins talk the same language as the XBee.

The only complication is that the XBee uses an usually tight spacing for its pins, just 2mm apart.  This makes it difficult to connect jumper cables to the pins, and impossible to plug the XBee into a breadboard.  Most people use a breakout board to solve this problem, such as Sparkfun’s XBee Breakout Board or Adafruit’s XBee Adapter PCB (which is the PCB used in their Adapter Kit – it functions well as a breakout board).

As shown in the photo below, I plugged the XBee into a small breadboard using an Adafruit XBee Adapter.  This makes it easy to share the Netduino’s 3.3V pin with other devices besides the XBee, and it allows you to connect an LED to some of the XBee’s pins for troubleshooting:

XBee To Netduino Connections

Running the Python Software

Once the hardware is in place, the next step is to start the Python software running.

You’ll need to have pySerial installed.  If you followed my instructions for configuring Angstrom Linux on a Beagleboard, or configuring Ubuntu on a TonidoPlug (earlier in this article), then you’re all set.  Otherwise, see the install instructions on the pySerial page.

Download the Python code from my Google Code project. Save the file to any folder on the host PC – if you are using a TonidoPlug or Beagleboard running Linux, then your home folder is fine.

If you are using Linux, you’ll probably need to run the Python code as root in order to access the serial port:

i.e.

sudo python TimeServer.py

The command line parameters for the program are all optional:

TimeServer.py [-nl] [-d] [-port xxxx]

• -nl – turns off logging.  Otherwise, any errors will be written to the current directory in a file named controller.txt.  Hopefully this file won’t get large, but turning off logging is necessary if you are running the code from a read-only directory.
• -d – turns on debug messages written to the console.
• -port – specifies the serial port that the XBee is connected to.  The default is /dev/ttyUSB0. On a TonidoPlug or Beagleboard, that should be the right port. On a Windows PC, the port will have a totally different name, such as COM3 – check Device Manager if you don’t know which one to use.  If you are using Windows, see the “Windows Weirdness” section below.

If you launch the program when connecting through SSH, you should first use the GNU Screen utility to open a virtual command prompt, and run the program from there. Otherwise, the program will stop running when you disconnect from SSH.

To run screen, just type “screen”. You’ll be brought to a new command prompt, at which you run TimeServer.py as above.

When TimeRelay is up and running, you can disconnect from your screen session by pressing CTRL-A, then D.  This leaves the Python code running in the background.   To reconnect to the screen session, enter screen –r –d.

GNU Screen is handy for running a background process like this because you can reconnect at any time, from any place, and run screen –r –d to see the current output from the TimeServer.py program.  This is really helpful for debugging, and for restarting the Python code if (heaven forbid!) it should start misbehaving.

Windows Weirdness

Running the Python code on Windows presents a couple of other complications, besides that whole “global warming melting glaciers” thing.

If Python throws an “The system cannot find the file specified” Exception when it tries to open the serial port, you may have run into the problem described on StackOverflow here. Try specifying the COM port as ”\.COMxx”

e.g.

-port \.COM10

Hey, don’t ask me — try it.

Also, opening a serial port sometimes results in an “Access denied” exception on Windows if you don’t close it before opening it.  Like I said, don’t ask me.  Since closing the serial port before opening it doesn’t seem to cause any problems, I’ve updated the Python code with this fix.

Running the Netduino Code

You can get the Netduino code in one of 2 ways:

1. If you are familiar with Subversion, you use any SVN client(my favorite is TortoiseSVN) to get a copy of all the project files.  The SVN URL is https://gigamega-micro.googlecode.com/svn/trunk. If you want the older version of the code to match one of the past articles, get the corresponding revision. (See the revision list here)
2. Or, download the good old .zip file from the Downloads section of the Google Code project. Here is a direct link to the code for this article.

If the Python program is running, then you’ll know within a minute of powering on the Netduino whether everything is working correctly: the date and time will automatically be set correctly via the XBee connection. Tah-dah!

OK, OK, maybe that wasn’t worth a Tah-dah.  But, that’s not all: the LED you connected to the XBee’s receive pin will light up once a minute. Tah-dah!

Hmm, tough audience.

About the only other change to the Netduino software this time is that I finally added a Daylight Savings Time setting to the Netduino’s menu: you can turn DST off or on in the “Date” menu. (Tah-dah!) This is used when calculating the “Sun Up” and “Sun Down” times, if you are basing the plant light timer on those times. Purists will point out that an Internet-connected Netduino shouldn’t have to rely on a manual setting to know when DST is in effect, but we’ll ignore those purists for now.

Programmer’s Show and Tell

The Python code is admittedly very simple: all the better to learn Python programming from, right?

The XBee’s serial port connection is opened during startup. If the connection can’t be opened, the program will display an error message and terminate by calling the Python sys.exit function. If you had some problems getting the serial port working earlier, you’re no doubt well acquainted with this error message.

try:
    ser = serial.Serial(SERIALPORT, BAUDRATE, timeout=TIMEOUT)
    ser.close() # workaround for known problem when running on Windows
    ser.open()
except:
    print "Unable to open serial port ", SERIALPORT, ". See ya."
    sys.exit(1)

The main loop checks whether the time has changed (just the minute, actually – we’ll let the Netduino handle the seconds). If it has, it calls the sendDateTime() function. It then sleeps for awhile before repeating the loop.

def displayTime():
    lastMin = -1
    while 1==1:
        try:
            currTime = datetime.datetime.now()
            if DEBUG:
                print "in displayTime, currTime = " + currTime.strftime("%Y-%m-%d %H:%M:%S")
            currMin = currTime.minute
            if lastMin != currMin:
                lastMin = currMin
                sendDateTime()
            time.sleep(15);
        except Exception:
            LogMsg("Error in displayTime")
            print sys.exc_info()[0]

Note that that the Exception handler is different from the one used when opening the serial port: it logs the error, displays some information about the error to the console (using the Python sys.exc_info function), then optimistically keeps going. If there was no Exception handler, any exception in this loop, or in the functions called by this loop, would cause the application to terminate.

This “log it and keep going” approach is generally frowned upon in software design, but I’d recommend it for a simple program like this that runs in the background.  You should find that this application, and Python code that communicates via XBee in general, is very low maintenance: you can just start it, then forget about it.  You can occasionally check on the code using GNU Screen or by looking at the contents of the controller.txt error log.

The sendDateTime function just converts the date and time to the format that the Netduino is expecting it, then writes it to the serial port. Note that the “ser.flush()” function is used to make sure that the data is sent right away rather than being buffered – this is literally time-sensitive data.

def sendCommand(strPrefix, strCommand):
    if ser:
        ser.write(strPrefix + ":" + strCommand + chr(0))
        ser.flush()

def sendDateTime():
    timeStr = time.strftime("%Y-%m-%d %H:%M")
    sendCommand("D", timeStr)

On the Netduino side, surprisingly little new code is required to talk to the XBee. We’re using the XBee as a serial port device, rather than making use of the XBee’s API (a more versatile option that I might explore in a future article). From the Netduino’s perspective, the XBee might be some guy sitting at a serial terminal, occasionally typing in a”set time” command. We’ve already implemented that support, back in the “Netduino: Time and Weather” article .

The XBee connection is opened during startup, using almost the exact same code as was used to open a serial connection to a PC terminal:

// last parm should be true for UART terminal (to send
//   user confirmation), but false for UART XBee
uart = new clsUART(strUARTPort, 9600, 512, false);

The event handler is the same one that was used for the PC terminal: the event is raised when our generic serial port class, clsUART, receives some data. In fact, if you’re using the SparkFun Serial LCD Kit, this event handler is still used when the Netduino receives data from the Arduino, as explained in the “A Little Help From Its Friends” article .

Sharing a serial command event handler between Arduino and the XBee works well, and is easy to implement in .Net. The Netduino doesn’t know, or need to know, who’s sending the command, and the .Net Micro framework ensures there isn’t any conflict between the 2 serial ports.

The only thing that’s changed here is that the XBee sends the date and time together in the same command:

D: 2011-07-25 18:01

So, I modified clsUART to check to see if the string is in this format, and it raises both a “date set” and “time set” event if it is.

if (strDateParts.Length == 5)
{
    intHours = Int32.Parse(strDateParts[3]);
    intMinutes = Int32.Parse(strDateParts[4]);
    intSeconds = 0;
    blnTimeSet = true;
}
...
// tell rest of application that the date has been set
raiseCommand(strCommand);
if (blnTimeSet)
{
    // raise the time event too
    raiseCommand("T:" + Program.Int_ToZeroPrefixedString(intHours, 2) + ":" + Program.Int_ToZeroPrefixedString(intMinutes, 2));
}

The event handler already knew how to handle date changes and time changes, so no code change was required there.

The only other significant change was modifying the menu to add a “Daylight Savings Time” switch. Since the menu code was designed to be easily extended – as explained in the Ordering Off the Menu article  – this required just 1 new line to the strMenuItems array:

"2-1", "Hour `HH`", "_UD-`HH`",
"2-1", "Min `mm`", "_UD-`mm`",
"2-1", "DST `DS`", "_DST", // NEW - Daylight Savings Time setting

The format looks rather cryptic, but it will make more sense if you read the “Programmer Show and Tell” section of Ordering Off the Menu.

The code in clsLCDMenu passes the “`DS`” keyword to its callback method for getting the current DST setting, and it passes the “_DST” keyword to the callback method for handling a new setting entered by the user.

For example, the code added to the setValueFromMenu() callback method for the “_DST” keyword is:

else if (menuCode == "_DST")
{
    // toggle DST
    blnUseDaylightSavingsTime = !blnUseDaylightSavingsTime;
    calculateSunriseAndSunset(DateTime.Now);
    if (blnUseDaylightSavingsTime)
    {
        SaveSettingToEEPROM(ADDR_DST, 2);
    }
    else
    {
        SaveSettingToEEPROM(ADDR_DST, 1);
    }
}

This recalculates the Sunrise and Sunset times (and updates the plant light timers if they are based on these times), then passes the new setting to the Arduino in the Sparkfun LCD Kit to save it to EEPROM.

I love it when a plan comes together!

Wrapping Up

I’ve found the XBee to be a very dependable add-on to the Netduino. I actually wrote the code used in this article back in May and have been running it continuously since then, and I’ve yet to have any cases where the XBee connection permanently drops or stops responding. In my experience, this is a big advantage over using an Ethernet connection – when the Netduino’s Ethernet connection drops it tends to stay down.

So, if the Xbee’s so great, why am I using it only to set the time?

Good question.

If you browse through the main Program.cs class in the source code, you’ll notice a new method called uploadSensorData, that I didn’t address in this article.

Stay tuned for the next step in the project, in which the Netduino talks back, and I draw together a couple of previously separate topics in my blog: the Netduino and Nimbits.

Updated Dec 3: I made a small code change to fix the autoscale feature, as marked in green below.

In my last post, I gave a user’s overview of a Google Gadget I wrote to display graphs from the Nimbits data logger.

This post is directed at developers. I’ll explain how the gadget interacts with the Nimbits API and the Google Charts API, and give a few tips to those who are getting started with Gadget development.

Actually, my tips would only be useful to gadget noobies, since I’m a gadget noob myself. Fortunately, Google’s Gadget development framework is pretty easy to get started with. This is partly due to the abundance of sample code – you can examine the source code for any gadget by clicking the “View Source” link near the bottom right of the gadget’s page in the Google Gadget directory.

However, another reason why the Gadget framework is easy to learn is that there isn’t that much to it. The limitations can be frustrating, and it doesn’t seem like the framework is under active development by Google any longer.

That’s fine with me, though: the gadget is just the container, and the contents are supplied by the Nimbits and Google Charts APIs, both of which are very much moving forward.

Viewing the Source Code

If you click on the “View source” link for the Nimbits Gadget , the first thing you’ll notice about the code is that there isn’t any. The link displays the Gadget’s XML definition. The JavaScript code is often embedded in the XML file (that’s actually what Google recommends for better performance), but I decided to put it in a separate .js file to avoid duplicating the code in “canvas” view.

So, the code is actually here: http://www.gigamegablog.com/gadgets/nimbits.js. (Yeah, I know your Mother told you not to run strange JavaScript, but if you click this link it will just display the code, not run it.)

Most of the JavaScript code is a series of functions that build up 2 calls to REST APIs:

• The Nimbits Chart API to display the graph
• The Nimbits CurrentValue API, to (optionally) return the Status line

The Nimbits Chart API

A good introduction to the Chart API is Benjamin Sautner’s blog post. . His post gives a couple of examples of calls to the Chart API, and the 2nd one is very similar to the one that I ended up using in the gadget:

http://app.nimbits.com/service/chartapi?point=TempF&email=bsautner@gmail.com&cht=lc&chs=200×100&chxt=x,y&chxl=0:|Apr|May|June|1:||100+F

If you aren’t familiar with REST APIs, then you might wonder why this looks like a URL. That’s the whole idea of REST: API calls are URLs with all of the parameters embedded in the URL string. Ideally, the URL’s parameters tells you (and, most importantly, the server) everything you need to know to handle the request, with no dependencies on previous requests, etc.

Authentication aside, if you and I paste a REST URL string into a browser, we should both get the exact same result. (And, for the purposes of this gadget, I basically tossed authentication aside – that’s why it only works with Public data points).

If you paste the above URL into a browser, you’ll get back a graph in a .png file. The Nimbits Charts API will always return a PNG. This makes it easy to use trial-and-error to experiment with the Charts API. Other Nimbits APIs, and most REST APIs in general, return XML – we’ll see an example of that in the CurrentValue API, below.

Before I get into the code I used in the gadget to build the chart…

How To Build A Nimbits Gadget in 5 Minutes

Mine took a little longer, actually, but displaying a Nimbits chart in a gadget is quite simple:

1. Open the Google Gadgets Editor: basically a Notepad that saves files on one of Google’s servers.

Replace the “Hello, world!” line with

<img
src="http://app.nimbits.com/service/chartapi?point=TempF&email=bsautner@gmail.com&cht=lc&chs=200x100&chxt=x,y&chxl=0:|Apr|May|June|1:||100+F"
/>

2.Click File, then Save. In the Save File dialog, change the filename from helloworld.xml to nimbits.xml. It should now look like this

3.Click File, then Publish. The Gadget Editor will nag you about some of the things that are missing, but that’s OK because you’re only going to publish it to your own iGoogle page, not Vanity Fair. Click OK.

4.The “Publish Your Gadget” dialog will be displayed. Click on the “Add to my iGoogle page” option.

5.Another page is opened, with a big blue “Add Hello World! To iGoogle” button. Click it.

6.If all goes well, you should see a brand new gadget on your iGoogle page, as below.

If you jumped ahead and started changing the settings in the Gadget Editor to see what would happen then, hey, sorry about that whole nervous breakdown thing. I should have mentioned that Google automatically caches your gadget’s code, so your changes won’t be automatically applied to your iGoogle page. There is a way to turn that off – see the Caching section below.

Your shiny new gadget will faithfully report the temperature in Benjamin Sautner’s aquarium. But what if you actually wanted your gadget to display something else? That’s where the code comes in…

Nimbits Chart API Options

The code block below is taken from the displayChart() function of the gadget’s JavaScript. It builds an HTML img tag similar to the one you pasted into your gadget, but getting its parameters from variables instead.

The variables are loaded from settings in the User Preferences dialog. I won’t cover that code here, since this section of Google’s Gadget Developer docs tells you all you’ll need to know.

html += '<img src=http://' + serverName + '/service/chartapi?'
  + 'points=' + pointsParm;
html += '&email=' + userEmail;
html +=  '&chds=a&cht=lc&chco=' + colourParm + '&chxt=y&chf=bg,s,EFEFEF';
html += '&chs=' + graphWidth + 'x' + graphHeight;
if (spanType == "Hours")
{
	html += "&st=-" + timeSpan + "h&et=now";
} else
{
	// # of readings
	html += "&count=" + timeSpan;
}

 if (displayLegend)
{
	// display it at top listing all data points
	html += '&chdl=' + legendParm + '&chdlp=t';
}
var gridStyle = "&chg=";
if (displayGrid == "Vertical")
{
   gridStyle += "10,0,1,0";
} else if (displayGrid == "Horiz")
{
	gridStyle += "0,10,1,0";
} else if (displayGrid == "Both")
{
	gridStyle += "10,10,1,0";
} else
{
	// no grid
	gridStyle = "";
}

Let’s break this block of code into parts

html += '<img src=http://' + serverName + '/service/chartapi?'
			  + 'points=' + pointsParm;

Every call to the Nimbits Charts API needs a server name and at least 1 DataPoint. If you want to include multiple Data Points, they must be comma-delimited. If your Data Point names include spaces, they must be replaced with plus signs (which is a Googly standard for handling spaces in URLs). For example, if there are 2 data points named “Plant Light” and “Home Meter 2”, then pointsParm is set to “Plant+Light,Home+Meter+2″.

html += '&email=' + userEmail;

Unless the user has already been authenticated (logged into iGoogle isn’t enough), the e-mail address is a mandatory parameter for the Nimbits REST API calls. Surprisingly, there is no longer a way for a Gadget to automatically determine the user’s Gmail address.

html += '&chds=a&cht=lc&chco=' + colourParm + '&chxt=y&chf=bg,s,EFEFEF';

Autoscale is a very useful setting – so useful that you can’t turn it off in my gadget.  Update Dec 3: Well, autoscale *was* a very useful feature, until it stopped working.  I’ve replaced it with the Google Chart API’s “&chds=a” parameter, which appears to have the exact same effect.

It tells Google Charts to automatically scale and label the Y axis based on the value of the points you are graphing. Without it, the graph is always plotted on a scale of 1 to 100, regardless of the actual value of the data.

For example, here is a graph of 2 Data Points with very different scales: one for a relatively low-power set of devices (DuinoFarm) and one for a big fat PC.

Here is the same graph with autoscale turned off. The scale defaults to 1-100, and data points above 100 are cut off.

The &cht parm is another mandatory one: it specifies the type of graph, with “lc” meaning line chart. Leave it out, and you’ll get the default “polar bear in a snowstorm” chart. There are a bunch of other types of graphs available, as shown in the Google Chart Gallery, but line charts are how I ride.

The &chco parm specifies the colours to be used for the lines in the graph, assigned in the same order as the data points in the points parm. The values are RGB hex codes: for example &chco=FF0000,00FF00, for red and green. Without this parm, everything is orange. (Well, Google being Google, probably whatever colour they feel like using that day. Today, it’s orange).

The &chxt is another one that I always use: it provides the vertical scale. Without it, the graph looks like this:

The last parm in this section, &chf=bg,s,EFEFEF, is responsible for that lovely gray background you see in the screenshots.

html += '&chs=' + graphWidth + 'x' + graphHeight;

This one’s pretty self explanatory. The actual value used in the gadget looks something like &chs=320×200, with the 320 part being hard-coded. The Chart API is quite happy to serve up PNGs that are much larger than this, so if you are displaying a graph outside of a gadget container (such as the “canvas” view displayed when you maximize a gadget) then you can go crazy with megapixel monstrosities.

if (spanType == "Hours")
{
    html += "&st=-" + timeSpan + "h&et=now";
} else
{
    // # of readings
    html += "&count=" + timeSpan;
}

Unlike the last few parms, this one is a Nimbits API parm, not Google Charts. It tells Nimbits which set of data point values to pass along to Google Charts. I’m using a relative timespan, where the end point is always now, and the start point is whatever offset the user specifies in the Settings panel (in hours). The Nimbits API supports a lot of other formats for the start and end time parms, allowing you to chart any time period you like, as documented in the TimeSpan page of the Nimbits Wiki, and a more flexible time span setting is definitely on the To Do list for this gadget.

The remaining settings in the code are self-explanatory, controlling the legend and grid used in the graph.

The resulting tag is plopped into the gadget’s content area, where it gets magically replaced with a PNG of the actual graph. It really is magical, since the response received from REST APIs are usually not so easy to handle.

Which brings us to the Status line…

The Nimbits CurrentValue API

The Status line displays the data point name, its most recent value, and the date and time it was received. All of that information is returned by the Nimbits CurrentValue API. This API’s main purpose is to record new values, not return existing ones, but since it always returns the latest value it’s handy for seeing when the last data was received.

For example, try pasting the following URL into a browser

You’ll get back a series of tags and values that looks like this :

{"id":7865660,"lat":0.0,"lng":0.0,"d":57.9303189764,"timestamp":"2011-03-15T00:15:03 +0000","pointFK":4405630,"note":""}

This is in JSON format, which is a simplified variant of XML that is handy for Javascript code (and most other languages) to work with.

Unlike the graph, this return string isn’t something we can plop into the gadget’s HTML and display unaltered to the user. Instead, we need to process it with code. And that means that we have to invoke the REST API in a way that passes the result to a piece of code.

In the Google Gadgets framework, this is done by calling one of their APIs:

gadgets.io.makeRequest(url, callback, params);

The “callback” parameter is the name of the Javascript function that will receive the response, and the “params” parameter are primarily used to override Google’s caching algorithm, as explained later. I took the code that invokes makeRequest pretty much verbatim from Google’s documentation for calling REST APIs.

The code which processes the data returned by the CurrentValue API is in the buildStatusString() function:

if (obj.data == null || typeof obj.data == "undefined")
{
	// something went wrong
	if (typeof obj.text == "undefined" || obj.text == null || obj.text == "")
	{
		return obj.error;
	} else
	{
		return obj.text;
	}
}
jsondata = obj.data;

var alldata = "";
for (var key in jsondata) {
	var value = jsondata[key];
	alldata += key + ":" + value + ", ";
 }
 var time1 = jsondata['timestamp'];
 var date1 = new Date(time1);
 // retrieve values as local time
 //var dateAndTime = date1.getMonth() + "/" + date1.getDate() + " " + date1.getHours() + ":" + date1.getMinutes();
 var monthNames = [ "January", "February", "March", "April", "May", "June",
	"July", "August", "September", "October", "November", "December" ];
 var dateAndTime = monthNames[date1.getMonth()].substr(0,3) + " " + date1.getDate() + "  " + date1.toLocaleTimeString();

 var value1 = jsondata['d'];
 value1 = Math.round(100 * value1) / 100;

 return dateAndTime + ' -- ' + value1;

Admittedly, this code is anything but beautiful. It got real ugly real fast when I tried to grapple with API requests that fail.

The code starts off with an attempt to figure out if the response was data or an error message. The object returned from the REST API contains 3 properties. If all goes well, the only one that you have to worry about is .data. Otherwise, .error and/or .text may contain error messages.

JavaScript’s handling of null/undefined variables must make sense to someone, but not me!

If you got data, then JSON makes processing it quite easy. The tags that you saw displayed in the browser are keys to an array, so to retrieve the timestamp value you use code like this:

var time1 = jsondata['timestamp'];
var date1 = new Date(time1);

The time1 variable is actually automatically handled as a date/time object by JavaScript, so why feed it into the Date function afterwards?

If you look at the value that is returned, it looks like this: “:”2011-03-15T00:15:03 +0000″. This is a UTC date and time string, so to make it more meaningful to the user you’ll want to convert it to their local time. In some languages, like Python, this conversion is a major pain.

JavaScript makes it easy: just feed that string into the Date function, and it automatically converts it to your local time when you access methods like date1.getDate() and toLocaleTimeString(). (Locale is different from local, by the way – toTimeString() also converts to the user’s local time.)

Google Gadgets and Caching

Google’s automatic caching of gadget code and data is something that trips up most first-time developers – it certainly got me.

Caching affects gadgets in 2 ways:

1. Gadget code is cached. So, if you change the code, then click refresh on the iGoogle page, an old copy of your code will still run, not the change you just made. This is a major performance benefit for users, but a major pain for developers.

Google feels your pain, sort of. They document this behavior, way down at the bottom of the FAQ, and offer a “My Gadgets” gadget that lets you disable this caching. The gadget has a tendency to flip from its initial state (it’s supposed to look like this screenshot in Google’s documentation) to a minimalistic “Go to developer mode” link – I’m still not sure if that’s a bug or a feature – but “Going To Developer Mode” actually does disable the caching.

2. Remote data is cached. So, for example, calls to the Nimbits CurrentValue API are intercepted by the API framework, which returns the same data as you got last time. This is another performance boost, but at least Google has documented this one properly.

The default behaviour is to cache any request that is within 60 minutes of the last time your gadget got data. You override this by passing an extra parameter in the call to the REST API – in my case, I set it to 5 minutes.

Note that this doesn’t mean your gadget will automatically update itself every 5 minutes – the update will only occur if the user refreshes the browser page, or if iGoogle automatically refreshes itself.

Other Google Gadget Gotchas

Where to host your code?

This is one of the first things you’ll have to decide, since you can’t test your code unless iGoogle can get to it through a web server. A good place to start is let Google host it – this is what happens if you enter your code into the Google Gadget Editor. Google gives you 4M to play around with, and the GGE makes it easy to add the gadget to your iGoogle page while you are testing it.

Google’s Gadget framework isn’t picky though – your gadget just needs to be an .xml file with a URL.

The one hitch with hosting it outside of the Google Gadget Editor is how to add it to iGoogle without publishing it to the world. I find the most reliable method is the Gadget Directory page for Google’s “My Gadgets” gadget . Not the gadget itself, whose “Add a gadget” feature inexplicably disappears, but the web page for the gadget, which has a convenient if equally inexplicable text box labeled “Add a gadget”. Works every time.

Edit Settings: Google’s Playground

As I discovered to my dismay after releasing the Nimbits gadget, you shouldn’t get too attached to the layout of your Edit Settings dialog. The layout is automatically generated by Google’s gadget framework, and you have no control over that layout.

Google can, and will, change that layout without warning or explanation. Other developers have experienced hidden settings that pop suddenly into view, and deleted settings that refuse to go away.

So, don’t go overboard with the options.

Debugging your JavaScript

For years and years, I heard web developers raving about Firefox’s Firebug plug-in without fully appreciating what they were raving about. Actually, I still don’t. But I can now say I’ve used it.

You’ll definitely need FireBug, or a JavaScript console of some sort ,when debugging your gadget. The console is used for catching errors in your JavaScript (it will report both the error and line number it occurred at), and for logging debug information using the JavaScript console.log() function.

I’ve left some console.log() calls in my gadget code because they are a handy way of seeing how the underlying REST API is being called. In the screenshot below, you can see the call being made to the Nimbits CurrentValue API, the JSON-formatted response a few lines later, and the call to the Nimbits Chart API at the bottom. (The console output also contains an error logged by an, ahem, “competing product”. Just kidding – Pachube’s great too, and I’m sure their request wasn’t that bad!)

Actually, FireBug’s functionality is so indispensible that Chrome and Internet Explorer have built similar features right into the browser. Having this built-in Javascript console leads to another common pitfall: don’t assume that a call to console.log() is going to work for other users, who might be running an older or non-mainstream browser. To ensure that console.log() won’t cause the script to fail, you can add the following to the JavaScript (I took the code from Alvin Abad’s blog.)

if (typeof console == 'undefined') {
    var console = {};
    console.log = function(msg) {
        return;
    };
}

Wrapping It Up

As mentioned in the introduction to this article, I’m not terribly impressed with Google’s Gadget framework.

I like the fact that it’s a small set of tools with a relatively shallow learning curve. However, the simplicity of the framework becomes a weakness when you inevitably bump up against some limitation that has no apparent coding workaround.

The documentation is so-so, which is admittedly a notch above the average web development platform’s documentation.

As someone who has been a user of iGoogle on a daily basis for many years, I’m a little alarmed to see how the sausage is made. I can’t shake the feeling that Google has lost interest in moving the platform forward. There haven’t been any significant changes to the API or user interface in a couple of years, and they haven’t bothered to expand, or even clean up, their documentation.

Google Gadgets are a fun place to visit, but I wouldn’t want to make a living there.

If you’ve made the switch to programming with .Net Micro Framework devices like the Netduino and GHI Panda, you inevitably find yourself asking the question: “What the heck do I do with all my Arduinos?”  They have a minimalistic IDE and no debugger to speak of.  They make lousy cat toys and anniversary gifts. (Unless you’re receiving one, that is.)  Who needs them?

Well, actually, you do. You might think that .Net’s “managed code” runtime environment is there to usher your program past the hardware, but it’s more like a bouncer protecting the hardware from your crummy code.   It’s great that your code can’t mess with the hardware, unless you intentionally want to mess with the hardware.

I ran into this problem when trying to upgrade the Light Controller’s analog temperature sensor with a digital “one-wire” sensor.  These types of temperature sensors have become quite common, and are generally much more accurate than analog sensors. While the Netduino’s firmware doesn’t officially support the one-wire standard yet (as of Version 4.1.0 Update 6), it is available in the GHI firmware, and I was planning to incorporate it into a Panda version of the Light Controller.

Unfortunately, not all “one wire” temperature sensors are created equally, and the one I have doesn’t follow the Dallas Semicondutors / Maxim 1-wire standard. It’s a Seeed Studios Temperature and Humidity Brick (now replaced by a Twig version) , both based on the DHT11/AM2302 chipset.

The interface to this standard is well documented (quite enthusiastically documented, in fact).  As demonstrated in the Arduino sample code, communicating with the chipset requires toggling a pin high-and-low at specific sequence of microseconds.

That’s microseconds, not milliseconds.  .Net MF doesn’t let you send signals at the microsecond level.  That’s dangerous, son.  Might shoot your eye out.

Wouldn’t it be nice if we could combine the productivity of managed C# code with the versatility of the Arduino’s AVR-C code library? Mush them together somehow, with solder and duct tape.  Hmmm…

Sadly, I happened to be out of duct tape, so I turned my attention to a Serial Enabled LCD Kit that I had recently bought from Sparkfun.  I was looking to replace the parallel LCD that comes with the Seeed Grove Starter Bundle with a serial LCD to free up some pins.

I hadn’t considered the upgradable firmware on this model as a big benefit – there isn’t much that you can do with an LCD beyond the basic set of commands that all serial LCDs support.  The kit actually incorporates an ATMega328 with the Arduino bootloader — a full Arduino, essentially.  The Arduino code reads the occasional LCD update command and passes it along to the LCD.  Between LCD updates, it sits about, twiddling its thumbs, waiting for something to do.  Hmmm…

Maybe we could use the Kit’s ATMega328 as an interface to the temperature sensor?  That’s so crazy, it might just work.

The Sparkfun Serial Enabled LCD Kit

And it does work.  (Most of the time, as I explain below.)

The Sparkfun Kit’s design makes it quite easy to implement this type of project.  It breaks out the standard 5 pins used by the FTDI cable (the serial-to-USB cable I previously used with the Light Sensor’s dearly departed UART interface), and also breaks out all of the Arduino pins not used by the LCD.

The LCD interface “firmware” is just a standard Arduino program, nothing weird, so it’s easy to merge in the sample code for reading the DHT11 sensor – mostly just copy and paste.

While I was at it, I also added some code to interface to the Arduino’s built-in EEPROM memory.

So, the end result is that the Plant Light Controller can now display the humidity, a more accurate temperature, and save the settings from the menu too.

Actually, the “more accurate” temperature is debatable.  The DHT11 chip always returns temperature as an integer, and always in Celcius.  It appears to truncate the actual temperature, rather than rounding it.  So, the temperature can be off by .9C, or 1.6F.  Humidity is similarly truncated to an integer.  But, hey, I’m not complaining – at $5, Seeed’s sensor is still a steal.

Pin Assignments

From the Netduino’s standpoint, not much changes from the previous installments of this project.  All communication with the Arduino goes through the same serial pins used by the LCD interface.  However, since the LCD now talks back, we need to connect a Netduino Serial Rx pin to the Serial LCD’s Kit TX pin, which is part of the FTDI breakout on the end of the backpack’s circuit board (the right end of the photo below).  My code uses Netduino’s COM2 (pins D2 and D3) for this.

Sparkfun Serial LCD and Seeed Temperature/Humidity Sensor

From the Serial Kit’s standpoint, the following pins are used:

-          The white JST plug contains the 5V, ground and RX pins, which we connect to the 5V, ground and TX (D3) pins of the Netduino

-          The TX pin at the end of the board is connected to the RX pin (D2) of the Netduino

-          The breakout pins at the bottom of the Serial Kit are used to connect to the Temperature/Humidity Sensor: specifically, the 5V, Ground and Analog 0 pins.

User Interface

Not much has changed here from the last installment of the project, either.  As shown in the photo below, I’ve used the leftover space on the Temperature row of the LCD to display the humidity instead of the Sunset time.

The number in parentheses is the temperature value from the sensor – the other temperature value is from the analog sensor, as added to the project in an earlier installment.  Which one is more accurate depends on how well your analog sensor is behaving – as mentioned, the digital sensor’s temperature can be off by 0.9C, while my analog sensor is apparently a little high.

Demonic LCD displaying strangely moderate temperature and humidity

I’ve also added a couple of new items to the “Timers” section of the menu, allowing you to disable the plant light timer.  This became necessary once I started saving the menu settings to EEPROM, since that ruined the handy Disable-the-timer-by-turning-the-gadget-off-and-on ™ feature.

Programmer Show And Tell

The Serial LCD Kit is now the 4^th LCD interface supported by the code, joining the parallel LCD, the Matrix Orbital Serial LCD, and the Sparkfun Serial Backpack.  I considered making the LCD Kit fully compatible with the Sparkfun Backpack’s command set by changing the Arduino “firmware” code.  The 2 command sets are quite similar, differing only when in the cursor positioning and backlight control commands.

However, I like the Serial LCD Kit’s API better, so I subclassed the Backpack’s clsLCD_SF class and overrode the commands that differed.

Subclassing is supported the same way in .Net Micro Framework as in the full blown .Net:

public class clsLCD_SFKit : clsLCD_SF

Overriding a class method is also done the standard C#, using the “new” keyword.  So, to replace the cursor positioning method with code that works with the LCD Kit’s firmware:

new public void SelectLine(byte intLine, bool blnClearLine)

I added some new commands to the LCD class, used to receive data from the Serial Kit’s Arduino.  Since its unlikely that the other LCDs are going to do much talking, I didn’t add these  methods to the common ILCD Interface.  Instead, when the main program wants to check to see if the LCD is of the Serial Kit variety, it uses the “is” keyword, again just like in the full .Net Framework:

if (lcd is clsLCD_SFKit)
{
// get the humidity
((clsLCD_SFKit)lcd).RequestHumidityAndTemp();

Data from the Serial Kit comes in through the Netduino’s serial RX pin, just like we used to read terminal commands through the UART interface.  So much like it, in fact, that I used the same event handler.  When a humidity reading or EEPROM setting comes back from the Serial Kit, the code raises a uart_Command event, as described  back in the Time and Weather article.

There is actually one big difference between the Serial Kit’s behaviour and the UART Telnet terminal, though.  The Telnet terminal could send us a command at any time, so an asynchronous interface using an event was necessary.  The LCD, on the other hand, sends data only when we ask for it.  It would have been possible, therefore, to read the data synchronously after we send the request, using code like the following:

public void RequestHumidityAndTemp()
{
byte[] command = new byte[2];
command[0] = CMD_SPECIAL;
command[1] = 71;
lcd.Write(command, 0, command.Length);
// give the LCD time to retrieve the data
Thread.Sleep(1000);
intBytesRead = lcd.Read(readBuffer, intBufferOffset, lcd.BytesToRead);
. . .
}

This has the advantage of making the context of the data we are reading obvious, but the disadvantage of introducing timing problems.  What if the LCD doesn’t respond fast enough, or the response isn’t received at all?  How long should we wait for the LCD to respond, and at what point does the Netduino decide that a response isn’t going to be received, and carries on?

Asynchronous communication does away with that problem.  In fact, you can unplug the LCD altogether, and the Netduino will faithfully carry on with its other duties, turning on and off the plant light.  It won’t receive any humidity readings, and its request to save menu settings won’t be performed (good luck using the menu without an LCD, by the way), but the Netduino doesn’t notice.

The downside of asynchronous communication is that data doesn’t necessarily come back when you want it.  You’ll see this when you first plug in the Netduino – although it requests the humidity from the Arduinio during startup, the response may not be received before the first “display the temperature” timer fires.  As a result, it may take another 30 seconds (the interval of the temperature timer) before the humidity is displayed.

The code I added to the Arduino to save settings to EEPROM takes advantage of the Arduino EEPROM library – the library makes it simple.  One minor twist is how to handle the very first interaction between Netduino and Arduino.  The Arduino has no idea whether it has any settings saved – it just returns whatever happens to be in the address you’ve requested.  The first time, this will be 0s.  So, all settings are saved as non-zero values: I add 1 to the values where necessary.

One other design decision worth mentioning is the issue of which settings to save, and when to save them.  I decided to save the date, but not the time, since it’s likely that the date will be unchanged, or very slightly changed, when you power the Netduino back on, but the time will almost certainly have changed, perhaps by hours.

The other downside of saving the time is that you’d have to do it every minute, putting wear and tear on the EEPROM.  It is best to write to EEPROM as little as possible: in this case, only when the user changes a setting in the menu.

Fortunately, I already had a callback method for responding to the user’s menu selections (as described in the Ordering Off the Menu article), so I added code like the following to that method.

static void setValueFromMenu(string menuCode, Int16 intValue)
{
. . .
if (menuCode == "_CELCIUS")
{
. . .
SaveSettingToEEPROM(ADDR_TEMP_FORMAT, 2); // 2 means Celcius

With Friends Like That…

This configuration actually did a good job of proving the point about unmanaged code: it’s awfully easy to crash the microcontroller when running it.

When I got things up and running, I found that the LCD would freeze at some point, generally a few hours after powering it on.  The Netduino soldiered on, faithfully turning the plant light off and on, so clearly it was the Arduino that dropped the ball.

Commenting out the code that requests the temperature and humidity reading caused the problem to go away, so whatever was going wrong involved the DHT11 interface.

The right thing to do would be to debug the Arduino’s DHT11 code, but the easy thing to do was to just “reboot” the Arduino on a regular basis.

Crude, but effective.  I’ve had this configuration running for over a month now, and it works like a charm.

The “rebooting” is done by using a transistor to turn the Arduino’s power off and on.  This requires a hardware change: instead of connecting the Sparkfun LCD Kit’s ground pin directly to the Netduino, I used a small breadboard to route the connection through a transistor.  The transistor is controlled using the D12 pin.

Specifically, the configuration I used (shown in the photo) is:

-          Collector pin of 2N2222 transistor to LCD Kit’s ground

-          Emitter pin of 2N2222 to Netduino ground

-          Base pin of 2N2222 to D12 pin of Netduino

-         Base pin of 2N2222 to ground through pushbutton – this can be used to manually boot the LCD Kit Arduino if necessary

Arduino Reset Circuit

The Arduino is powered up during the Netduino code’s startup:

arduino = new OutputPort(SecretLabs.NETMF.Hardware.Netduino.Pins.GPIO_PIN_D12, false);
arduino.Write(true); // turn on Arduino
Thread.Sleep(2000); // give Arduino a chance to do its initialization
lcd = new clsLCD_SFKit(SerialPorts.COM2, 9600, 2, 16);

I added a new method to reset the power to the LCD Kit’s Arduino by temporarily setting D12 low:

static void ToggleArduino()
{
arduino.Write(false);
Thread.Sleep(1000);
arduino.Write(true);
// let Arduino do its initialization
Thread.Sleep(2000);
// restore the brightness setting
lcd.SetBrightness(bytBrightness);

The new method is called when the Netduino hasn’t received a response from the Arduino for over an hour.  It asks for the temperature and humidity every 30 seconds, so an hour is obviously lenient.  If you need the temperature/humidity readings to be more reliable, you should shorten this interval:

// if we haven't received any data from the Arduino for over an hour, reboot it
if (datlastArduinoRead != DateTime.MinValue)
if (DateTime.Now.Subtract(datlastArduinoRead).Hours > 0
datlastArduinoRead = DateTime.MinValue; // prevent another reboot
ToggleArduino();

The voltage out of the Netduino’s digital pins is 5 volts, so why bother with the transistor – why not power the Arduino directly from D12?  The problem is current: the Netduino (and pretty much every microcontroller under the sun) only allows a trickle of current to flow through its pins.  You can power an LED with it, but anything more power hungry needs to get its voltage another way.

Wrapping Up

This installment of the Light Controller project is admittedly off on somewhat of a tangent: connecting a temperature sensor to a Netduino via a Serial LCD interface is not a common configuration.

However, offloading some of the Netduino’s responsibilities onto another processor is actually quite a handy technique.  There are a lot of microcontrollers out there, each with its own libraries of code capable of doing some things that a Netduino can’t (yet), so when looking to add a new feature it’s smart to think “outside the box”.

One other improvement I’ve made to the project is to put the source code into a proper repository, rather than just linking each article to a .zip file.  I’ve created a Google Code project named gigamega-micro.

You can get the code for each of my Netduino articles (starting with Let There Be Light) in 2 ways:

-          If you are familiar with Subversion use SVN to get the latest code.  If you want the older version of the code to match one of the past articles, get the corresponding revision (see the revision list here http://code.google.com/p/gigamega-micro/source/list)

-          Or, download the good old .zip file from the Downloads section of the Google Code project. Here is a direct link to the code for this article.

The modified Arduino code for the Sparkfun LCD Kit can also be found in the Downloads section of the Google Code page (direct link to .zip file).  90% of that code is copied from the original Sparkfun firmware or from Sheepdog Software’s DHT11 page.  I’ve left the original authors’ headers in the code.

Don’t worry, I haven’t forgotten the “road map” of hooking the Netduino up to an XBee.  We’ll get back on track in a future article.