Hardware Design and Construction Manual

Introduction

A CUWiNware network is a mesh of nodes running the CUWiNware software. A node consists of a computer, a wireless NIC, an enclosure, an antenna, and all the neccessary cable and mounting equipment. This manual will walk you through all the decisions that need to be made when purchasing and building hardware for a CUWiNware network.

Many of the decisions that you will make will be based on the local circumstances of your community. Topology, budget, available labor, available materials, network type, and network size will all be factors that go into your decisions. One of the biggest tradeoffs is between time and money -- for each option there is generally a way to do it cheaply that takes a lot of time and a way to do it quickly that is more expensive.

Supported Chipsets

One of the most important choices you face when building a CUWiNware node is what wireless NIC to buy. Wireless NICs use a variety of chipsets depending on the the manufacturer that makes them. The chipset that your wireless NIC uses must have drivers in NetBSD that support Ad-Hoc mode.

Theoretically different chipsets can be used in different nodes on a CUWiNware network as long as they are supported by NetBSD but because of various bugs/quirks in the firmwares of various chipsets, it is often best to standardize on a single supported chipset for all the nodes in your network.

Chipsets

• Intersil Prism - The prism (version 2, 2.5, or 3) chipset is used in some PCMCIA and PCI cards manufactured by D-Link, Proxim, SMC, and others. This chipset is well supported in Ad-Hoc mode with no real bells and whistles. This may be the best chipset available if low cost PCI cards are needed for low-resource nodes.
• Atheros - Atheros makes the most advanced 802.11x NICs currently available. This is the chipset currently recommended by the CUWiN team. Most tri-mode a/b/g cards on the market use the Atheros chipset. The Atheros chipset features a Hardware Abstraction Layer which puts most of what would traditionally be in the firmware into software. This means that when/if the interface to the cards is fully opened up to programmers, arbitrary control of power levels and other low level features will be possible. Because future versions of CUWiNware will take advantage of this fine grained control we recommend investing in cards with this chipset now. Atheros cards may be a bit more expensive than some of their competitors and the drivers for the Atheros cards are not as mature as those for the Prism cards so there may be bugs, but these drivers are being actively developed and debugged by the CUWiN team.
• Hermes - Most cards made by Orinoco, Lucent, WaveLAN, and Proxim use the Hermes chipset. These cards have very good sensitivity with low power radios and they are supported in NetBSD but they suffer from serious firmware problems relating to their IBSS ids. The IBSS id needs to be the same throughout the mesh but cards with this chipset often "split" the IBSS in a way that can only be fixed by rebooting all nodes on the network in a certain order. This chipset should generally be avoided for any Ad-Hoc network as just one Hermes chipset mixed in with other nodes can cause a serious network split. Firmware upgrades do not appear to completely solve this problem. Aironet/Cisco chipsets have the same problem.
• Other Chipsets - Development of drivers for new chipsets is constantly in progress. Some manufacturers publish the specs for their hardware which makes driver development fairly straightforward. Other manufacturers keep their specifications proprietary which means that NetBSD developers have to reverse engineer the interfaces. Support for the ADMTek ADM8211A is available now and support for other ADMTek chipsets such as ADMtek ADM8211C/CR is expected soon. Support for the Realtek RTL8180 chipset is in progress.

Links

Constructing a Rugged Outdoor Node

One of the biggest sources of signal loss in a wireless network is in the cable between the low power radio in each node and the antenna. High quality antenna cable is very expensive. In order to maximize signal strength and to minimize cost, it is best to put the node as close to its antenna as possible. This generally means putting the node outdoors on the rooftop on the the same mounting pole as the node itself.

Outdoor nodes need to have no moving parts, withstand extreme temperatures, withstand rain, snow, wind, sand, and other local climate conditions and be upgradable without physical access.

The Single Board Computer

The first component of any outdoor node is the single board computer. This is a small form factor motherboard built for embedded computing applications. Typically a single board computer will be built to industrial standards and will be much more rugged than a typical consumer motherboard.

Typically a single board computer will have a processor (CPU), some memory, some compact flash storage (or a slot for it), one or two Mini-PCI or PCMCIA slots, one or more ethernet jacks, a serial port, and an input for DC power.

In order to run current versions of CUWiNware you will need a single board computer with an x86 CPU, preferably 486 or better running at 100Mhz or better. It will need at least 64MB of RAM and 64MB of compact flash.

The 64MB of compact flash is neccessary to contain two full versions of the software which is currently around 32MB in size. The space for the second image is used for on-line upgrades of the node. It may be possible to build custom versions of CUWiNware with on-line upgrades disabled that would only require 32MB of compact flash.

Make sure there is an enclosure available for the board you choose!

We recommend Soekris Engineering's net4526 board which has 1 ethernet interface, 2 Mini-PCI slots, 64MB of on-board compact flash, and 64MB of on-board RAM. This board is small, powerful, rugged, and has good enclosures available.

Links

The Wireless NIC

The three requirements for a wireless NIC are that it must have an external antenna connector, it must be compatible with the motherboard you have chosen, and it must use a chipset that is compatible with NetBSD in Ad-Hoc mode.

Interfaces between the motherboard and the wireless NIC include: Mini-PCI, PCI, PCMCIA (a.k.a. Cardbus or PC Card), and USB. Most rugged single board computers will use either PCMCIA or Mini-PCI cards.

Make sure there is a pigtail available for the card you choose!

We recommend Netgate's NL-3054MP Aries Mini-PCI 802.11b/g or SL-5354MP Mini-PCI 802.11a/b/g cards. These use the Atheros Chipset.

Links

Enclosure

The enclosure you choose must allow your single board computer to be mounted. Many enclosures come with a special mounting plate that the manufacturer can custom drill to work with any single board computer that you specify. Many manufacturers make enclosures specifically for Soekris boards. The enclosure must have holes for an ethernet cable gland and and an N connector for the antenna connection.

Any enclosure for an outdoor wireless node should be rated at least NEMA Type 4x. Type 4X NEMA enclosures protect against falling dirt, rain, snow, blown dust/sand, splashing water, and ice.

The enclosure should be white in order to reflect rather than absorb external heat from sunlight. If the enclosure is plastic, it must be a type of plastic that doesn't break down when exposed to UV light.

It is vitally important that any and all holes on the enclosure be on the side that faces downwards. No matter how well sealed the hole, with mastic tape or with a cable gland and gasket, if it is on top water will pool around it and freezing/thawing will allow the water to seep in. After months or years of operation the node will fill with water and fail.

It is best not to mount the antenna directly onto the enclosure but to mount both the enclosure and the antenna on a mast. Antennas directly mounted on the enclosure put a huge amount of stress on the joint between the antenna and the enclosure and this can cause leaks.

There is some debate among community networkers as to the utility of a drain hole in the bottom of your node enclosure. If you trust that your node will remain absolutely water tight even after months or years of freezing and thawing then it may be best to not include a drain hole. A drain hole is for water that does get into the enclosure due to condensation/humidity or due to leaks. A dessicant package placed in a sealed node may do a better job of defeating small amounts of moisture. A drain hole may be susceptible to sand or dust in a desert environment and would definitely not be recommended in that case.

A very labor intensive but low material cost way to build an enclosure is to use a surplus military ammunition can painted white with a custom plexiglass mounting assembly. These cans are built to be very rugged and waterproof and are essentially free if you can find a source of them. They do, however, corrode after prolonged exposure to the elements and will eventually fail.

We recommend the NEMA-4x enclosures sold by Metrix Communications for use with Soekris 4526 boards.

Links

Antenna/Cable

Antennas must be made for the band that you are using: 2.4GHz (802.11b/g) or 5.8GHz (802.11a). They should use an N connector for the cable connection. The N connector is most appropriate for the high frequency signals used by 802.11x.

Choices in antenna selection include: omni vs. directional, manufactured vs. homebrew, and amount of gain.

Typically mesh nodes will use omni-directional antennas. This is recommended for most cases. These transmit and receive equally in any direction towards the horizon. This allows the formation of arbitrary meshes without having to aim (or re-aim after a high wind) antennas. It also means that there may be "wasted" radio energy that causes interference for nearby nodes.

You may want to choose a directional (sector or beam) antenna if you are creating a long distance point-to-point link within the mesh or if you know for sure that no nodes will ever be placed on the null side of the directional antenna. Sector antennas focus all radio energy in an area of a wide angle (e.g. 60 or 120 degrees). Beam antennas focus radio energy into an even tighter beam. Directional antennas come in many types such as Yagi, Sector, Dish, and Patch and Panel.

In a very dense network the interference caused by nearby nodes using omni-directional antennas can cause a major drop in throughput. This problem can be addressed by replacing key omni nodes with multiple nodes with sector antennas. Collectively the nodes form 360 degree coverage and are linked via their wired ethernet cables.

There are many designs on the internet for homebrew antennas that you can make from common household items and cheap hardware using simple tools. The antennas can be pretty time intensive to build and their quality will vary greatly. The easiest designs are for directional beam antennas. Homebrew omnidirectional antennas are very difficult to build.

Every antenna will include a "gain" rating. This is a measure of how much the antenna increases the signal power in the direction that the antenna points. For omni-directional antennas a high gain means that less power is radiated "up" or "down" and more power is radiated "outwards". For a directional antenna a high gain means less power is radiated out the "back" and more out the "front" and higher directional gain also means a tighter beam. Gain is logarithmic so every increase of 3dB gain means a doubling of power.

The best antenna cable to use for microwave applications such as 802.11x is LMR-400. The antenna cable is a major source of signal loss so it is very important to use only short runs of high quality cable with N connector ends. Installing your own ends on raw cable is very time consuming and requires special tools.

For most generic mesh applications we recommend the HyperLink Technologies HGV-2409U 2.4GHz 8dbi omnidirectional antenna. This antenna features a flared base which makes it less susceptible to windshear than other similar antennas from other manufacturers.

Links

Mast/Mounting Hardware

Microwave signals are extremely dependent on line of site. The higher your antenna can be the more likely other nodes are to have line of sight to it.

Generally, you can use an antenna mast on a rooftop to acheieve height. If you live in a tall building you may be able to just place the node in a window. If you have access to a radio tower then you can acheive the best heights.

There are several ways to mount an antenna mast on a rooftop. For flat roofs there is a great non-destructive flat roof mounting platform that is held in place by cinderblocks. There are special mounts for gables and chimneys, as well as tripod mounts.

Any hardware designed for television antenna mounting will suffice for a mesh node antenna.

We recommend the Radio Shack ratchet style Chimney Mount (cat no 15-839) and any 5 foot antenna mast typically sold for television antennas.

Grounding/Lightning Arrest

Tall metal polls on your rooftop can be a major fire hazard as well as a risk to all electronic equipment connected to the node in any way. It is important to follow proper grounding guidelines to protect your safety and your equipment.

Lightning can travel down either of the 2 conductors in the antenna cable and they can't both be directly connected to ground. In order to protect this signal from lightning you need to install an inline gas-discharge lightning arrestor and connect that to ground.

See the ARRRL Antenna Handbook for tips on grounding antennas.

Outdoor CAT-5 Cable and PoE Injector

The cable from the node into the user's house is a CAT-5 ethernet cable which also carries the node's power. Although it is more expensive, it is important to get special outdoor CAT-5 cable. The outdoor cable's jacket will not break down in UV light and it is filled with a waterproof gel that prevents condensation-related corrosion of the conductors.

Constructing a Low-Resource Node

While rooftop nodes are more managable, more efficient, more durable, and more reliable, their cost of a few hundreds of dollars can be prohibitive to many communities. Community networks built on a budget may wish to make use of recycled desktop computers. This is possible with CUWiNware's CD (ISO) image.

Typically a low resource node is placed in an attic to minimize the cable run to an antenna on the rooftop. A low resource node may also be placed near a window with an antenna in a tall building or a building very close to another CUWiNware node.

Most hardware issues from the Rugged Outdoor node apply to the indoor low resource node. Here are notes on additional considerations.

Motherboard/CPU Considerations

While the CUWiNware software will operate on most 486 or better platforms, most old consumer-grade 486 and original Pentium desktop machines have such old motherboards that PCI Wireless NICs or PCI/PCMCIA bridge cards will not function properly. Check to see if your wireless card requires PCI 2.0 and, if so, whether the motherboard of the desktop you are using has a PCI 2.0 compliant motherboard.

Your options for a wireless NIC are to use a PCI wireless card, a USB wireless card, or a PCI/PCMCIA bridge card with a PCMCIA wireless NIC. It is probably easiest and cheapest to find a compatible PCI card.

It is easiest if the BIOS on your low resource node supports booting from CD-ROM. Some very old motherboards do not support this. If you can't boot from CD-ROM, it is possible to bootstrap with a special floppy disk which will then hand over control to the CD image.

An attic-based node will probably not have a keyboard attached to it. You'll want to disable the requirement in the BIOS that a keyboard be attached. Otherwise everytime the node is turned off and back on it will hang on boot while it waits for a keyboard to be connected.

Other Hardware

A low resource node has many moving parts, this is what makes it less reliable.

You will need a CD-ROM drive, optionally a floppy drive (if you can only boot from floppy or if you want the node to remember configuration changes across reboots), an wired ethernet NIC, and a wireless NIC. There is no need for a keyboard, monitor, mouse, or harddrive.

If you wish to reduce the amount of moving parts you can install an IDE compact flash drive instead of a CD-ROM and Floppy.

Console

In order to interact directly with the node you will need a serial console. Some messages may show up on a monitor connected to the node but the only way to interact with the node is not through the keyboard but through a terminal program on the serial port. The settings are 19200 N81.

You will however need to use a monitor and keyboard to access the BIOS settings when you first set up the node.

Antenna Options

In addition to the high gain antennas that are typically used with outdoor nodes, it is possible for an indoor node that is very close to another CUWiNware node to simply use the low gain omni-directional "rubber duck" antenna that comes with the wireless card, especially if it can be put in a window facing the other nearby node.

Other Node Possibilities

There are many other possible form-factors for CUWiNware nodes. These are currently untried but an adventurous community network designer could easily experiment with these possibilities.

• An old laptop or tablet PC could be converted into a node. It already has the PCMCIA slots and a small form factor. Because it is battery powered you could create a truely ad-hoc on-the-spot network anywhere if you had several of them.
• It is also possible that a laptop running the HSLS daemon on top of it's regular operating system could be a sort of "roaming" node on the mesh.
• The dream of cheap mesh nodes could be realized by flashing Access Points with CUWiNware. In order to do this CUWiNware will need to be stripped down to operate under MUCH tighter constraints but we believe this may be possible.
• CUWiNware could be installed on a harddrive in a node for R&D purposes. Then it'd be easier to tweak the contents of the running software without having to build and burn a new CD or compact flash image.

User's Guide

The CUWiNware package provides a completely self-contained self-configuring bootable disk image for creating mesh wireless nodes for community wireless networks. The CUWiN developers distribute pre-built generic disk images for common disk types on the x86 architecture. Advanced users can also custom build a CUWiNware disk image from source code.

Getting an Image

All images are distributed compressed with gzip. ISO and Compact Flash images will need to be uncompressed before they are copied onto media.

ISO

ISO images are meant to be burned to CD-R which will then be bootable. Use your CD-R burning software to burn the disk. If the target platform can not boot from the CD-ROM drive then you will have to create a boot floppy. The CD-ROM you create will contain a file called /cuwboot/boot.img.

The command to copy the boot.img file to a floppy disk in NetBSD is:

dd if=boot.img of=/dev/fd0a bs=1024 count=1440

Compact Flash Images (img)

Compact Flash images must be distributed according to the geometry of the compact flash device for which they are targetted. We distribute official images for SanDisk 64MB and for SanDisk 126MB Compact Flash cards. If you have a different compact flash card you will have to do a custom build from source code. It is important to note that the Soekris net4526 board which has 64MB of compact flash on board is NOT compatible with the SanDisk 64MB image. If you install a SanDisk 64MB image on a Soekris net4526 it will be bootable but it will not be online-upgradable.

The command to copy the staboot.img file to compact flash is in NetBSD is:

dd if=staboot.img of=/dev/rwd0d bs=8k

These files are just gzipped tars of the contents of the disk image. They are geometry independent. These tar files can be loaded by a special updater program on the compact flash versions of the software. See below for instructions on upgrading an already flashed node.

PXE Booting

A brand new net4526 node has to be flashed with an initial version of the software. This is done by PXE booting the node, logging into it, and executing a dd to it's flash drive. PXE is a protocol for booting remotely over ethernet.

On-Line Upgrades

If a compact flash node is bootable then it should be possible to do an on-line upgrade without a PXE boot server. This can be done over the wire or over the air.

Log into the node, become root, and run the script:

/sbin/upgrade user@host:/path/to/upgrade.tgz

You will need to have the .tgz file on a host that is running sshd and is accessible to the node. Luckily sshd is available for all major platforms, including Windows (with Cygwin) so it is easy to turn any laptop into a roving upgrade platform.

Configuring a Node

Compact flash nodes and CD-ROM nodes with a floppy drive installed can be configured via a web-interface by visiting http://node.ip.addr/ and clicking on "Config".

Configurable values are:

channel

This can be any valid channel for the protocol being used. For 802.11b this is any integer between 1 and 11. Every node in a given mesh must be on the same channel.

wired interface

This can be set to "auto", "dhcpd", or "dhclient". Depending on the setting the node will either act as a DHCP client or a DHCP server. If it is a DHCP client then it will act as an "internet connected node". The default is "auto".

hostname

This is the name by which this host identifies itself.

latitude, longitude

The latitude and longitude for use by the vizualization engine. Do not set these unless you have also custom configured RouteViz with a local map and it's latlong corners.

nameservers

A comma separated list of nameservers that will be used locally and given out to DHCP clients to use for their lookups. You should choose something local to your ISP or a public DNS server.

ssid

A unique identifier for the whole mesh cloud, this must be the same one all machines that wish to be part of the mesh.

Configuring a node creates the file /etc/cuw_config on a node. That file which is simply a flat file containing variable assignments. You can untar an upgrade image, replace it's cuw_config file, then retar it and then apply it as an upgrade. This is an easy way to create a localized distribution from the standard builds.

Vizualization Guide

When you click on "RouteViz" you will be shown a virtual map of the network as the current node sees it. This map is made from a dump of HSLSd's adjacency database.

Unless you customize the installation of CUWiNware with a special local map file for the area around your community network, the layout of nodes will simply be what "looks good" and won't have much to do with where the nodes really are. It will reflect how they are connected though.

Click on a node to get information about it.

The links between nodes show the current link quality as measured by ETX. Red links should be investigated as possible trouble spots on the network.

Troubleshooting Guide

The vizualization tool is the best way to start troubleshooting.

From the viz page you can test the connection between you and the node. You can also test the connectivity along the route to the internet. If the local network has packet loss then the problem is probably with your Wireless Access Point or your LAN rather than with the CUWiN wireless mesh network. If your path to the internet has packet loss then it is time to investiate the viz map for bad links. If you can't access the software at all then either the node is down or you have found a but in the node code.

Building CUWiN Software

You can build your own CUWiNware images from the CUWiN sources with your local modifications included. Local modifications that you can make include setting up a default config file, changing the default root password and adding default local users, adding public key authentication for login to the nodes. You can also build against newer NetBSD sources to update the available kernel drivers and if you are a programmer you can freely modify the code as you see fit.

In order to build the software you will need a working NetBSD build environment. Once you have that you will need both the CUWiN sources and an appropriate NetBSD source snapshot.

HSLS/ETX Documentation

HSLS

HSLS is the routing protocol that has been developed by CUWiN based on a technical paper from BBN. This routing protocol is specially designed to be scalable for ad-hoc mesh wireless networks. The HSLS daemon communicates with the kernel's routing tables via the Zebra daemon which provides some future portability to other operating systems.

ETX

ETX is a metric used with the HSLS protocol specification. Our HSLS daemon loads arbitrary metric modules on the command line and libetx.so.0 is the most advanced such module currently available.