SAM7-EX256 Pingable!

2008-03-23 23:00:00

Today I got FreeRTOS running on my SAM7-EX256 board. And the best part? I can ping it: 

PING 10.0.1.37 (10.0.1.37): 56 data bytes
64 bytes from 10.0.1.37: icmp_seq=0 ttl=64 time=0.241 ms
64 bytes from 10.0.1.37: icmp_seq=1 ttl=64 time=0.268 ms
64 bytes from 10.0.1.37: icmp_seq=2 ttl=64 time=0.253 ms
--- 10.0.1.37 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.241/0.254/0.268/0.011 ms

It answers HTTP requests on port 80 too. FreeRTOS is damn cool. I can even change the stat of one of the LEDs by clicking on a button on the webpages.

However, this did take some doing, so I'll go into what I had to do to get it to work on the SAM7-EX256. I was able to get both ARM7_AT91SAM7X256_Eclipse and lwIP_Demo_Rowley_ARM7 to work and the patches below address both of them.

The first bit of trouble I ran across was that the Makefile wanted to build some of the code in Thumb mode. This was problematic because the libraries I was linking against didn't support "interworking" mode and the compiler complained. I simply disabled the Thumb compilation for now. Perhaps later I will need to enable it to gain some space back, but for now it's gone.

Once that was working I had to adjust which IO pins various things were using. In particular the FreeRTOS demo toggles four IO lines to blink LEDs as a simple way to show status of the system. The orginal IO lines were PB19-22 which were partially in use by the SAM7-EX256. PB19 controls the speaker and PB20 controls the backlight for the OLED display. I'm pointing this out because this was actually my first indication that the code was at least somewhat working -- the backlight flashed and the speaker made an annoying clicking sound. I remapped the status LEDs to PB27-30 and hooked up some LEDs and resistors to the lines.

Next I needed to adjust the clock rate from 47923200 to 48054857. I also tried to remap the USB demo code to use the joystick on the SAM7-EX256 by mapping those buttons to the appropriate IO lines PA7-19, 14, 15. The board was recognized by my Mac but it did not function as a mouse. I'll probably revisit that eventually.

Now for the fun stuff. I had to set the IP address which was easy enough to do but things still didn't work. I noticed that the LEDs on the ethernet controller were blinking in a pattern that led me to believe the controller was getting continuously reinitialized. At this point I remembered that I had seen a post in the SparkFun forums about the PHY on the SAM7-EX256 being different than the one on the ATMEL dev board. After poking around for awhile I found the appropriate place to update this and Voila! I was able to ping my board.

You can grab my patches and give them a whirl if you are so inclined. These patches are against the FreeRTOS 4.7.2 release ZIP file.

The next steps are to get the serial port up and running. Craig and I are working together to build a NTP Clock Controller that will use an SAM7 chip. He's posted a lot of good information as we've been working on this project.

If I have the gumption I may see if I can get Plan 9 running on this board some day. But for now FreeRTOS will have to do.

Iperf 2.0.3 Release

2008-03-18 23:00:00

I am happy to annouce that Iperf 2.0.3 has been released. Here are the release notes:

Iperf 2.0.3 addresses several issues with Iperf. This release is a bug fix release and does not add any significant new functionality.

Bugs fixed in 2.0.3:

  • Compilation issues on several platforms, notably Mac OS X and Solaris
  • Excessive usage of CPU due to updating stats for every TCP packet received
  • several minor nits, see the ChangeLog for details.

These changes come primarily from user patches and have only been lightly tested. Special thanks to Andrew Gallatin for his work on making the current threading model better.

The code is available on SourceForge.

Crossing Boundaries

2008-03-17 23:00:00

It is crucial to our development as both individual human beings and as communities of human beings that we be willing to cross boundaries.

Each one of us has our own unique and subjective view of reality. While this view is widely held by educated people and sometimes by uneducated people it is too rarely practiced as fundamental to engaging in day to day life. Nearly all of us assume that what we perceive, experience and feel as we go through the day is the same as everyone else or that we understand how others perceive existence. This assumption leads to profound misunderstandings and often to grave wounds. This assumption is also a great pacifing force in society. The topic of subjective reality is far too big to delve into in any depth right now, but I intend to explore it further as I write.

Before I continue, I would like to make one thing plain: although I write these entries in a tone that suggests what I say is true it is possible that it is only partially true or perhaps even wholly untrue. I encourage you to challenge me and in doing so we can help one another attain deeper understanding.

Crossing boundaries is the deliberate choice to attempt to understand someone else's subjective reality or to share your own subjective reality with them. To put it another way, it is choosing to receive ideas and/or share ideas with another person. It goes beyond a literal exchange of information and promotes a deeper understanding.

It is surprisingly difficult to do because our frame of reference is ourselves and that continually gets in the way. Here is an experiment that you can try that will demonstrate this difficulty. In the very next conversation that you have with someone make a conscious choice to only listen to what they are saying and do not formulate your answer until they are done. Consider their tone and their choice of words. Do they mean something particular by each word? Were you able to let them finish before you started your rebuttal?

I had had at least three interactions on my journey back from PyCon that were crossing boundaries. The first was the conversation I had with the woman who sat next to me on the plane. It started as just an exchange of information but after learning that she had studied history and was involved in the interpretation of the exhibits at her musuem I decide to explain the concept of Seattle Saturday House. This led to a long interaction about subjective reality, community, and sociatal ownership (or "social responsibility" as she referred to it.) While I know our communication was imperfect I believe we both came away encouraged and hopeful because we had participated in real interaction in the present. The second interaction was sparked by the first as the person next to her joined the conversation. There was some simple information exchange but there also was some deeper communication going on.

In these examples crossing boundaries provided a way for the subjective realities of three individuals to overlap for a time and be influenced by one another. This is a profound event. So much of our lives is spent in the literal even though a fairly small gamble may lead us into something far more profound. Making the deliberate choice to cross the boundary from our subjective reality into someone else's is one of the foundations for a vibrant community.

The final interaction I want to talk about was with the cab driver on the way home. I was tired, my flight was late and I was just ready to be home after ten days on the road. However, when I got in the car I immediatly had a sense that my cab driver had an outgoing demeanor and if I talked to him a little bit he would probably open up quite a bit. He described a lot of things I had never considered about being a cab driver. For example if a crime was committed in his cab (such as a drug deal) he was resposible to report it to the authorities or else he could be held liable for it as well. This is a hard place to be if you think about it. He went on to describe some very bad experiences he had had with african american passengers ditching out on paying for their cab rides. What surprised me was that although the cab driver was a minority himself he had generalized this behaviour to decide that Obama was not a worthy candidate for president.

This is in my mind a failure in crossing boundaries. He neglected to look beyond those bad experiences and refused to engage with the person of Obama because of the color of his skin. I should note that it is possible to cross boundaries without being in person but it is generally not as profound.

As societies or communities we tend to believe that we understand other societies or communities but unless we actively interact with them we do not understand. It takes effort go get beyond ourselves and to really understand other people, other communities and other societies.

It is easier and more comfortable to remain in our own subjective reality and to assume that our crystalized snapshot of the others we know and our assumptions about those we don't know are true. It is easier and more comfortable but it is not fulfilling. Fulfillment requires work, requires action and it is never found in what is simply comfortable.

I know that I do not cross these boundaries as often as I could. It is a personal goal to grow in this area. There is a fundamental truth to realize too: all people want to be understood and known and if done with sensitivity nearly everyone will respond positively.

Thrift Lightning Talk for PyCon 2008

2008-03-12 00:00:00

Thrift provides lightweight, cross language RPC by generating code for each target language from a simple definition file. There are lots of RPC mechanisms out there, what's so interesting about Thrift? Good question.

Thrift is easy to use and Thrift in any given language "feels right" for that language. Using Thrift in Python feels Pythonic. Using it in C++ feels, errr, uhm, C++ish?

Thrift ships with a binary protocol to transport the data so it's lightweight and efficient. It doesn't try to solve all the problems of the world, it just tries to do one thing well. The architecture is very clean and allows changing the behaviour at any layer in the RPC stack. This includes the Protocol layer which encodes the data structures and the Transport layer which is responsible for moving the data from point A to Z.

You can choose the language that is most appropriate for the component and the component can live on the most appropriate system. For example, the search component of your site may need to troll through a huge amount of data so writing it in C++ and running it on a search cluster may be the best choice. This seach component can be easily called from your the TurboCherryDjangoLons application running on your web server farm. Perhaps you need Geolocation but would prefer not to burn a ton of RAM on each of your web nodes you can use Thrift to talk to a remote Geolocation service which is written in Python. This leads to my example.

Here we have a very simple Thrift definition file: 

struct SpiffyGeoIPLocation {
   1: string country_code,
   2: double latitude,
   3: double longitude,
   4: list<string> possible_cities
}

service SpiffyGeoIPService {
   SpiffyGeoIPLocation by_ip_addr(string ip_addr);
}

It defines a structure called SpiffyGeoIPLocation. This struct represents the answer we will recieve from our GeoLocation service. The definition file also defines a service called SpiffGeoIPService which exposes a single procedure call: by_ip_addr which takes in a string representing the dotted quad of the IP address.

Once we have the definition file we run the Thrift compiler which will generate the code for the structure and the service in each of the languages we request. In the case of Python this will be a pair of class definitions. There will be a class which represents the structure and one that represents the client interface. It also generates a simple client script that will allow you to make RPC calls to the service from your shell. This is very handy for testing.

Here is the implementation of this service in Python: 

import sys
sys.path.insert(0, 'gen-py')
sys.path.insert(0, 'thrift-trunk/lib/py/build/lib.macosx-10.4-i386-2.5')
from thrift.transport.TSocket import TServerSocket
from thrift.transport.TTransport import TBufferedTransportFactory
from thrift.protocol.TBinaryProtocol import TBinaryProtocolFactory
from thrift.server.TServer import TThreadedServer
from simple_example import SpiffyGeoIPService
from simple_example.ttypes import SpiffyGeoIPLocation

class SpiffyGeoIPHandler:
     def by_ip_addr(self, ip_addr):
         return SpiffyGeoIPLocation(dict(country_code='us',
             latitude=41.90, longitude=87.65,
             possible_cities=['Chicago, IL', 'Rosemont, IL']))

processor = SpiffyGeoIPService.Processor(SpiffyGeoIPHandler())
transport = TServerSocket(3773)
tfactory = TBufferedTransportFactory()
pfactory = TBinaryProtocolFactory()
server = TThreadedServer(processor, transport, tfactory, pfactory)

server.serve()

As you can see we set up some Thrift infrastructure and define a handler class which implements by_ip_addr procedure as a method. It is this handler class where you put the code to perform the desired task.

The code to make the call is equally simple: 

import sys
sys.path.insert(0, 'gen-py')
sys.path.insert(0, 'thrift-trunk/lib/py/build/lib.macosx-10.4-i386-2.5')
from thrift.transport.TSocket import TSocket
from thrift.transport.TTransport import TBufferedTransport
from thrift.protocol.TBinaryProtocol import TBinaryProtocol
from simple_example import SpiffyGeoIPService

socket = TSocket('localhost', 3773)
transport = TBufferedTransport(socket)
protocol = TBinaryProtocol(transport)
geoip_service = SpiffyGeoIPService.Client(protocol)
transport.open()

where = geoip_service.by_ip_addr("192.0.2.37")

print where.country_code, where.latitude, where.longitude, where.possible_cities
for city in where.possible_cities:
   print city

As you can see we get back an object which is an instance of the SpiffyGeoIPLocation class which has all the fields defined in the structure definition in the first slide. It's acts just like a Python object because it is a Python object.

Why do I like Thrift so much? It's made distributing the components of my current project at my day job very easy. This is good because I have to run two instances of my system for redundancy, one on the east coast and one on the west. Each one will periodically backfill holes in it's database with the other coast. Most of the code in other parts of our system are in Perl. Using Thrift I have been able to use the data generated by my Python system painlessly in Perl parts of the system. Also, I may need to optimize the speed of my system, with Thrift it would be pretty easy to rewrite the slow components in C++. I'm repeating myself here, but think about this -- you can easily integrate new code with old systems in other languages. This frees you from having to write the whole system in one language.

If you'd like to play with these examples, feel free to grab the code.

We are live!

2008-03-03 00:50:06

The new site is live and looking pretty good.

Now I just need to finish adding all the features I have planned in my head.