And it worked, sort of. It gave me land forms that come close to representing the actual terrain. In hindsight it’s pretty obvious how distorted they were, but for the first few iterations of the program they were good enough.

I’ve now started looking into representing paths with more accuracy. At the moment Elevation’s speed scale is totally relative to the data, but I’d like to peg actual km/h values to it. I’d also like to indicate the scale of the map with sliding meter/kilometer markers that adjust based on zoom level. As I started going down this road I quickly realized the math needed a closer look.

Without a clue where to begin, I turned to Wikipedia and various open source Java/Javascript libraries to see if they could offer a clue about converting my GPS coordinates into a proper grid. Long story short, what I needed was a map projection system.

Mercator is probably the most familiar projection out there, and it’s the same one Google, Yahoo, Microsoft etc. use for their online mapping services. While it seriously distorts extreme high and low latitudes (Greenland isn’t really the same size as Africa, after all), it has the advantage of treating smaller local areas more uniformly. Mercator won’t work at the North Pole or in the Antarctic, but at a regional level like city or state/province it’s a fairly uniform distortion so the maps just look right; since Elevation is intended for those smaller areas, that’s perfect for my needs.

So, how does one go about converting GPS lat/long coordinates to actual meters? That’s where the math comes in, and Wikipedia handily lists a few formulas for it. Processing doesn’t have sinh, cosh, or sec functions, so only the first two functions for y will work; I chose the second:

x = λ - λo

y = 0.5 * ln * ((1 + sin(φ)) / (1 - sin(φ)))

The x value basically ends up being your raw longitude coordinate, though it’s handy to offset it from the central x axis of your map. The y value requires a bit more heavy lifting. In Processing, the above functions end up looking something like this:

float lambda = gpsLong - (mapWidth / 2);

float phi = radians(gpsLat);
float adjustedPhi = degrees(0.5 * log((1 + sin(phi)) / (1 - sin(phi))));

While Wikipedia doesn’t explicitly say anything about radians, I found it necessary to convert y before the math would work in Processing, then convert the result back to degrees.

The resulting values still don’t represent real units on the surface of the globe; to get that you have to multiply each by the degree length, a variable value that corresponds to the surface distance between a single degree change in latitude. At the equator there are approximately 111km between degrees, but this number shrinks the closer you get to the poles.

Solving this problem is something I’m currently working on. An unfortunate red herring was that the degree length in Vancouver appears to be almost exactly 1.6 times less than the value at the equator, which looked for all the world like a flubbed imperial/metric conversion. If I were to start with the equator distance and divide by 1.6:

x = (adjustedPhi * 111319.9) / 1.6;

y =(lambda * 111319.9) / 1.6;

I would get values that look fairly darn close to accurate in Vancouver. I thought that had solved it, but I now suspect the math at different latitudes is still very much wrong. I’ll update this post once I know more.

Update: Thanks Microsoft, for publishing this piece on MSDN about how your Bing Maps tile system works. This bit in particular seems like it contains what I’m looking for:

The ground resolution indicates the distance on the ground that’s represented by a single pixel in the map. For example, at a ground resolution of 10 meters/pixel, each pixel represents a ground distance of 10 meters. The ground resolution varies depending on the level of detail and the latitude at which it’s measured. Using an earth radius of 6378137 meters, the ground resolution (in meters per pixel) can be calculated as:

ground resolution = cos(latitude * pi/180) * earth circumference / map width

= (cos(latitude * pi/180) * 2 * pi * 6378137 meters) / (256 * 2^level pixels)

The map scale indicates the ratio between map distance and ground distance, when measured in the same units. For instance, at a map scale of 1 : 100,000, each inch on the map represents a ground distance of 100,000 inches. Like the ground resolution, the map scale varies with the level of detail and the latitude of measurement. It can be calculated from the ground resolution as follows, given the screen resolution in dots per inch, typically 96 dpi:

map scale = 1 : ground resolution * screen dpi / 0.0254 meters/inch

= 1 : (cos(latitude * pi/180) * 2 * pi * 6378137 * screen dpi)
         / (256 * 2^level * 0.0254)

Let’s back up about 13 years for a second; in the mid 90’s I spent a lot of time cutting my teeth on programming in Basic, a popular beginner’s languages at the time. I was building fairly simple GUIs for applications and games, but even those were taxing what little processor speed I had available. Back then I stumbled across what I’d imagine to be a fairly core tenet of any sort of programming that involves drawing to the screen: don’t, if you can help it.

When you’re just starting out, you don’t really consider the work involved in throwing thousands of pixels at the screen every iteration, so you default to simply redrawing the screen during every cycle of your application loop. In the 90’s this was significantly more of a problem since the CPU just couldn’t keep up, and we had to learn to selectively redraw only when it was necessary. These days the processor speed is less of a barrier, so the temptation is there to use the extra power we have available. This is fine for learning, but when you’re producing something people are going to use, your application needs to be more responsible about how much of the CPU’s time it actually needs.

By the very nature of the draw loop, Processing isn’t really set up for optimization. The default mode is simply to draw, and redraw, and keep doing it over and over again. When you can take it for granted that the screen will be wiped every cycle, you don’t necessarily have to think about things like cleaning up old pixels before placing new ones. It’s convenient, but it comes at a price. Currently an instance of Elevation will run at something like 99% of CPU time, all the time, until it’s closed. This is a problem.

So last night I started digging into the problem to see if I couldn’t knock that down a little. The updated code is only available from the GitHub repo so far, but it’ll likely be rolled into the next version of the app.

I started by adding a global refresh variable that gets set to false, and then wrapped most of the draw loop contents in an if statement that checks to see if it’s been switched to true. Then just before the loop closes, I make sure to reset it back to false so the next loop won’t also redraw.

void draw() {

  if (scene.refresh == true) {
    // perform the redraw
  };

  // reset the refresh switch for each loop so we don't peg the CPU
  scene.refresh = false;
};

The question now becomes, when is it appropriate to set refresh to true? That’s something I’m still working through; if I simply switch it back on in response to keyboard and mouse event handlers, that’s a start; but in Elevation I also have on-screen form controls that glow on hover, and switch images between when they’re selected and when they’re not. I now have to rework some of the assumptions those controls were making about the draw loop and code them to be a bit smarter about when they actually need to redraw.

Still, just these basic steps were a great start. Manipulating the scene with the mouse and zooming in and out will still tax the CPU, but as soon as you take your hands off the controls usage drops down from 99% to a more reasonable 4% or 5%. That’s progress.

This project was a too-perfect alignment of things relevant to my interests. I got a chance to really stretch my legs with Processing, had an excuse to tinker with GitHub finally, spent some serious design time in Photoshop thinking through the site and app UI visuals, designed a high-res app icon, got to play with mapping and GPS software, and it all gave me the motivation to go out and pedal around this beautiful city to create my route data in the first place. Spending a month coding while racking up over 400km on the bike seems like a fairly healthy work/play balance.

And happily there’s plenty of fodder in here for future posts, from parsing differing XML formats to building a functional GUI. More on those soon.

Thanks for the link love to: CreativeApplications, blprnt, Rubbishcorp

There’s an app called RunKeeper for the iPhone 3G/3GS that uses the phone’s GPS to track your route / elevation / time as you’re out being active. If you upload your routes to the web site, you can get the data back out in a couple of XML formats. (Google’s KML kind of sucks, it turns out; I’ve been finding GPX files easier to use. RunKeeper’s web app exports both.)

I’ve started hacking up an app to plot the data in 3D space. Very early stages, but it’s working quite well so far. Here’s a quick video:

Make sure to wait for the shift in plotting modes through the movie; I’ve built three so far: lines, points, and colour-coded points that indicate elevation (blue in the low areas, red in the high areas)

You’re seeing 7 routes I’ve ridden over the past couple of days. I’m already seeing outlines of Vancouver’s city features forming: the distinctive duck’s head outline of Stanley Park, the south downtown seawall, the massive hill heading out to UBC, etc.

What’s obvious is that some of the elevation data, particularly around Stanley Park, is seriously whacked. I think the cliffs obscure the GPS signal or something, because it should be more or less sea level all the way around. (In line view this is most obvious: it’s a series of jagged peaks and valleys, which strikes me as physically impossible). Hard to say whether it’s the app or the phone’s GPS to blame here, but my guess is the latter.

Next step: adding speed indicators, once I figure out how to work with ISO time stamps in Processing. (And going out and biking more paths this weekend to start fleshing out the terrain a bit more. Nice way to stay motivated.)

First I throw a set of photos into the sketch’s data folder, sequentially named 1.jpg through x.jpg (though I’ve since discovered Daniel Shiffman’s excellent directory listing functions that wouldn’t require me to touch the photo filenames). I choose a random image from this folder and load it into an off-screen buffer, then choose random pixels and load their colour values into a palette array.

// create buffer and setup palette variables
PGraphics buffer;
PImage paletteSource;
color[] palette;
int palettePhotos = 9;
int paletteCount = 24;

void setup() {
  loadPalette(paletteCount);
};

void loadPalette(int paletteCount) {

  // load in an image
  paletteSource = loadImage(int(random(1, palettePhotos)) + ".jpg");

  palette = new color[paletteCount];

  // load the image into the buffer
  buffer.beginDraw();
  buffer.image(paletteSource, 0, 0);
  buffer.endDraw();

  for (int i = 0; i < paletteCount; i++) {
    int colorX = int(random(0, 300));
    int colorY = int(random(0, 300));
    color pick = buffer.get(colorX, colorY);
    palette[i] = pick;
  };

};

That's it. Stupidly simple, but all the interesting colour combos I've posted on here in the past have been a direct result of this. Not every palette looks great, but if you're using interesting photos as the source, 7 or 8 times out of ten you get something workable.

A bit of code documentation: since I use both portrait and landscape photos, I decided to constrain the area I grab pixels from to a top left square of 300x300 pixels. You can adjust those values to suit, if needed.

You can control the number of colours you load in with the paletteCount variable, and the values ultimately end up in an array called palette[], which you can use sequentially:

 background(palette[0]);

or randomly:

 int paletteValue = int(random(1, paletteCount));
 obj.R = red(palette[paletteValue]);

Turns out the QTJava library that sites between Processing and the Quicktime codecs that allow it to output video has been deprecated as of Quicktime X. Last night I did my first test render and it looked great; today I don’t have the ability to output .mov files anymore. So, that’s fun.

I’ve got a second computer with plain old Leopard on it that I can still render from, but I wish I’d taken advantage of that second of hesitation when I saw the “install Quicktime 7″ checkbox as I upgraded this morning. There’s probably a way to get it back, I’ll update if I find out what it is.

Update: So getting it back wasn’t hard. On the Snow Leopard install DVD there’s a folder called “Optional Installs”; run the package in there and select Quicktime 7 from the list.

Except that doesn’t seem to work. Even with Quicktime 7 I can’t seem to save out a movie file. Drat.

In an interesting case of things coming around full circle, apparently the whole idea behind WallBlank’s site was inspired by a post I wrote on the main blog last year. Seems only fitting that I’d contribute something sooner or later.

It started like this: I had an idea to plot some points on a sphere. Now, I understand with some crafty use of the rotate() function I’d probably be able to get by without doing the math myself, but I guess I had the urge to dig a bit deeper in case I need to really know this (again) at some point.

So I started light and plotted some random points around a circle. By setting the X coordinate, I can reverse the Pythagorean equation enough to work out the math for the Y coordinate:

  X = random(0 - radius, radius);
  Y = sqrt(pow(radius, 2) - pow(X, 2));

That only gets me a positive value for Y though, which renders as a semicircle. So a bit more randomization was necessary to get the full circle:

  int rev = int(random(0, 2));
  if (rev == 1) {
    Y = sqrt(pow(radius, 2) - pow(X, 2));
  } else {
    Y = 0 - sqrt(pow(radius, 2) - pow(X, 2));
  };

Which doesn’t strike me as terribly elegant, but that’s about the limit of my coding ability.

So that was the start. Now that I can plot individual points around a circle, why not use something other than points? With some fairly basic box() objects and various rotations and transformations, things get interesting quickly:

I still haven’t gotten the math for the sphere down yet, though. That was a “walk away for now, and come back to it in a few days” type of problem.

Starting with a spreadsheet filled with a few dozen values, I can get at the CSV with a simple export:

The loadStrings function pulls in a local file or a remote URL as a String array:

String[] inputFileLines;
inputFileLines = loadStrings(
  "http://spreadsheets.google.com/pub? ->
    key=tSLqWX2BTMg1-fUyRV5shBQ&output=csv"
);

It’s fairly trivial to walk through the array and do something useful with the numbers:

inputFileIndex = 0;

while (inputFileIndex < inputFileLines.length) {
  String pieces = inputFileLines[inputFileIndex];
  if (pieces.length() > 0) {

    // do stuff with the 'pieces' string here,

  };
  inputFileIndex++;
};

The data exists as strings though, so it’s probably more useful to convert each value to float() or int() before using it. With the simple CSV file I linked up earlier as a source, I used the basic loop to assign the data points as the radii values for each of a set of spheres plotted randomly along the Y axis, or the height of a set of bars slightly tilted along the X axis:

If we were to say the linear-increasing X value is time, that makes for a couple of fun 3D graphs that even Tufte can be proud of. (Or not.)

The immediate challenge was outputting something usable in print. I saw two options; figure out a way to use a canvas thousands of pixels across for the sake of saving out a bitmapped screen grab as a file high-resolution enough to print, or, instead find a way to output as vector.

The latter is clearly the better option and it turns out Processing has a built in PDF library that does all the hard work for you. It’s not totally straightforward; you can’t just dump the screen contents to a file, instead you have to capture the entire draw loop from the start and save that out. But even that is fairly simple. My draw loop ended up something like this:

void draw() {

  // setup the PDF save
  if (writePDF) {
    beginRecord(PDF, "frame-####.pdf");
  };

      // draw things here

  // now that the draw buffer is full,
  // dump the contents to a file
  if (writePDF) {
    endRecord();
    writePDF = false;
  };
};

The writePDF variable was an important check; without it I’d be writing PDFs on every frame. Instead I can toggle it on keypress and just capture a single frame:

void keyPressed() {
  // if 'p' is pressed, save out a PDF
  if (int(key) == 112) {
    writePDF = true;
  };
};

The resulting PDFs are surprisingly svelte; 8000 ellipses saved out to about files around 110k. My PNG screen grabs are usually at least 4 times as large.

What was particularly fun about this project is that I dared to ask whether each print could be unique, or if we were stuck working with the same source PDF for each print. I was pleasantly surprised to discover the printer was totally cool with printing a unique file for each one of the 100-print series, and happy to take advantage of that. With a slight modification to my draw loop:

void draw() {
  if ((createStuff > 0) && (createStuff < 101)) {
    beginRecord(PDF, "print-####.pdf");
  };

  if ((createStuff > 0) && (createStuff < 101)) {
    endRecord();
    createStuff++;
  };
};

...all I had to do was hit a key and wait about a minute, and voila, 100 totally unique prints, but united as a series with a common palette.

I'll make sure to mention it again on here when the prints are available for purchase, probably closer to the end of August.