on Quality

My new home: write up in Lion's club, Staffansdorp, Sweden

2012-02-06T17:38:00.000-08:00

Wicked! My new home in Mountain View got a nice write up on the Lion's club site: http://www.lionsstaffanstorp.se/loppis-i-staffanstorp/

Part 2: Setting up Marionette for automating WebAPI testing on B2G

2012-02-02T18:03:00.000-08:00

NOTE: The following article is now available as a series of articles on MDN: https://developer.mozilla.org/en/Mozilla/Boot_to_Gecko/ and will be maintained and updated there.

Here, we'll set up marionette and try out some of its functionality with the interactive python prompt.

Marionette is a test automation framework build by the "ateam" internally at Mozilla. Marionette provides "a new and unified way to automate testing on the B2G environment" since we're booting directly into Gecko rendering engine and don't have the traditional browser support previously afforded.

It's already assumed you've gone through the steps (cloning marionette client and B2G) in our previous tutorial.

Setting up Marionette requires installing Python 2.7.2 and pip. You'll also need the ManifestDestiny and MozHttpd packages from Mozbase:
$ git clone https://github.com/mozilla/mozbase.git
$ cd mozbase
$ python setup_development.py

If you're using (or planning to use) the selenium proxy to bang on UI elements, you can also install the selenium bindings:
$ pip install selenium

Interactive Python tests work similarly against the device or against QEMU so the only thing different, is that with QEMU, you, of course, must launch an emulator
$ ./emu.sh

Whereas, with the device, you must make sure it's turned on, and physically connected via adb:

Otherwise, the steps are more or less the same.

Open two terminals. In one, forward the port:
$ adb forward tcp:2828 tcp:2828

In another, launch Python to get to the interactive prompt:
$ python

From the interactive prompt, run the commands necessary to invoke a Marionette session interactively:
>>> from marionette import Marionette
>>> marionette = Marionette('localhost', 2828)
>>> marionette.start_session()
u'session-b2g'

Here, we see that the system returns that a Marionette session is running.

The command "marionette.execute_script()" can embed JavaScript commands which run on Gecko. Next, you can see which DOM elements return HTMLElement objects:

>>> marionette.execute_script("return navigator.mozBattery;")
{u'onlevelchange': None, u'level': 0.91, u'dischargingTime': None, u'onchargingchange': None, u'ondischargingtimechange': None, u'onchargingtimechange': None, u'chargingTime': None, u'charging': True}
>>> marionette.execute_script("return navigator.mozBattery.level;")
0.91
>>> marionette.execute_script("return navigator.geolocation;")
{}
>>> marionette.execute_script("return navigator.mozSms;")
{u'onreceived': None, u'ondelivered': None, u'onsent': None}

You can traverse the DOM tree using this technique and evaluate which calls are available.

You can test out basic telephony using interactively with marionette. The following example requires two working phones, both with simcards. One is your Galaxy SII, running B2G.

Start an interactive Marionette session and forward the port:

$ adb forward tcp:2828 tcp:2828

$ python

>>> from marionette import Marionette
>>> marionette = Marionette('localhost', 2828)
>>> marionette.start_session()
u'5-b2g'

Now, try this, with dialer app invoked on the phone:
>>> marionette.set_context("chrome")
True
>>> marionette.execute_script("return navigator.mozTelephony;")
>>> num =
>>> marionette.execute_script("return navigator.mozTelephony.dial('%d');" % num)

or else, this (notice the quotes around the JS number variable:
>>> marionette.set_context("chrome")
True
>>> marionette.execute_script("""
... var num = ""
... return navigator.mozTelephony.dial(num);
... """)
{}

This kicks off a phone call, the output of which can be monitored in $ adb logcat

Part 1: Setting up Boot-to-Gecko build environment

2012-01-30T19:32:00.000-08:00

NOTE: The following article is now available as a series of articles on MDN: https://developer.mozilla.org/en/Mozilla/Boot_to_Gecko/ and will be maintained and updated there.

There's been already a fair bit posted on how to set up a boot-to-gecko building- and flashing- environment.    However that information tends to be scattered about a bit and rather asequential. So in the interest of keeping everyone on the same page, so to speak - and one path to success - I'll endeavor to spec out one set of instructions anyone can follow to get up and running.

This tutorial assumes a basic grasp of Ubuntu Linux.

You should be able to copy most commands from this blog and paste. Note, however that there are no guarantees and you proceed at your own risk!

I built B2G from a VM.    You can build from a Native Ubuntu Linux platform.

The environment:
You'll need:
*64-bit platform running Ubuntu 11.10
with at least:
*4 gb RAM
*7 gb swap partition
*80 gb hard disk

This deliberately overstates the requirements on B2G github.   You'll find eventually you'll need the resources.

Once that's installed, you'll need to add adb to your environment's path:

This is where it gets interesting. There is a circular dependency concerning adb. The first time around, you will need adb to build gonk (which, then, builds the adb you will use subsequently). So, even though we say otherwise, you STILL need to install the Android SDK and edit your ./bashrc path to point to that adb instance - so that you can build gonk the first time!

Once you've built gonk the first time, adb is created under "/glue/gonk/out" . You must edit ./bashrc again to point to this new instance.
$ cd ~
$ sudo nano .bashrc

and add the following lines (depending on where you've extracted android sdk):

PATH=$PATH:/home/mozilla/gits/B2G/glue/gonk/out/host/linux-x86/bin/adb
export PATH

You'll need to setup adb talk to your galaxy S2. To do that:

$ sudo nano /etc/udev/rules.d/51-android.rules

and add the following line

SUBSYSTEM=="usb", ATTR{idVendor}=="04e8", MODE="0666", GROUP="plugdev"

Make the file readable:

$ chmod a+r /etc/udev/rules.d/51-android.rules

You'll need to install heimdall to flash B2G (and backup the preinstalled Android OS) to your Galaxy S2 device. Get it from

http://glassechidna.com.au/products/heimdall

and run the install.

Next, you'll need to install basic build dependencies for building firefox on 64-bit Ubuntu 10.11 VM.

Download and install the build prerequisites with two commands (as root):

$ sudo apt-get build-dep firefox

$ sudo apt-get install mercurial libasound2-dev libcurl4-openssl-dev libnotify-dev libxt-dev libiw-dev mesa-common-dev autoconf2.13

We've also found it necessary to install the following dependencies (note that there may be some repeats here):
$ sudo apt-get install ia32-libs gcc-multilib g++-multilib bisonflex gperf lib32zl-dev lib32ncurses5-dev lib32ncursesw5-devlibidl-dev lib32gomp1 autoconf2.13 ccache libx11-devlib32readline-gplv2-dev

Also, if you haven't installed a JDK, get one:
$ sudo apt-get install default-jdk

Almost inevitably, you will, at various times, need to install some additional dependencies. Click "Y" when prompted to do this.

The next steps depend very much on what you wish to do. If you wish to build only Boot 2 Gecko Kernel, Gonk Backend and Gaia, clone the following repo:
$ git clone https://github.com/andreasgal/B2G.git

You'll also need to clone the marionette client for later:
$ git clone https://github.com/jonallengriffin/marionette_client.git

Building for QEMU
The following steps allow you to build - and run - B2G on your emulator. Do NOT run them as root.
$ cd B2G
$ make sync

$ make config-qemu
$ make gonk
$ make

NOTE: if ./emu.sh - your emulator - doesn't run at first, you'll need to make it executable:
$ chmod 777 emu.sh

Lastly, run the emulator:
$ ./emu.sh

Backing up your Galaxy S2

You'll want to back up your phone's factory image before attempting to build or flash anything to it. As it happens, the builds or flash process itself can be unstable and you don't want to end up with a brick.

These instructions assume that you have already installed heimdall.

From some other directory than where you have B2G or NDK/SDK installed:
$ cd /tmp
$ wget http://cmw.22aaf3.com/c1/recovery/recovery-clockwork-4.0.1.4-galaxys2.tar
$ tar xvf recovery-clockwork-*tar
$ adb reboot download

When your phone is in "download" mode, it's ready to be flashed using Odin protocol.

To Flash Clockworked (and backup image):
With your phone in "download" mode, run the following:
$ heimdall flash --kernel zImage

Next, reboot the phone to recovery mode.
$ adb reboot recovery

With the phone in recovery mode, follow the on-screen instructions to make a backup image.

Building for Samsung Galaxy S2
Do NOT run any of the following commands as root! From the directory to where you've cloned your git repositories:

$ cd B2G
$ make sync

With the device connected and running:
$ make config-galaxy-s2
$ make gonk
$ make

Flashing to Samsung Galaxy S2
-making and flashing the kernel (first time only)
$ make kernel
$ adb reboot download
$ heimdall flash --kernel boot/kernel-android-galaxy-s2/arch/arm/boot/zImage

-making generic flash
$ adb reboot recovery
$ make flash-only

Quick reinstall gaia
Assuming you're in the root directory of B2G:
$ cd gaia
$ git pull origin master
(Pulls remote and merges files to your local repo)
$ make install-gaia
$ make kill-b2g

(Restarts Gaia)

Quick build and reflash of gecko only:
$ make install-gecko
$ make kill-b2g

How to force git repo into same state as fresh clone (comprehensive git clean)
$ make sync; git clean -xfd; git submodule foreach "git clean -xfd"

Alternately, you can try:
$ make mrproper

Once your system is up and running on your device, you can begin to use and debug very much like an Android phone. You may have noticed that we use the Android Debug Bridge adb for many purposes. For example, if I wanted to debug telephony or SMS on my device, and I had incoming calls/SMSes coming from a number with area code "216" I could monitor that with a command such as:
$ adb logcat | grep 216

To be sure, $ adb logcat with no params will simply output everything until you ctrl+c to close the adb session.

Below is an example of enabling debugging on the device.

How to enable debugging, for example, of radio component:
First, change DEBUG to true in gecko/dom/telephony/ril_worker.js
Next, rebuild gecko and reflash at least gecko on the device:
$ make install-gecko

and then provide $adb logcat output

and
$adb logcat -b radio

While performing the affected operation, e.g. trying to initiate a phone call from the dialer app.

2012-01-23T13:54:00.000-08:00

...a promo pic from a coffeehouse gig I did in Concord, CA (at the Panama Red Cafe), Sunday, January 22, 2012.

I had been wanting to reconcile the "nerd" and the "artiste".... thus the "Ion Monkey" (it's Mozilla's new JIT compiler) t-shirt. The set had a host of short jazz pieces, mostly intros, some culled and influenced by one of my favorite jazz guitarists, Joe Pass. Which of course, sounded amazing on acoustic guitar.

Links to some of the live stuff, to come.

Zippity - and automated performance tests for Firefox Mobile

2011-12-27T17:17:00.000-08:00

Many have wondered how we get the device coverage with our automated Performance tests. Here's one way:

What you see running here are the Zippity tests (a plug in available for Firefox mobile ), running the pageload tests on a number of different devices. I also have logcat attached and running, to catch any possible crashes.

Pageload tests load a series of predefined webpages, to gauge pageload speed. There are also Startup tests (which start the application several times to measure startup speed); as well as SunSpider and V8 tests. Lastly, you can ping your memory metrics to Zippity.

Having the crowd available to help run these tests (just install the plug in), helps us get the device coverage up and ensures we find things like native crashes, hopefully sooner, rather than later.

Kudos to Mark Finkle for this great tool.

2011-12-27T00:19:00.000-08:00

I made this video last winter, when I was still in Helsinki:

...and here is one of my songs from my performance at Hotel Utah, San Francisco on December 19, 2011: http://theutah.org/performance/johnhammink/20111219/againsttheglass

A plug for the Fullscreen, Gamepad, and Mouse Lock gaming APIs

2011-12-11T15:55:00.001-08:00

Sometimes, when things are moving as quickly as they generally do here at the Moz (as in the open web generally), the best that one can do is channel the work of others to try and stay on track with everything. It certainly beats trying to do everything oneself, at the breakneck speed at which envisioning the new technology - and technology itself - unfolds.

On that note, kudos to Dave Humphries, Rob Hawkes and Chris Pearce for their recent blog posts on these brand new APIs. And so, by way of a rehash...

Fullscreen, Gamepad and Mouse Lock APIs are available in a single experimental desktop build of Firefox. There are also builds that support these features here.

These APIs are estimated to land soon in the nightly builds.

The Fullscreen API:

Scheduled to ship in firefox 10 by the end of January, you can still try out some of the examples in the latest nightly builds.

Based on the W3C's draft spec, this API enables any HTML element to be made "fullscreen", which hides the browsers UI and stretches the element to cover the entire screen area.

To use it, the developer calls void mozRequestFullScreen(), which, if the request is granted (there are security implications here) gets a mozfullscreenchange event back to the specifying element; otherwise a mozfullscreenerror is dispatched.

The user can ESC or F11 to exit full screen mode. As with YouTube or other fullscreen videos, navigating (ALT+TAB) will cause fullscreen mode to exit.

There is a cross-browser full-screen API example here.

The Gamepad API:

As directed by the W3C spec, Javascript and Firefox now gives developers the possibility to connect a gamepad directly to the computer (using USB or Bluetooth) - a domain previously only available to gaming consoles and native computer games.

You can try it out here; note that you''ll need that custom build. This wiki page contains more implementation specifics.

Rob Hawkes made a great demo video:

The Mouse Lock API:

This API hides the cursor and locks it to the center of the screen, thus enabling developers to make games and visualizations of a 3D world (otherwise the cursor would go off the side of the screen or hit the edge). With the cursor locked, the user can roam X, Y, Z axes without restrictions.

David Humphrey and his students at Seneca College are currently implementing this API based on the W3C spec.

There are demo files available.

We're tracking Mouse Lock API on Bugzilla. You can also check out David Humphrey’s series of posts on the development of the Mouse Lock API.

I'm just now trying these out myself; of course, if you find any issues, please feel free to file a bug.

Fall tour of the Acoustic Circuit, S.F. Bay Area

2011-12-10T23:31:00.001-08:00

From Freight & Salvage Coffeehouse, Berkeley, November 22, 2011

If I don't (yet) get to unleash the beast on my software evangelism (a quintessentially nerdy pursuit) the least I can do is hit the acoustic singer/songwriter music circuit around the Bay area (which I'm currently doing). It's only been a few short weeks since I've started touring regularly in addition to my awesome day job (longer geographic swaths in the works on both the music and work fronts), but it's already been a hell of a ride.

From St. Lukes Church, San Francisco, November 20, 2011

From Rockit Room, San Francisco, November 24, 2011

From Panama Red Coffeehouse, Concord, December 9, 2011

From Berkeley City Club, November 17, 2011

Here is one of my songs from my performance at the Hotel Utah, San Francisco, from December 19, 2011.

When going down is good

2011-12-06T18:43:00.001-08:00

I've often complained about how jumping out of an airplane with an experienced tandem guide is safer than merging onto the 101. It's certainly more fun.

Let's have a look at some recently landed Web APIs for Firefox

2011-11-28T13:49:00.001-08:00

So, without further ado (as there's been too much "to do" about it) let's take a look at what we presently have (landed or about to land) regarding our latest (Firefox) Web APIs.

You'll need the latest xul fennec builds to run these. Technically, these also run on the latest birch nightly, but that one is presently undergoing a major facelift due to rendering and related issues.

I'd suggest using a tablet. (For the SMS app, you'll need a phone with simcard and a special build, but we'll get to that presently).

All example testpages to which we'll be referring are available under the directory at: http://people.mozilla.org/~jhammink/webapi_test_pages/

If you find a bug with any of the demos and/or APIs themselves (and we hope you will:) please write them here:
http://shrt.st/2adv

Make sure to include your device model & Android version, as well as the nightly fennec version you are using, in the bug.

Battery API demo:
http://people.mozilla.org/~jhammink/webapi_test_pages/BatteryAPIdemo.html

This API tests the different charging states of the battery, along with charging level and discharging time. Try launching the page with the device plugged in, and then unplugging (and/or vice versa). Note that the charging/discharging time presently shows Infinity until the battery has charged/discharged about 2% of its total capacity.

Does the testpage show the correct battery state for the device? Does it correctly generate an alert when the charging state changes? Is the level correct (when compared against native battery app level)? Does it give a numerical discharging/charging time after some time in the same charging state?

Camera API demo:
http://people.mozilla.org/~jhammink/webapi_test_pages/CameraAPIdemo.html
This API presently tests the browser's ability to directly capture pictures from the device camera.

Point the device camera at the object you want to shoot and click capture.

Is the picture rendered correctly (as opposed to upside down, or sideways) both in the pre-capture and the preview? Is the picture quality consistent with what your camera normally delivers? Are there any other issues? Please feel to play around with it a bit.

IndexedDB API demo:
http://people.mozilla.org/~jhammink/webapi_test_pages/IndexeddbAPIdemo.html

Since accessing the native file system from the web introduces a huge risk, there needs to be a way to handle offline storage directly on a device. Enter IndexedDB. Actually, this api has been around since Firefox 4 (and subsequently for quite a few Chrome and other browser versions since) but the W3C spec got a rewrite recently and it's been well overdue for an upgrade.

When the browser first tries to create a local database, a prompt requests permission:

Subsequently, once this is allowed, the db initializes, and after some time, reports back on the number of records:

Our current demo matches a hash of zipcodes against a city (you must have zipcodes.csv present in same dir to work - this demo does), doing all the database access operations. (Note that we're presently missing the functionality to do the reverse!) Once the CSV is loaded into the device local database, the lookup should be instantaneous (note that all the steps are not depicted here):

Note that I've been getting dramatically different results from device to device. Sometimes it works, sometimes not. To be sure the feature is still a work in progress, so let me know what your experiences are.

Vibrator API demo:
http://people.mozilla.org/~jhammink/webapi_test_pages/VibratorAPIdemo.html

Web apps, just like native ones, need a way to send feedback to various sensors - including notification vibes. The W3C spec lays a foundation, which we've implemented. Note that you'll need a device with a vibrator to test this one:

You'll need a special build for this one also. Try out each of the scenarios and verify the result. Are we getting the correct number of buzzes, given the spec? As with any of our APIs here, you can check the implementation by typing the string "view-source:" (without the quotes) before the "http://....". Try it.

SMS API demo:
Note that you'll need an android phone with a working SIM card to try these demos. Until this feature lands, you may also may also need a special build with this feature.

http://people.mozilla.org/~jhammink/webapi_test_pages/SMSAPI_simplesend.html

The simplesend is a page that lets you send SMS messages to any phone. Try it! Is your message able to get through? Any other issues? Note that, for testing purposes, you can also try sending the messages directly to the same phone from where they originate.

This next example lets you do exactly that: send and receive SMS messages from the same webpage instance:
http://people.mozilla.org/~jhammink/webapi_test_pages/SMSAPI_send_receive.html

This page is similar to the previous one, but also has an event handler for incoming messages.

Don't forget to study the page source, for each of these pages, to understand how these APIs are implemented.

And, once again, don't hesitate to file a bug if something doesn't appear to be working correctly, or if you have an idea about some features you'd like to see.

Addendum: The Web API test pages are now available in github: https://github.com/jhammink/WebAPI-test-pages

Mozilla's Web API - on testing a framework.

2011-11-21T00:20:00.001-08:00

So, hello again to you all. It's been ages since I last posted at this space - but for good reason: I've taken my performance show on the road again, and, at the same time, begun my work at Mozilla. I find myself, once again, in the happy place of doing, on a daily basis, two different things that I love.

Primarily, my work has been around our Web API, and how we will write -and eventually automate - tests against it. But how will we do this?

Our API is intended to "expand what the web can do". One of the most important implications is the ability to access - in a secure fashion - our device hardware from the web. To that end, there are such things as Vibrator API, Battery API, Settings API, Accelerometer and Sensor API, a new and improved Geolocation API, Sound API....from which we can work our device directly from the web!

So, how does one test a bunch of APIs as a user would use them? Well, one thing (and I've been involved with such an effort before) is to build an Application Framework - essentially a series of mock-up and atomic apps that each demonstrate a single API.

Over time, these atomic apps will be folded into more feature rich ones - to demonstrate the capabilities and (hopefully - we're QA after all) reveal any bugs that mixing and matching these APIs will uncover.

Another aspect that is important - particularly where automation is concerned - is a way of getting "golden values". We will build a series of simple native applications for each OS that will help us to obtain these values for comparison.

Watch this space - in the (very near) future we will demonstrate how many of the web apis work; how we will build HTML/JS test pages around them and test many of the features as a user would use them; build our golden apps in java and/or python (depending on our target platform), and, eventually, work the lot into an automation framework. All of the code will be freely available on github.

I'm hoping for many outcomes; one of them being that you will learn (if you don't know already) how any of the technologies depicted here actually work, but also that you might jump in and help out - when the time is ready - by expanding what we've started here and our web api testpages, enriching our automation core apps, and really putting those apis through the paces, at their intended purpose.

Stay tuned!

Absence is the inverse of presence

2011-10-12T00:12:00.000-07:00

"When the changes come, hallelujah! When the changes come, there's nothing you can do."

- from a song by Matt Weidemann

Kisu. Helsinki, 2010

Kisu, my buddy of 14 years, the "Itty-bitty-kitty" passed away Monday morning at the age of 15. I think the sense of loss and absence has only just begun.

We had planned for a veterinary visit already on Thursday, but the week went on as I became more focused on my new job.

So we changed our plans to Monday. I had some idea in the back of my mind that she wouldn't make it that far. She started failing already on Saturday; that was the last time she could make it to my lap; by Sunday, she could no longer get up on her own. I moved her from the window (it was the last beautiful day of the rainy California autumn), which was open, to the bed. I played her a little dulcimer for a few minutes and also sang her a few old songs she may remember from her earlier years. During the dulcimer part, she rallied around and with much shaky effort and managed to give me that loving look I've been blessed with so many times. During the night, she reached out with her paws to me a couple of times. By Monday morning, she had slipped away. I wonder, what, if anything, she's experiencing now?

We all have our legacies and journeys to tell about, but somehow the years with Kisu were extraordinary. She came into my life when a younger, more worried me was living with a girlfriend, anxiously unemployed in a tiny Soviet-looking apartment in Helsinki. Although the situation eventually resolved itself by the end of summer, having the one-year old Kisu around (who was left to us by a recent divorcee) was, in a pattern set for the rest of her life, an incredible reassurance. Her playful antics were the perfect (or as good as could be expected) distraction from the anxiety of a 27 year old, who doesn't know any better how things usually work out for the better; literally thousands of CVs later and I had found myself some pretty lucrative work, and had reached an island in the storm.

Over the years, Kisu was with me during the meteoric rise and riches accompanying the IPO of the company which gave me my first real job; she moved with me from the heights of Kallio, Helsinki, up to the forests of Espoo, and eventually across the Gulf of Finland to Tallinn, Estonia, where I set out on my own and eventually connected with another company that encountered even more meteoric success than the first one had. She patiently waited with friends during my jet-setting years, when I always seemed to be catching a flight to some remote corner of the world (sometimes she caught the flight with me!). Then she eventually reunited with me when we moved together to a huge house in the Swedish countryside. Eventually, she even moved with me back to my original homeland of Ohio, U.S.A.; enduring my irritating months with one not-too-happening nor well-advised startup, and finally made one last great journey of her amazing feline life: a trip, by car, from Cleveland, Ohio to Silicon Valley, California. Again, enduring the stresses of a dysfunctional contract (but one where I learned oodles), taking me, through another contract hunt, finally to my present job, one where I feel secure and appreciated for the first time in years.

Kisu lived in Finland, Estonia, Sweden, and the United States (including traveling across it) and visited Chile and Russia. She's been to homebrew robotics and programming nights, with gaggles of caffeinated pals, living room jam sessions, real commercial recording sessions, Christmases and Thanksgivings, hosted more guests than I can imagine and been the guest of at least five families that I can remember. She's been through the most excessive of good times and the leanest of lean times and everything in between. She's sat by the side of my computer as I've worked, in my lap when I've cried and across from me, watching eagerly, when I've sang. And we met in that tiny apartment on Alppikatu; she died in a motel. In some ways, it seems, we've taken a step back together. Just her and me.

I live alone, and have for some time now. Sometimes I have thought that cat is the only thing keeping me from going crazy.

On the Monday evening following hear death, I removed all traces of her from the apartment (well, not exactly all, but the unused litterbox, food). I knew that I couldn't stand to have that stuff around, and that it would be difficult to deal with, the more time passed.

I still sense her in the ways that she's not there: expectations of a soft padding up to the bed, of her watching me from the bed or chair when I came in the door everyday; of her sniffing the breeze out the window, and watching the activities, the seasons change, the rain, the sunshine. And then there are the intangibles of absence: the absolute dead quiet of the room, the sense that someone else is right out of earshot or just around the corner. Kisu is present in her absence.

Like a boat making its way through a calm stream, our legacy is as much about the ripples that remain after us. That which is present, and of us, in our absence. In this way, we persist. Although I have to give myself some credit, I feel that Kisu, in her lifetime, took me (well, she took the journey with me) from a place of anxiety, through some incredible moments and backdrops, to a much more hopeful place. If I make the best of this, now, will she, as she persists now, be ever intertwined with this new future?

Kisu and me. Denver Colorado, Summer 2011

Liet International 2010, playing bass with Jousnen Jarved

2010-12-05T17:44:00.000-08:00

It's always been one of my big dreams to play bass for world music and although, since having this dream in the early 90's when I was briefly a fill-in disk-jockey at an Ann Arbor, Michigan radio station - I've recorded fairly little in this regard - and of extremely variable quality, in relation to where I've actually been over the last 10 years or so. Most of my recorded stuff is from Russia, Siberia, China, (I've recorded others from Taiwan), the middle-east (stylistically), or Fenno-Scania - and I can add both Americas and Greenland to the itinerary. Occasionally, however, I do something which represents a bit of a zenith.

Ahh, but enter Jousnen Jarved, from Russia. I've been travelling to the Russian side of Karelia since the early 2000s and have worked with the project's creator, Peter Coon, for almost as long. Peter's quite the producer, having written, scored, and produced a few hard-rocking singles for me out of his own studio in Petrozavodsk, and being an honest-to-goodness music professional in his own right, something which I am most definitely not. And so, it was like the dream fairy came and dropped this thing in my lap, as I was sitting in Helsinki, bored to death with my musical prospects there, and about ready to give up on the whole damn thing. The Liet International festival in Lorient, France, was just the ticket.

TV Renne in France, Fryslan T.V. in the Netherlands and CNN covered the event for a global audience. We're up for a spot (and hopefully rotation) on Radio St. Petersburg at some point, but I don't know the details yet.

Off the beaten path - Test Automation that can vary (and learn)...

2010-11-15T01:46:00.000-08:00

The problems with most software test automation is essentially twofold: for the most part, it's only capable of following the same path, over and over again, and, it's incapable of ever learning. It's as dumb today as it was yesterday. That said, it's just not as classy as it could be.

What, it can be asked, is such repetitive test automation actually useful for? Well, most importantly, for verifying that something doesn't get broken as new features get added to our software; we opt to run our tests to give us a sense of confidence in what's already there.

Of interesting note, though, is the fact that we nearly always need to pair our automated tests with hands-on exploratory testing. Except that it never really happens like that. The hands-on exploratory testing gradually gives way to the Manual Monkey Test as the testers and managers lose confidence in those unique skills of perception and hunches that make us human and instead seek something quantifiable, repeatable and reproducible.

So how can we turn this around? Boredom with repetitive work begets automation; automation begets need to exploratory test which somehow though time pressures turns into more of the same....in the meantime, the defects to be found live "somewhere" off the path of this tedious nightmare.

There are no complete solutions presented here, but I hope you might get some idea of the possibilities available.

Well, one thing to do is to make our test automation more intelligent. Exploratory testing essentially involves two skills, before thought to be uniquely the domain of homo-sapiens - 1.) the ability to randomize inputs and 2.) the ability to assess and learn if a given output is appropriate given a non-linear input. Let me elaborate.

One of the great benefits of open-source software on test automation is (aside from not being bound to a proprietary language) the ability to leverage the wealth of extension libraries that come with the open source languages, and you're supported for free by a massive user community of volunteers. Tools like Squish and Selenium use Python (among others); Nokia's TDriver uses Ruby.

Without going into too much low level detail ;-), both Python and Ruby have randomizer functions. In Python, one can use the rand()function with an integer as an argument, for example rand(7) returns any number between 0 and 6; Ruby has exactly the same thing. Here's a recipe in Ruby for a simple sort and shuffle of a deck of cards, producing n log n variable swaps:

Now, this won't be the fastest way if you're dealing with a very large list, but for generating a small set to explore alternative paths, it could well suffice.

Now this randomizing can apply to just about any input that comes as an index - so this means just about anything: selected items in a list, buttons in toolbars, parts of a string or a numerical sequence. Once we are randomizing, we are dealing with inputs. But, how to know when to randomize?

Most users out there are going to use a software application in a given or fairly static way - just enough steps to suit their purpose. Configure the music player, cue it up and play the song. But every ONCE in a while, they might deviate from that sequence. So knowing when to randomize, means knowing how often, based on probability, to do something completely different.

By way of a practical example....Cucumber has become a popular way to express automatic tests in plain language. A cucumber test may read something like this:

Given the MusicPlayer application is started
When I press Options softkey
When I select Songs from menu
And I Select the Song number 1
Then Now playing view is opened correctly

Where each line in that block relates to a function implemented in the core language below it, for example:

Once we pass a certain threshold (e.g. the function has been accessed n times, we are twice or three times likely to deviate from our original number and choose a random index). Doing this effectively requires a trick though - the ability to visualize a software system as a state machine in 4D. And if we randomize at the function level, our cucumber test writer gets the benefit of that randomization without having to do anything differently

Once our scripts start to randomize, however, the fixed answer sets to our test runs will no longer suffice; our tests will require the ability to be able to be trained, to learn, and to guess based on previous history as per a correct answer. Fortunately, our open-source languages have the tools to allow us to be able to do just that.

Bayesian gems are already in common use in email spam filtering. Python, for example offers the PEBL and Classifier libraries; Ruby offers Bishop and Classifier gems. Training for the Classifier gem works along the lines as follows (note - NOT TESTED):

require 'rubygems'

require 'classifier'

classifier = Classifier::Bayes.new('Song', 'Not_Song)

classifier.train_Song('%r{.+\.(?i)mp3}')

classifier.train_Song('%r{.+\.(?i)3gp}')

...

classifier.train_Not_Song('%r{.+\.(?i)jpg}')

...

And the good stuff - where the real demonstration of learning happens - is here:

classifier.classify "bubba_chops.jpg"

#=>"Not_Song"

classifier.classify "song.3gp"

#=>"Song"

classifier.classify "song.m3u"

#=>"Song"

In the case where the script gets it wrong - you have to train it. But after more and more iterations - it will start to make the correct decisions in more and more of the cases.

This is where your script can think, and make decisions as you do.

And that's no monkey business.

The sine wave and quantum beats

2010-08-11T10:16:00.000-07:00

s long as I'm mucking about trying to find parallels between quantum mechanics, set theory, and rythm, I set about with a bit of a musical experiment on my trusty korg electribe mx as piped through my soundcraft lexicon mixer's drum plate. The tubes in the electribe give the drums a predictable warmth, while the plate adds room overtones and depth.

The problem is no doubt simple from a purely musical or musician's perspective, but interesting from a mathematical one.

Just like an ordinary sine wave, we simply build a rythm that alternates parts from one measure to the next. For example, we can build a rhythm consisting of bass drum, snare, and decorate it up top with a constant closed hihat (with an open on the last half beat).

Our bass drum hits on [1, 5, 9, 13], where as our snare is on [3, 6, 9, 12]. What we end up with is a pretty straightforward bossa nova, with the intersection of both parts occuring on the half four [9]. Enter the sine wave: we alternate the bass and snare every other measure! (Incidentally, the effect that this has - if I first switch that opening snare to an opening kick - is a binary alternation between bossa nova and rock-steady beat!).

Granted this is the simplest way of looking at our beat composition and arrangement, but the best way I can think of to introduce the analogy. But think of the tapestry we could weave when we apply more sophistication to this schema? Like introducing limits, Integration/Differentiation, partial derivatives to the same set of numbers? (Note: I haven't figured all this out myself yet - still some of the results I've observed are really interesting - not to mention they groove, albeit in ways you probably don't expect!) It's probably possible to build this on e.g. complex functions, while keeping the groove. Then you're in uncharted territory, sweet stuff, you're sailing in blue oceans. Of course, there's more than just these musical variables to tweak.

When combined with acoustic instruments, the intent is hopefully to sincerely emulate "the groove" within the prescribed toolset with this added rythmic complexity as part of the mix. But like everything I'm into at the moment, I've kind of ripped the machine down to its parts to muse and dwell on particular small bits of the problem that fascinate me. Hopefully I'll get down to proper business with it soon enough!

Conservation - of energy and matter - would solve humanities major problem(s)...if we could just get there!

2010-08-11T05:58:00.000-07:00

The key to all of this is about humans being able to convert matter into energy, transmit it, and reconvert it back into matter, perhaps conveniently reconfigured!

Yes, it's more or less right out of Star Trek (we could beam people around, we could project objects like holographs), is only remotely possible with our current understanding of quantum mechanics (but is apparently only as far away as our grasp of a grand unification theory) which is being stabbed at constantly by professionals and amateurs everywhere (which introduces a promising possibility that something will emerge). Which reminds me, I must look into whatever exists out there in terms of P2P scientific communities and start farming for ideas...same with my fractal art, my music, my programming, my performance style, and just about anything else I've ever smugly attempted to synthesize :P

When I was recently in Greenland, I was thinking about the simplicity of water, versus the apparent impossibility (technically speaking) to transform it into energy and back again - and how it would fundamentally transform the conditions of humans, and planet earth. Imagine...being able to take water gushing and streaming off the arctic and antarctic ice and make it bubble out of the ground across all the arid regions on planet earth - from the Sub Sahara across the 'stans to Mongolia. Certainly, the room full of Somalis, Kenyans and other Africans on Helsinki's north-east side agreed with me hands down that this is the major number one obstacle on earth: obviously it would feed and economically enable billions who are currently dormant. By the greening of the deserts, would transform back the environment into something far more oxygen rich and equally less CO₂ saturated.

And yet's what stopping it?

Humans have, in the 20th century, been able to convert very unstable matter into raw energy in the course of wiping out foes, and related pursuits. And we have harnessed the practical energy of grabbing electrons as they fly off bits of U²³⁸. This is used in many countries to supplement what we can't get from wind, solar, hydroelectric and other sources of free-in-nature kinetic energy, and wouldn't want to necessarily get from other carbon-based non-renewable sources. But the pure conversion of matter to energy (and back) still seems a long way off in spite of our best efforts. And I find that somewhat puzzling.

In browsing across the macro-patterns of human activity we observe attempts to get there, but, for whatever reason, no solution completely bridges the divide. Take
current recycling...what we are gaining in reduced quantities of discarded paper (and to a far lesser degree, plastics), we gain in e-waste. With what I call "next years junk" syndrome, technophiles and technophytes, eagerly snatch up the latest available doodad that has arrived to their point on the bell-curve. Off to the "recycling" center or in some cases, landfill, with 2001's latest. Stuff resells on the market for anything greater than junk value for only so long, and when last years doodad goes to Africa, it simply ends up in the ground and water there, because Africans like the latest and greatest, just like everyone else, and they raid their shopping malls in Lagos and Johannesburg just like we do ours in Helsinki, London or suburban LA. Anyone seen my track-pad iPod?

I'm not sure what the state of the art in e-recycling is, but I don't believe it goes far enough. What happens to non-renewable plastics, parts like transistors that are not easily reducible (except with the same sort of "expensive" equipment employed in their manufacture and assembly), etc. I have often said that for everything that is manufactured, there should be an equal and opposite demanufacturing process/facility waiting for it at the end of its life. To what extent we fall short of it, I cannot say, but any shortcoming in it's achievement sets the immutability of those physical laws to a different footlight.

"Gross bodies and light are convertible into one another..." Isaac Newton, 1704. But what's the catch? Personally I stick to my e=mc² as a matter of faith, but understand, that it's opposite, its anti-particle, the yin to its hang, is subtly transformed, like water, to patterns that our symmetrical approach to reason and practical implementation perhaps cannot yet fully grasp.

In-between moments

2010-08-10T03:09:00.000-07:00

...when you care enough to give the very least.

2009-10-22T13:08:00.001-07:00

Longevity, brings me back to the center.

2009-03-19T15:46:00.000-07:00

So I'm working on two fronts. Trying to stay on my project, here, get funding for it, etc. because it's going to change the world. But we're in Limbo right now...and I question whether I have what it takes to go out and work as an outside consultant while we play the waiting game. I'm really tired, see, and need a break from this endless churn of working a few months, then hunting, etc.

On the other front, I've built up the recording studio and found musicians, and am very eager about making somthing happen there. Those are my goals. Acoustic groove. I can't get these rythms out of my head.

So I was feeling pretty hopeless until I read about two guys I could always count on to break up - and then reunite.

Longevity, constancy, always brings me back to where I need to go. I reach into that real center to find myself again.

I wonder how a trip to New Zealand sounds?

25+ points about me.....

2009-01-24T13:57:00.000-08:00

In Facebook, there was a mail going around amongst friends where we were supposed to write a note - 25 things about ourselves. What better material for a blog about me? (I may update this in the future with more points, but for now here goes).

1. Everyplace I have ever lived, I dreamed there was a mountain near me, and I would climb it. I rather like the far north.
2. Years later, I found a place a world away that I could swear was the same place I dreamt about repeatedly as a boy. (When my friends and I went back to the forest to find this hilltop - we couldn't locate it again.)
3. I consider myself the least funniest person in my immediate family - but to my friends, I'm a laff-riot.
4. Even though I love them to death, I've never been geographically close to my folks, don't see them personally so often, and I'm cool with that.
5. I tend to find ways to avoid bureaucratic unpleasantries. Or dealing with bureaucracies of any kind. Whatsoever.
6. Even though I've been working with technology for most of my working years, I despise debugging /troubleshooting poorly - or even well - implemented products. I have exactly *one* requirement on usability - it should work like a train's toilet! Devices, networks, drives, partitions, yer momma's anti-virus, Skype not working right- I'll have none of it. No, I will NOT fix your bloody computer!
7. I've become fascinated with mathematics - particularly calculus - to the point where I will now sit up until 3 a.m. graphing some weird function or pondering some proof. (Heim's unification is the latest). And get up at 7 a.m. to continue. I'm also intrigued by the idea of a grand unification theory which unites general relativity and quantum mechanics, but don't believe that reconciliation exists yet - perhaps with our current understanding as humans we just can't get it.

8. Among other things I believe there's a workable alternative (we haven't discovered it yet) to our economic system (y'know, the one based on money?) that would solve a hell of a lot of the problems we face as a race on this planet. (What's the biggest shortage right now in our current global downturn - that's right - money!). No shortage in the world whatsoEVER of goods, services, food, people able to do things! And yet people are sitting at home, unemployed, with no way to manage. What's UP with that?

9. Even though I live in Sweden at the moment, and am thoroughly inspired and fired up by our new U.S. leadership, I'm nonetheless thoroughly convinced that the less government there is, the better. I consider myself a libertarian.

10. When I was envisioning my current home, which I'm crazy about - even though I don't own it - I was living at the time in a early soviet flat in Tallinn Estonia. That place had an interesting latticework pattern in the floor and a tattoo shop directly below. So I held a picture in my mind about where I WANTED to live, (it was based on my current surroundings of course, what else could I do?) focused on it, meditated on it, and - guess what? Years later, I've got the same floor and the tattoo shop directly below.

11. I always wanted to create my own reality, job, career, complete with travel - and I have!
12. No use for family and kids. But thanks anyway.
13. I value my freedom over my security. Bigtime.
14. I've always believed in my own power to be the captain of my own life and destiny. But I have lost that compass sometimes - those were my darkest moments.
15. Surprised that I haven't uttered a word about my music yet?
16. I've lived on both sides of that paradox - i've been well off and broke, self-determined and dependent, completely social and totally isolated. Seems like I've needed one to thoroughly appreciate the other.
17. I often soak up other bits of people's personalities and quirks and merge them with my own. Who you see as you observe me depends very much on who - and when - you are.
18. My ex's little daughter Susanna was thoroughly convinced I was a cartoon.
19. I rather enjoy myself and my own company.
20. After I left Finland, my Finnish began to improve dramatically. Same thing with my Estonian. (Kas te järgmises väljute?) Currently my Swedish right sucks.
21. During my university years, I used to get alot of praise for my creative writing, and even published some things, travelled around to recitals, etc.. Now you couldn't get me to write a poem or short story if you put a gun to my head. I'm just far too lazy about it. I keep a blog and hardly ever update it. But I do take immense perverse pride in the voluminous technical specs I've written for some of my jobs (some in the hundreds of pages).
22. I'm compiling a book of the best pranks of my friends - "1001 ways to be an unwanted guest". Submissions, anyone?
23. I'm fascinated by how the true nature of a particle - or person - reveals itself over time. Nothing - no one - can escape that. 4th dimensional thinking?
24. Ok. About the music then. Much of what I produce nowadays (and I'm producing alot) are "experiments" with different harmonies, instruments, tambres, recording techniques, microphones, etc. I may never release any of it, although probably some.
25. I've played with some really great players. But I pride myself in how I've been able to get great results out of whoever I happened to find.

in Madrid, thanks to Iberia Airlines strike....

2009-01-06T10:45:00.000-08:00

As I'm somewhat new to blogging this way (the CMS at my original website, johnhammink.com, has been broken for more than a year, and I'm too lazy to fix it), I'm simply not in the habit of reporting the circumstances of my life, travels, and travails from one day to the next. Indeed, as someone who works professionally as a writer (essentially), I'm not even accostomed to posting unedited thoughts in forums like these. Nonetheless, I have always been pretty committed about reporting crappy services and situations, as well as celebrating the good ones, so, as they say, there's no better time than the present.

I'm supposed to be in Santiago, Chile right now, giving a panel discussion http://www.mite.usm.cl/index.php?id=96 for the local media, and later teaching a master's degree module on consumer product realisation. Instead I'm in a cafe in Madrid. After flying in a very dirty plane, which was more than four hours delayed from my origin, I missed my LAN Chile (a very good airline) connection.

When we arrived in Madrid airport at 12.30 a.m. there was no information point, or person at the exit gate to redirect us to the place we were supposed to get our transfers. Instead, we were forced to go out of the country, through passport control, then back into the country (through passport control) which resulted in two stamps I really didn't want in my already nearly full passport. When we finally (on our own, no help from airport staff) finally figured out where we were supposed to go, we stood in that line for more than three hours, while they issues our transfers)!

Being diabetic, I went out to ask someone (one of the iberia clerks had wandered off to talk to his buddies) where I could find food so that I wouldn't go into shock. I was then told to go stand back in line and wait for my ticket (it turned out that there was a cafe downstairs). When I finally got to the desk, and insisted on a connection through LAN Chile, rather than Iberia, it took all of 2 minutes.

Iberia finally put us in a hotel, which although it was three stars, was inflexible about meal times.

So Madrid is nice (was nicer back before the EU), but I'd really rather not be here right now.

A couple of lessons I've learned.

When travelling with a plane full of South Americans, don't do the nice Scandinavian thing and expect other waiting travellers to exercise the same forbearance and respect for lines as you have (As I already knew, but my Scandinavian friends did not realize - South Americans have none - after an asshole-to-elbows scramble, we were last in line, while our tickets were the easiest to process).

Madrid airport is a mess. Don't fly there. And although Iberia respected our passenger's rights (they have to, don't they?) they go on strike several times a year, so somethings not right there.

In Europe, Consumer's rights and integrity are clearly absolutely LAST priority, if in practice they are even considered at all, light years behind those rights of public officials, followed by trade unions.

Why do other developing countries' services (LAN Airlines, Chile; Thai Airways) perform so much better in terms of customer treatment, and those services from nations where standard of living is supposed to be the best (SAS, Denmark; KLM, Netherlands, to name a few) offer, by far, the worst customer service?

on Fox news...

2009-01-01T19:06:00.000-08:00

....I just had something of a revelation (or just a question?) about inflammatory shows like the O'Reilly factor and fox news? I mean what if these guys were actually doing us a favor by acting as a counterpoint to public opinion? Public sentiment being as divided as it now is.

One could argue that Fox news, while seemingly a creation of the Republican party and right wing agenda as such, is actually a reaction to it. I can see people starting to watch it and thinking "wait a minute, this doesn't seem right", while actually starting to levitate in the other direction. What if this was created to show how absolutely preposterous that so called "fair and balanced" viewpoint were, in order to get people to oppose it?