The Dutch and Chocolate

Today a parcel arrived from the Netherlands. Which means it must be Christmas, in a way. My parents send out a shipment of Dutch goodies once a year, and sometimes it gets here in December, but usually it is easier to aim for another less-postally-congested month.

2015-03-30 - Step 1

Chocolate features heavily in the parcel.
There are some reasons for this.
Not good reasons, but… good enough for me.

2015-03-30 - Step 2

I have no idea where historically/culturally our predilection for starting the day with “beschuit met hagel” (crispbakes with chocolate sprinkles) comes from. But I dare you to find a household in the Netherlands that doesn’t have at least one pack of hagelslag somewhere in a cupboard.

2015-03-30 - Step 3

It used to be very difficult to get hagel in Australia. It is getting a little easier, even if the cost if extravagant compared to back “home”. And it always takes people aback when I explain it is a breakfast topping. My parents have been sending annual care packages with hagel for forever, but since it has started becoming available here too, I tend to get sent the more luxury flavours, like Mocha and Extra Dark.

2015-03-30 - Step 4

But the chocolate most directly tied to Sinterklaas (the Dutch equivalent of Christmas) is the chocolate letter. The Chocolate-Initial-Complex is a force unto itself in the Netherlands.

2015-03-30 - Step 5

And those that know me well, know exactly what happens when chocolate letters arrive. I really have no trouble leaving chocolate be, as my former experiments with The Chocolate Drawer have shown at my previous job.

2015-03-30 - Step 6

But when it comes to initials, the only way to consume them is all in one go.
That’s my story.
And I’m sticking with it.

…Back Again

I was looking forward to my Body Attack class for Tuesday evening, right up until Marayong on the train. I pick up my laptop backpack and have a telltale twinge in my back that I’ve come to respect as a sign to straighten up and pray it’s not too late to avert certain doom.

Although the evening takes a different course from the one I had planned, I don’t think anything more of it. Two episodes of House, M.D. – “It’s not Lupus”, and a failed attempt at doing some work (dev environment in some inconsistent state I couldn’t figure out right then), and feeling very weary by 9:30pm. I go to bed early.

I spend an hour doing that thing where you have audio playing that you think you’re actually listening intently to, staving off sleep… but then you startle awake and realise you’ve missed the first two questions-and-answers of the Writing Excuses podcast, and you just cannot bring yourself to fiddle with the controls to reset it to the start. I take it as a sign I should actually sleep and turn the audio off altogether.

At 3am, I know I am going to be in trouble. I wake up flat on my back, and my back has seized up completely. I have a 15 minute internal dialogue with myself about what to do next. I consider staying flat, but my back is feeling very uncomfortable. I consider rolling over but am dreading the feeling of using my shoulders to push myself around. I consider getting up to go to the toilet; damn, bladder full as well, just what I need.

No matter the circumstances, I almost never forget to think ahead.

I swing my legs over the side of the bed, and somehow manage to swivel myself into a sitting position with minimal use of my back or shoulders. First, I switch off the alarm on my phone, because I doubt I’ll wake up at 6:30am this morning, and even if I do, I know I’m not going to be nimble enough to dive for the alarm in a timely fashion when it eventually rings. Then, I go to the toilet and seriously consider whether I can sleep upright to limit the tension in my back. Clearly 3am is not a time for reasoned arguments.

For the remainder of the night, I alternate rolling onto my left and right sides with long stretches on my back. Each time waiting until the discomfort grows to a level where it overcomes my fear of the twinges that’ll run through my back when I go for a roll. Adding to that, putting my top arm in front or behind my body is immediately punished with stabs along my arm and shoulder blade, and it makes for an interesting night of careful balance in a half-sleep-state.

I don’t even consider going to work when I finally get up at about 7:15. I leave a message to my team-lead and start working out what I can do to speed my recovery up.

Luckily, it turns out the place I get my massages has a free slot at 10am still, so I book that in first, and then head to the doctors’ office down the road after a shower that takes me to their opening hours.

I bring my Kindle to try and read, but holding it up hurts my shoulder, and balancing it on my knee hurts my neck. I uncomfortably read half a chapter by holding it resting on the top of my knee with my neck slightly tilted and my eyes rolled down. I’m sure I look like a bad version of the Mona Lisa. It feels odd. It still feels uncomfortable.

Luckily the waiting room is practically empty. I try to look at the people in the waiting room with me, but my head doesn’t want to turn further than 45 degrees either side. I stare at the morning shows on the television. I stare through the inane dialogue.

I get a prescription for rest and some Panadeine Forte.

I head out to my massage and arrive 30 minutes early, so I read uncomfortably some more, after taking 2 tablets. Three-quarters through the chapter my head feels swimmy and it feels like reality is my imagination. Deep breath. I fail to read further and listen to a podcast instead.

Getting a massage with Codeine in my system proves an interesting experience. I can feel all the knots, but none of the pain. It is surreal, but far from unpleasant.

I feel relaxed and less sore right now, still a little detached, and absolutely determined to take carrying my backpack properly more seriously. It occurs to me that all those times I traveled to Melbourne and had back aches afterwards probably had little to do with the hotel beds, but everything with the fact I carry my luggage on my right shoulder only. It doesn’t feel heavy to my brain, but clearly my muscles privately have a different opinion. I’ll listen to them more from now on.

Regular Like Clock-work

That is to say; with a whole bunch of wobbly and spinny bits that nobody quite understands the need of, but without which the mechanism just suddenly fails in spectacularly unpredictable ways.

You guessed it… Regular Expressions.

The greatest, most terrible tool, ever invented to do quick-and-dirty validation in web front-ends around the Interspaces.

I’ve been working to improve first-pass URL validation logic in the web front-end. I started by trying to read the existing regex, but it looked like a cat had just mashed some random symbol keys to a length of about 200 characters. And I knew it wasn’t allowing all the URLs we’d like to accept.

I decided to go back to first principles; RFC 3986 – URI Generic Syntax. The first shock was learning that the following is a perfectly legal URL:

http://jerryjvl:password@[FE80::0202:B3FF:FE1E:8329]:8080/
This:would/Be-Funny+(if);I-didn't/Have-to?parse=it#sadface

And I haven’t even used Unicode characters anywhere in that example yet.

First, the temptation is to go to the back of the RFC, and just translate the BNF notation into a Regex and be done with it. Alas, I didn’t think I could accurately transcribe that many permutations without slipping up… and regexes are hard enough when you have a clear idea of what exactly you are parsing.

Second, the important realisation that it doesn’t have to disallow everything that isn’t a valid URL. This is about helping the users by catching the most important mistakes they might make. If anyone decides to actually use an IPv6 literal as a host identifier, then it really isn’t important to check whether the exact right number of hex words were used.

So, when squinting just-right at the RFC, it is easy enough to come to the following right-to-almost-right rules:

  • The group [\w\$-\.!;=@~] is a great approximation for the permissible alphabet for most of the textual parts of a URL; in some places that might allow slightly too much, but it restricts all the characters that really do not belong.
  • “#” is only permitted exactly once, after which all further text is considered the fragment identifier.
  • “?” is not permitted until the query portion at the end of the URL, but can occur as many times after that as you want.
  • Allowing excess square brackets makes capturing the part between the “//” and the first following “/” easier. Making the expression more specific helps break down the results into more logical parts.

What I have landed on for now is the following (finessed so that the capturing groups try to catch the logical parts of a URL):

^
  (?:(https?|ftp):)? # URL Scheme Identifier: http, https, ftp
  (
    \/\/ # Literal //
    ([\w\$-\.!:;=~]*@)? # Followed by optional username:password@
    ([\w\$-\.!;=~]* # Followed by hostname
    |\[[a-fA-F0-9\:\.]*\]) # Or IPv6 address
    (\:\d*)? # Followed by optional :port

    |\/[\w\$-\.!;=@~]+ # Or literal / and a path segment

    |[\w\$-\.!;=@~]+ # Or no slashes and a path segment

    | # Or... nothing at all!
  )

  ((?:\/[\w\$-\.!:;=@~]*)*) # Rest of the URL path

  (\?[^#]*)? # Optional query: ?...

  (#.*)? # Optional fragment: #...
$

I’m a little sad that named groups are not available in Javascript; remove all comments, white space and line-breaks from the above, and you can expect the capturing groups to contain the following:

  1. The scheme: http, https or ftp
  2. Either “//” followed by a host (authority), or otherwise the first part of the path
  3. The username:password@ of the authority, or nothing if absent
  4. The hostname from the authority
  5. The :port of the authority
  6. All of the URL path if there was a host (authority), or otherwise the remainder of the path after the first level
  7. The ?query portion of the URL
  8. The #fragment portion of the URL

Clearly some more post-processing needed to extract the actual values if you want to. Although I strongly recommend using a proper Uri class if you really want to process the content, rather than just getting a quick yes/no whether a URL seems plausibly valid.

Next stop… email addresses – RFC 5322.

As agonising as all this sounds, even to me, I am actually having a great deal of fun right now.

Agility

I joined Blackbeard only recently.

I was looking for a new challenge, and I found it in the Inn where he was resting his party with Friday beers and cheese platters. He was looking to take on an Epic quest, and could use all the skilled hands he could get.

His party was an unfamiliar mix of classes to me, which has something to do with agility and sprinting. I cannot argue with the results of a mixed party I have seen so far.

It is still early days, and I joined well into the current campaign. But the Stories of our Epic goal are enough to inspire a renewed sense of purpose in me.

I’m part of Team Blackbeard, our quest is bringing flawless E-Mayl to the masses.
(I think it’s a precious stone or something. I’m not entirely clear yet.)

Checking confluence with pythons

Heaven and hell.

Being a new starter at my new job, eager to absorb as much information as I can in as short a time span as possible, my attention immediately gravitated towards the Confluence Wiki with its plethora of pages (1800+) as a starting point.

There is a lot of information there, but the organisation of it could be better, so in line with the take-the-ball-and-run-with-it culture, I did.

First concern: how to improve things without making them worse?
Broken links suck.
Broken links I created, suck even more.

Sadly, and inexplicably, Confluence to this day does not include any tools to check for broken links, beyond the basics of “Orphaned pages” and “Links to new pages that haven’t been created yet”.

A quick search on the internet found a tool by an Atlassian employee that seemed promising: BustedStuffReport. Point it at a Wiki and it will scan all the pages and do some regex-magic to heuristically determine if all is in order.

Sadly: it only works on public Wikis, it does not follow any links to check them, it seems to target a somewhat older version of Confluence, it uses Python 2.

Having most of a solution already there, I decided I could hackimprove it to make it useful enough. Just let me get the Python language reference out and see what happens!?

After a week of playing around after work and in between tasks, I have a mostly rewritten script that is converging on the target I want to hit. I’ll post on Sciurus with the full details once I get there.

In the mean time, the experience has taught me the following:

  • The XML-RPC API to Confluence is very rich and regular, and reaches into almost all the corners I need (Yay!)
  • This XML-RPC API was deprecated in favour of REST,… while the REST API has not yet reached functional parity (D’oh!)
  • Confluence very helpfully does *some* classification of links through CSS classes… no idea why this isn’t visually represented by default (Huh?)
  • Python makes it incredibly easy to access the XML-RPC (Yay!)
  • Python still makes my skin crawl with its lack of type-safety, and no, I don’t want to write unit tests for a small tool like this (Boo!)
  • List comprehensions are awesome (just like LINQ is (double-Yay!))
  • Why do I need to end my conditional statement lines with a colon? I guess I can live with this, but for a language that strives for visual sparsity it seems like an odd requirement (*shrug*)

I think I can see why people love Python for scripts. But I’m still not convinced the productivity gained by the flexible typing system isn’t overshadowed by the extra test-cases you’d need to code in a non-trivial application. So, that leaves trivial for now, for me.

Culture

Great corporate culture is easy to spot; just count the happy employees. I’m sure there are studies into the benefits of a positive culture, but I doubt you need studies to be convinced anyway.

Creating a great culture can be incredibly hard though. And regaining it after losing it is just as hard itself. It is like exercise; the amount of effort it takes to regain condition after sitting on your ass for a month disproportionately overshadows the effort it would have taken to maintain the routine for that same period. It’s rolling that boulder back up-hill.

So, I guess the first rule of Culture Club is: maintain culture for all it’s worth.

What makes culture hard is that you cannot afford words and actions to become misaligned. Intent alone is not enough, because perception beats intent every day; if employees don’t believe it, you’ve lost already.

Which means that the second rule of Culture Club is: make sure you pick a culture you are prepared to stand behind with every action, pick the culture you mean, be specific… platitudes do more harm than good. Everyone wants to be the “company full of smart empowered and pro-active individuals”, but just don’t say it unless you’re prepared to live by it.

I mean, look at McDonalds… there are manuals and procedures for everything. No need for anyone to be empowered or pro-active. But at least everyone is on the same page regarding what McDonalds culture is like. Which means that nobody ends up complaining about it. It Does What It Says On The Label.

And there are many stops along the spectrum from completely running on process* to completely running on people*, so pick the spot that is right for your organisation and live and breathe it.

And that would be the third rule of Culture Club: once you have a culture that is realistic for your organisation, everyone has to work at keeping it there. You have to look for things slipping and nudge them back where your culture says it belongs.

I suspect that may well be one of the biggest purposes that the playful decor at a place like Google has in maintaining their culture. It’s not a direct part of the culture, but it serves as an overt reminder not to take things too seriously. Breaking traditional corporatey-officey rules in the way you decorate immediately makes clear not to make assumptions about how things work around the place.

And I couldn’t be happier at Campaign Monitor; it comes with a great built-in culture close to the People-based end of the spectrum. And I’ll actively be doing my bit to help keep it there along with everyone else.

Just a little ongoing maintenance is all it takes.


* It is almost impossible to maintain a culture that is purely process-based or purely people-based. The more process-based a culture, the better it is to have a publicly acknowledged “why” for each rule and process; it helps the “people”-people. The more people-based a culture, the better it is to make clear that not being sure of how to handle something is okay, and a process to follow to find the right person to help; because you’re going to have “process”-people as well.