YOW Conference – #yow15

I had not been to YOW before myself. In my previous job I had sent my team there though; the timing works out perfectly for a business that tends to be quiet over Christmas and the new year.

I had been to TechEd before, and that was the closest benchmark I had.

YOW definitely wins on a broader scope of topics, and yet not so scattered that it was hard to find a relevant topic in each session timeslot. I have been to TechEd with timeslots reserved for walking the floor, because despite 5-6 sessions running in parallel there wasn’t one that I actually wanted to see.

TechEd wins on the catering front, hands-down. But then, in all the years I went, it never ceased to amaze me how well-oiled the Microsoft conference machine is, and the amount of effort they put into a great show.

Having said that, YOW wins hands-down on cost… it’s kinda the way they pitch their conference – focused on the technology and affordability at the expense of spectacle. I don’t think that’s a bad call, because TechEd was definitely never a trivial sell to management.

And yet… the one point I have a hard time qualifying specifically; YOW doesn’t feel as amenable to making new connections on the floor. During sessions the focus is on the speaker, and during breaks it is on getting some food and finding a place to eat it. Maybe it’s the TechEd lunch set-up around large tables that force you to sit down with strangers that helps? Maybe it’s the social evening of fun they put together (one year: go-kart-racing in the parking garage… yes… seriously) that makes friendly conversations easier to come by. I don’t have a clear answer, but YOW does feel stiffer than TechEd.

Having said that… if you’re just there to learn, the set-up is ideal.

Keynote 1 – It’s Complicated…
Adrian Cockcroft

To my shame, I barely remember the opening keynote.

I hadn’t had coffee yet.
I was trying to work out when I was going to be where.
And then I had 11 sessions and 3 more keynotes with memorable elements.

I think it just displaced (almost) all my memories.

The one thing I recall with clarity is the question how it can be that the most complex piece of technology in existence today is usable by kids as young as 2 years old in a meaningful way; the mobile phone. Something about it’s design is the pixie dust that hides the million things you don’t want to do right now.

Session 1 – Rethinking MVC with React Native & ReactiveCocoa
Ben Teese and Sam Ritchie

I was only superficially familiar with React before this talk, so it probably wasn’t aimed at me. But I still learned a lot about the way React uses it’s Virtual DOM to do delta-updates on-screen, and how that can be made to translate to native apps on mobile devices as well.

I believe Angular 2 is going down a similar path of supporting some kind of Native variant, although I may have just dreamed that. Either way; not quite relevant to me at the moment.

Great talk though, good speakers, especially the groan-worthy punny pictures.

Session 2 – 40 Agile Methods in 40 Minutes
Craig Smith

Spoiler: it’s actually 50+ minutes; the session ran a little too slow to make good on its titular promise. Having said that, it was a very enjoyable whirlwind through a lot of Agile Development approaches (some of which didn’t look anywhere near as Agile as they purported to be).

There were a few slides that specifically piqued my interest, but the pace prevented me from taking notes, so I look forward to the slides getting published. Especially the slide with a book that Craig recommended as providing a good underpinning of why Agile works.

Great talk, and more useful than the frivolous title might lead you to believe.

Keynote 2 – Using Formal Methods to Eliminate Exploitable Bugs
Kathleen Fisher

Oh yes.

Totally this.

It’s been only 17 years or-so since I graduated from University, and at long last the central pillar of the Computing Science program of Eindhoven University of Technology is actually becoming useful.

See ma, my degree has practical applications in the real world!

And apparently, Australia is one of the fore-runners in this field. I don’t want to say it’s because of me, but hey… I’ve been here 17 years now. Coincidence? I think not!

In all seriousness, it is great to see Formal Methods taking their rightful place as a central tool for the development of provably correct software.

Session 3 – Adaptive Security
Aaron Bedra

The key tenet of this talk was: exploit your logs; exploit them for all it’s worth.

Know what your messages mean, count them, and then look for patterns. And then act on those patterns. And start simple, because the business knowledge that produces will lead to requests for more of the same, and before you know it you’ll be tracking and measuring everything.

I couldn’t think of better real-world advice.

Even beyond just security matters.

Session 4 – Production Haskell
Reid Draper

This was by far the greatest disappointment of the conference to me. Based on the excerpt I had hoped to see some samples of practical use of Haskell in a real-world production scenario.

In the end I walked out before the session was over, because I just couldn’t muster the will to look at further tool-chain scripts to build Haskell. That was so not the point I was coming to see.

I’m sure I could figure it out myself, but I wanted to come and be sold on the idea.

Session 5 – Pragmatic Microservices: Whether, When and How to Migrate
Randy Shoup

I slumped through this; not a bad talk, but after a full week of Udi Dahan, there wasn’t really a lot more anyone could tell me about big balls of mud and how to take bites out of them.

I had hoped those nice big open questions in the title would lead to new practical insights, but I think I just kinda zoned out and let my afternoon snack digest.

Not a bad talk, just nothing much in it for me.

Session 6 – Property Based Testing: Shrinking Risk in your Code
Amanda Laucher

This talk felt like a “missing link” between Formal Methods, Unit Testing and .NET Code Contracts. After listening to it all, I like the idea of higher-order tests, and I see how you could leverage them in procedural languages.

But it feels like perhaps it’d be easier to just go Functional, or use Theorem Provers instead of wasting this approach in its pure form on C#. Still, I like the ideas underpinning this way of testing, because as I’ve blogged previously, I’m very unsatisfied with the cost/benefit balance of most of the automated testing I have been exposed to in my working life.

Keynote 3 – Engineering and Exploring the Red Planet
Anita Sengupta and Kamal Oudrhiri

Anita is a great speaker. Kamal was clearly nervous.

Having said that, it’s hard to botch a topic as inherently interesting as trying to land complex and fragile technology on another planet within a very small target area. It’s hard to appreciate how awesome the stuff is that JPL and NASA do without someone explaining all the details that you’d never have thought of.

I secretly suspect there are still a lot of Waterfalls in these places to deal with the careful design required for a release that you don’t get a second chance at. Once it leaves the planet, it better work.

Also, it made me want to go and see The Martian again.

Session 7 – Building Microservice Architectures in Go
Matt Heath

This was actually a very fun Microservices talk. Not as much hands-on Go as I might have hoped, but some very salient points were made. It hadn’t occurred to me before that the combination of a language that statically links everything together with a very lightweight container is immensely powerful for Microservices / Service Oriented Architecture.

I guess he just made a very compelling case for dotnetcore without even realising it.

Also, he was an excellent and engaging speaker. He felt by far the most polished out of the speakers I saw. Having said that,… the thongs were visually incredibly distracting.

Session 8 – Agile is Dead (Long Live Agility)
Dave Thomas (Pragmatic)

And just as I thought thongs would be about as distracting as it could possibly get. Bare feet on stage.

After this talk I feel very self-conscious about my use of the term “Agile” as a short-hand for Agile Development Practices. A very well-put rant against the commercialisation and co-opting of common-sense to extremes where it just stops making sense altogether.

I suspect the fifth agile tenet that he can no longer remember might have been “Pizza over Steak”; that sounds like something a programmer would declare.

I guess the biggest lesson from this session is a cautionary tale; to keep an eye on the practices you follow and to make sure you don’t fall in the trap of trying to buy Silver Bullets. We all love them so much, and we know they don’t exist, but we still buy ’em.

Keynote 4 – Thriving in a Stochastic World
Don Reinertsen

And this one takes the title for “dullest yet most worth taking note of”.

The speaker reminded me of a professor with a slow drone. I’m glad I managed to barely keep my eyes from shutting, because the key thesis on how to exploit the asymmetry of the upside and downside of experimentation in an unpredictable environment is a great lesson for all start-ups. Heck, all technology businesses.

I would have hated to miss that nugget.

The lead-up to it, I could have done without I think. Maybe start closer to that point and then spend more time on concrete examples, and it’d be a much more relatable talk.

Session 9 – Making Hacking Child’s Play
Troy Hunt

This one was a lot of fun. Also terrifying.

For opening gambit: a dumb YouTube video showing a terminal with a clueless teenage voice-over explaining how to DDoS someone with a “ping -t” command, and how it’ll only work if “your network is powerful enough”.

A brilliant feint.

Later in the session, the most terrifying thing ever, an early-teen girl, in her early-teen bedroom, speaking into a laptop webcam selling DDoS services to knock your gaming competitors off the net. And completely serious and real.

We live in a world where the tools to disrupt services on the internet can be wielded by the completely clueless. It’s like a phaser; you just point and hit the button and stuff happens.

Very effective presentation.

And also, we’re all doomed.

Session 10 – DevOps @ Wotif: Making Easy = Right
Alexandra Spillane and Matt Callanan

Another talk that didn’t quite make good on its title, but nevertheless a talk with some interesting points.

Basically, Wotif ended up crawling out of the pit of despair by creating a better deployment story, but rather than using the hard-sell, they developed it alongside their existing deployment path and then let market economy take care of the rest.

“Do you want to go in the release queue, wait weeks, then have your code hit production; all safe and secure, or would you like to use this faster SLIPWay which can turn around your deployment in an hour, but you’ll have to change a few of your assumptions and processes?”

These were the only paired speakers that had put their talk together so that their perspectives complemented each other well. Not flawless, but definitely seamless.

Session 11 – Play in C#
Mads Torgersen

The biggest under-sell of YOW.

The title doesn’t do the content of this session justice by a long stretch.

For warm-up, we walked trough the history of C# (Mads leads the language team at Microsoft), with some miscellaneous barbs and snipes aimed at Georges Saab who did a Java talk before this session.

“C# 1.0, back in 2000, where we introduced value types”, /significant long silent glance to Georges/.

Poking fun was only the secondary purpose of this quick retrospective, because his real purpose was to show the language evolve to where it gained Roslyn. And then he went into a live demo.

He starts up Visual Studio.
He starts a language rule project.
He starts a nested Visual Studio within which the language rule he is developing lives.
He edits the language rule with code-and-resume.

And as he adds this new language rule and it incrementally applies squigglies in the nested VS, and then adds automatic correction options to apply fixes to the code; I want to play with Roslyn so desperately now. It is FxCop on steroids. It is magical. And also a little meta-insanity. But the good kind.

And then to finish he runs through some of the new language feature options on the table for C# 7. Note that hyperlink right there. That’s the GitHub repo where Microsoft keeps the live list of language feature discussions going on for the next version of C#. Microsoft are now not only open-sourcing their framework, but they have also opened up the design process. So have a look around and be part of the discussion!

It sounds like Pattern Matching by types, strong Tuples and non-nullability are strong contenders for features that might be in. But no promises just yet.

I could not have wished for a better closing session, because it sent me into the weekend very energised. I then proceeded not to play with Roslyn for lack of time after my other chores, but I think that flame will burn through a while longer.

My next goal: devise a talk worthy of YOW and get onto the speaker roster.
It is easy to criticise, but much harder to step up and do it.

Regular Like Clock-work

That is to say; with a whole bunch of wobbly and spinny bits that nobody quite understands the need of, but without which the mechanism just suddenly fails in spectacularly unpredictable ways.

You guessed it… Regular Expressions.

The greatest, most terrible tool, ever invented to do quick-and-dirty validation in web front-ends around the Interspaces.

I’ve been working to improve first-pass URL validation logic in the web front-end. I started by trying to read the existing regex, but it looked like a cat had just mashed some random symbol keys to a length of about 200 characters. And I knew it wasn’t allowing all the URLs we’d like to accept.

I decided to go back to first principles; RFC 3986 – URI Generic Syntax. The first shock was learning that the following is a perfectly legal URL:


And I haven’t even used Unicode characters anywhere in that example yet.

First, the temptation is to go to the back of the RFC, and just translate the BNF notation into a Regex and be done with it. Alas, I didn’t think I could accurately transcribe that many permutations without slipping up… and regexes are hard enough when you have a clear idea of what exactly you are parsing.

Second, the important realisation that it doesn’t have to disallow everything that isn’t a valid URL. This is about helping the users by catching the most important mistakes they might make. If anyone decides to actually use an IPv6 literal as a host identifier, then it really isn’t important to check whether the exact right number of hex words were used.

So, when squinting just-right at the RFC, it is easy enough to come to the following right-to-almost-right rules:

  • The group [\w\$-\.!;=@~] is a great approximation for the permissible alphabet for most of the textual parts of a URL; in some places that might allow slightly too much, but it restricts all the characters that really do not belong.
  • “#” is only permitted exactly once, after which all further text is considered the fragment identifier.
  • “?” is not permitted until the query portion at the end of the URL, but can occur as many times after that as you want.
  • Allowing excess square brackets makes capturing the part between the “//” and the first following “/” easier. Making the expression more specific helps break down the results into more logical parts.

What I have landed on for now is the following (finessed so that the capturing groups try to catch the logical parts of a URL):

  (?:(https?|ftp):)? # URL Scheme Identifier: http, https, ftp
    \/\/ # Literal //
    ([\w\$-\.!:;=~]*@)? # Followed by optional username:password@
    ([\w\$-\.!;=~]* # Followed by hostname
    |\[[a-fA-F0-9\:\.]*\]) # Or IPv6 address
    (\:\d*)? # Followed by optional :port

    |\/[\w\$-\.!;=@~]+ # Or literal / and a path segment

    |[\w\$-\.!;=@~]+ # Or no slashes and a path segment

    | # Or... nothing at all!

  ((?:\/[\w\$-\.!:;=@~]*)*) # Rest of the URL path

  (\?[^#]*)? # Optional query: ?...

  (#.*)? # Optional fragment: #...

I’m a little sad that named groups are not available in Javascript; remove all comments, white space and line-breaks from the above, and you can expect the capturing groups to contain the following:

  1. The scheme: http, https or ftp
  2. Either “//” followed by a host (authority), or otherwise the first part of the path
  3. The username:password@ of the authority, or nothing if absent
  4. The hostname from the authority
  5. The :port of the authority
  6. All of the URL path if there was a host (authority), or otherwise the remainder of the path after the first level
  7. The ?query portion of the URL
  8. The #fragment portion of the URL

Clearly some more post-processing needed to extract the actual values if you want to. Although I strongly recommend using a proper Uri class if you really want to process the content, rather than just getting a quick yes/no whether a URL seems plausibly valid.

Next stop… email addresses – RFC 5322.

As agonising as all this sounds, even to me, I am actually having a great deal of fun right now.

Day 112 – Developing My Project

This Post

This post will get progressively more technical. Apologies if this does not appeal to you. We’ll be back to my regular everyday topics shortly.

Just let your eyes glaze over slightly and let the words fall through your brain.

It’ll almost be as if I’m making sense.

The Project

In the spirit of not biting off more than I can chew, I’ll start with something relatively modest. Something that undoubtedly already has many solutions. Something that is a stepping stone to a bigger problem I’d like to solve.

Baby steps though…

I have a very specific workflow for dealing with my RSS feeds:

  1. In the mornings, and sometimes once or twice throughout the day, I’ll pop into Feedly to skim through post headlines
  2. For every headline I make a snap decision whether it’s something I’d like to read more about (Star), or not; this usually triages my feeds from 200-300 articles for the day down to maybe 10-15 of real interest
  3. At some later point I get back to them, and I read the articles
  4. At long irregular intervals I go through the starred posts and un-star any that have no lasting value, leaving stars on anything I want to keep for reference

And that’s where it usually ends.

But I’d really like the third step to immediately result in a classification of “Not worth keeping“, or “Worth keeping under category X“. I want my workflow to keep up a well-organised library of links to refer back to later so I can actually find stuff.

So, a workable project to me seems to take RSS and my intended workflow and mash those together into a minimalist web application.

The Iterations

I’m going to have to learn many aspects of web development for a somewhat polished result, but I don’t want to overwhelm myself with too many concerns simultaneously. So, I think I’m going to have to do several versions of this project.

  1. The prototype with just the bare-bones workflow and lots of ugly hard-coded crap to get my head around MVC and Knockout.js
  2. The maintainable version where I tidy up the internals and introduce Dependency Injection (not to be confused with an injection dependency which falls outside the scope of technology) and unit testing/mocking
  3. The pretty version where I apply some CSS3 and JavaScript transitions to make it all look better within the confines of my design sense 😉
  4. The secure version where I introduce some form of account management / login, and analyse the code for typical attack vectors (session hijacking, insufficiently secured requests, etc.)

I am sure there will be another couple of important versions past this that I’m forgetting about now. But that seems like a manageable separation of concerns. I fully expect to throw lots away between these iterations.

The Tools

I’m a .NET developer, so this will be very .NET skewed.

At the moment I’m thinking of using the following tools:

I wonder which of those will prove to be bad ideas in retrospect? Luckily, with at least 4 iterations ahead of me, I’ll have at least 3 opportunities to drastically change my mind 😉

Oh, and I should probably have mentioned GitHub, where I’ll be hosting my attempts. I may even have a look at the issue tracking built in to GitHub to see if it suffices for the scope of this project. Otherwise I might have to set up my cheapie 10-user Jira license locally somewhere.

This is all both daunting and exciting at the same time.

I suspect the first iteration will take the longest because there will be the most reading material associated with getting the first attempt off the ground. I’ll try to blog about that as well along the way. I’ll try to make it a separate stream to the 365-challenge, lest I bore my non-tech audience to tears.

Planning and Learning

This morning I successfully acquired the right SIM for my new phone. The transaction at Virgin Mobile Blacktown was quick and painless. I will need to spend a few more days making sure I get everything I care about across from the Galaxy Nexus to the Nexus 5, but then I can wipe the former and dispose of it.

In an hour I have a 30 minute cycle class to break up the day.
And then…

I’m going to start on a Web 2.0 adventure. I’ve meant to learn some new skills and build something for too long now. Time to bite the bullet.

I have the outline of an idea for a set of related problems I want to solve (scratching your own itch has been proven the best place to start). I have VS2013 Beta on my machine (so an upgrade will be in order first). And I have a list of compatible technologies to explore.

I guess this also means my personal GitHub account will finally get some love.

Wish me luck!

Day 79 – Driven to Distraction

I did some programming tonight. I got so absorbed in what I was doing that I completely lost track of time. *yawn*

I was actually playing around with some JavaScript for the first time, and it was an educational and frustrating experience. I really need to have a bit more of a play with Visual Studio to figure out what tooling is available for debugging.

Biggest frustration so far: Razor looks great for ASP.NET, but it was completely baffling how to get my JavaScript to come through. Sorted it out in the end, but then decided that using an MVC project was overkill to play with JS, so nuked it all. I’m sure there is an easy way to do it all, but I’ll have to look at a few MSDN tutorials to get the gist of the recommended approach here.

I definitely went in the deep end there.

And now I need to sleep.
One more day, and then a nice long weekend… to play more with this?

PS: Sorry, no header picture today. I just don’t have the energy to think of something.