Day 257 – Progress and Perfection

After a lot more effort than I had anticipated, I decommissioned my 4-Bay QNAP NAS this afternoon. All files finally transferred off (400MHz of 2007 ARM grunt is really not adequate to saturate my network), and the 8TB of space re-formatted ready for sale, I have put up my final eBay listing for my little electronic re-org that has lasted the last couple of weeks.

It feels good to be able to stop thinking about the old, so that I can focus on the new.

As of yet, I have not played enough with my Windows 8 start menu to optimise it for its new consolidated uses. Only today, after a driver install and a re-boot, did I manage to get my 27″ monitor to run on 2560×1440 again. It had been stuck on 1680×768 stretched out to fill the surface.

I had hoped to make more progress on some software-development posts, but I really had to focus on my tools first and foremost.

Having said that, over my chores I did do some thinking about the larger structure of those posts, and I think I have come up with a great way to illustrate the “why”. I think the underlying theme I was looking for all along was to strive for a more perfect language. A future-state in which incorrect programs are impossible. Every abstraction along the way ought to at least try strive towards that goal.

And now I have to turn my mind back to the distraction of the TV in the background.

Day 256 – Digital Dusting

The inevitable and unenviable down-sides to upgrading network storage are the clean-up duties that come with it. Time turns pristine terra-bytes into atrocious terror-bytes.

We all succumb at times to the convenience of wide open spaces that make us just drop some data in a disorganised pile. I’ll tell myself I’ll get back to tidying it later. It’ll just be there till the weekend; a couple of weeks at the most. I’ll delete what I no longer need and I’ll group and re-name everything I intend to keep.

Today I “enjoyed” cleaning up 85GB of music with low quality meta-data tags. It had never really mattered that I had never gotten to this clean-up until now. As it turns out, for music playback, Plex really cares about having some nice high-quality tags to identify all the tracks from. It’s the difference between a crude image with a disclaimer of failure, or a polished album cover off the internet.

I was really not looking forward to this task, and I’d have gone even more mental if not for MusicBrainz and their wonderful “MusicBrainz Picard” tool.

MusicBrainz-the-site I believe has existed for some time, and it is basically an open collaborative Wikipedia for music album meta-data. To say it is comprehensive is an understatement. For many albums it has every single variant released in every single geographic area with all the minute variations in meta-data.

Wading through this could be a pain.

But, that’s where MusicBrainz-the-tool comes in; I point it at my music folder, drag all the tracks across the be identified… and it starts by grabbing the details that are already embedded in all the files.

If that data is already sufficiently detailed, tracks and albums will immediately move into the “identified” column. If not, you can click the magic “Lookup” button, which will match the available details against its extensive database. It’ll match tracks by album, track name, track number on the disc, duration… whatever it can use to uniquely identify the album a group of tracks were on together.

When this succeeds (as it does most of the time), again, the albums move straight in the “identified” column, only padded out with much more comprehensive tags including contributing artists, year of release, publisher, and on and on.

In the rare cases where this still does not pinpoint the album (for example, multi-disc albums that were split into separate directories seem to usually fail), you can hit the button that takes you into the search page of MusicBrainz-the-site. Every entry in the search result has a banner that you can click, which will add the album into the “identified” column where you can then drag the album that you think matches. Again it’ll do a best-effort automatic match of the tracks you drag across, but ultimately you can drag-and-drop the identification results till you are happy with the matches that have been made.

Then all that’s left is hitting “Save” and the tracks get re-tagged in-place with the new information.

I like the way the workflow has been laid out. I like that the automation is merely assisting me and not telling me what the answer is. Too many poor solutions suffer from over-automation leaving no room for human intervention if the automated outcome isn’t quite right. This is not one of those solutions.

And I think it saved me a lot of time today.

Now I get to sit back and relax with my music library.
Before I have to go and clean up a decade of photos I’ve put in all the wrong places.

Private Data and the Cloud

Keeping all my stuff in the interwebz never felt entirely comfortable to me, but after a recent crash that took my personal mail install with it I had an epiphany.

I have GMail. I have Steam. I have a NAS for backups. Why am I keeping so much critical data on a single machine that may very well crash (as it did)?

Some of the changes are no-brainers in my opinion. I don’t mind trusting Google with my email; they have been doing this long enough, and most of my mail was already flowing through my GMail account anyway. Adding the POP3 pickup from my personal account was a small formality, and now I can use the powerful search features across all my emails.

There are however other sorts of data that I’m having more trouble figuring out.

I like the idea of access to data everywhere. I’d love to have a single to-do list that I can access from any machine in the world. But there are also going to be to-do items I genuinely do not want to make publicly available. Even more so, there are probably items I would rather not even give to a cloud-based provider in the first place.

And that’s where it gets tricky. I don’t think there are any applications out there at this point that have a model that scales across the full gamut of privacy boundaries.

I’d like applications that can store data publicly in the cloud, privately in the cloud and privately on my own personal devices, synchronised only through my own personal network at home. But I do not think any application with that ability exists.

For a while I was thinking that DropBox may be a part of the solution, but their recent security breach and admission that they have full access to data stored in a DropBox, when previously they were claiming nobody but the user can decrypt the data, is less than comforting.

Now, one possible solution is to store data into an encrypted volume into a DropBox, but now I am wondering if even that is enough. Storing private data in an encrypted volume on a public share is still less secure than not letting it leave my own network in the first place.

I think there is room in the marketplace for someone to develop infrastructure usable by apps in the cloud that can deal with the full range of these security domains in a seamless fashion. Something that’ll work across my Linux-NAS, my Windows PCs and my Android phone. Oh, and an associated dev model that can allow web-apps to run in some limited fashion on my Windows PC and Android phone to deal with the private-only data without ever touching the web.

Is that really too much to ask for? *sigh*