Hello to Planet Ironman & CPAN!

The Ironman comptetion has gone off to an impressive start, and the planet has been set up with all sorts of interesting posts coming in. So that’s one hello to Planet Ironman, and fellow bloggers! Secondly, I finally got my PAUSE account approved today, so I can finally start throwing some modules up onto CPAN – it feels nice to finally be able to give some things back to the Perl community.

I have a larger post coming up tomorrow about prototyping in Perl from some work I had to complete recently, but I just wanted to quickly milk this little bit of “fame”

CPAN and MooseX::Bitmask

One of the most impressive features I see about Perl, is CPAN – the Comprehensive Perl Archive Network. For those unaware, this is a massive (and I mean massive) database of Perl modules that are all packaged up ready to be used in your project. Need email validation? Then go grab Email::Valid. Need database interaction? Go grab DBI and a relevant database driver. U need 2 haz lolcat in ur app? Acme::LOLCAT is there for you (as a joke ) Just about anything imaginable is in the CPAN, and it’s tested and package up ready for your use.

It’s hard to describe just how rapid this can make application development. My Genius program that I’m working on now needs Gnome bindings (and specifically panel-applet bindings). I thought this would be a major roadblock for me, but no – there they are, right on CPAN: Gnome2 and Gnome2::PanelApplet – sweet! I needed a robust object management system, something flexible that I could prototype with, so I just grabbed KiokuDB (more on this awesomeness later).

However, while CPAN covers 99% of my needs, there are times when things don’t do quite what I want. We use Moose at MusicBrainz, and one of our attributes represents a bitmask – I had a look on CPAN before tackling this, but non of the modules really took my fancy. So I’ve written my own, and hopefully this will be my first CPAN contribution soon Behold, MooseX::Bitmask (name subject to change, I expect).

This module provides an attribute metaclass and attribute traits for marking an attribute in Moose as a bitmask. It will automatically create some helper methods for you to check flags, and also another helper method to toggle a flag on and off. Here’s some example usage:


package Foo;
use Moose;
use MooseX::Bitmask;

has 'flags' => (
isa => 'Int',
is => 'rw'
flags => [qw( shiney hard tasty )],
traits => [qw( Bitmask )],
);


Now, you can easily toggle attributes:


my $object = Foo->new;
$object->flags_toggle('shiney');
$object->flags_has_flag('shiney') #true


Easy! I have some changes I want to add to this still, mostly support for renaming the provided methods (_has_flag and _toggle), and adding support for inspecting a bitmask in human form. But it’s really coming a long, and I look forward to getting this onto CPAN!

I’m a hopeless blogger, as anyone who’s subscribed to this blog can probably tell. But mst’s post raises some excellent points. Perl is a beatiful language, in my opinion. Yet constantly, it seems to get bad rep. People ask what I do for a job and I tell them I use Perl and they can’t help but smirk at that statement – and I’ve never got why.

So, if you don’t feel like meandering yourway through the linked wall of rant – the general gist of the movement is to get as many people blogging as possible, so I thought hey – might as well help. So a quick intro:

I’m a 2nd year student at Lancaster university student (comp sci, of course) and also work for MusicBrainz.org. Currently, MusicBrainz.org is going through a massive rewrite – new schema, and a new codebase. The latter I’ve been working on since Google Summer of Code 2008 and it makes ample use of Catalyst, Moose, Form::Processor and Template Toolkit, amongst other modules.

I’m also working on a few little pet projects at the moment too – one is a tool to help my revisions (read: slack off from revising). The idea is you feed the tool a set of questions and a set of answers, and it will periodically question you and record your progress. It’s proving to be an extremely fun project to work on – my first experience with App::Cmd and Gnome Perl bindings – but it’s all very smooth. Hopefully some more on this project a bit later! For now though, here’s how it looks:

Applet

Light at the end of the tunnel?

While the next server release is rapidly being prepared by the other developers, I’m happy to announce that at the end of this month, this branch will be straight up for testing! I’ve been a bit quiet recently on the blog, but that’s not due to lack of work! Some recent changes implemented have seen the completion of the add release wizard and the moderation system seeing significant progress towards being fully usable.

However, it’s not quite their yet. Here’s a copy and paste, straight from gtd.org:

***** Hacking
******* Odd jobs
********* TODO Move MB::S::Controller::Entity -> MB::S::Controller
********* TODO Relate artist to URL should keep artist header
********* TODO Move DBDefs.pm to MusicBrainz/Server.pm
******* Tags
********* TODO Move root/tag to root/tags
********* TODO Pagination on display action
********* TODO Displaying 'all' doesn't show links to drill down to just artists
******* Complete user controller
********* TODO User profile -> review links
********* TODO Edit profile not pre-filling
********* TODO Preferences form messed up...
********* TODO User profile -> subscribers
********* TODO User profile stats
********* TODO Login from the login page itself causes redirection to break
******* Internals
********* TODO MusicBrainz::Server::ReleaseEvent->country should work using objects :@work:
******* Editing
********* TODO Relate entities between each other
********* TODO Edit/remove relationships
********* TODO Links to edit attributes form from releases
********* TODO Import release from FreeDB
********* TODO Releate release to URL
********* TODO Add CC license to release
********* TODO Edit release
For this it would probably be sweet if we moved the current system to build up the
$release /as/ we work (instead of passing query params).
********* TODO Edit release events
********* TODO Auto editor section
******* Moderating
********* TODO View subscribed entities
********* TODO Edit navigation module
******* Tagging
********* TODO Support for entering and editing tags
******* Viewing
********* TODO Amazon cover art for releases
See release http://musicbrainz.org/release/b4f4cd9a-a019-4bc8-8ced-2cbc10fac174.html
********* TODO No status on label page
********* TODO Label page does not correctly link to artists?
********* TODO Popups on release status/type

As you can see, it’s not a small amount of work. But I’m happy that these are the features that stand out to me as “missing” the most – once this list is done then it’s off to the testing servers!

Now, I’m off to get the tag controller fully working today!

aCiD2 ♥ MusicBrainz

The last bunch of posts that I’ve made have all been about immediate work, so I wanted to write something a little lighter. I was going through mb-bugs, as I do daily, and it just hit me. I really do ♥ MusicBrainz

I’ve been around the project for 3, maybe even 4 years now – with increasing activity (obviously at the peak at the moment with the employment) – and I’ve seen MusicBrainz in various participation ’states.’ But at the moment, I’ve got to say – this is easily the coolest.

We’ve got 2 paid developers, myself and Rob – but there’s so much more going on than that. #3981 is a brilliant example of this – we get a feature request from murdos (one of our great developers who’s been doing an incredible job recently), and shortly that’s turned around by outsidecontext who says he’s willing to do this work.

Seeing all this interaction between us developers really makes working for MusicBrainz at the moment a very exciting time, because I think when my current work is done (which is really out of the way for the community at the moment) – we’ll have 4 developers all working on the same codebase (or minor branches) for the MusicBrainz.

Also, it’s not just the developers – it’s the community too. The reason I’m here right now, is because of the positive “make it happen!” attitude in the general community – which I think is brilliant. Work a side, everyone is just generally a good person – and I’ve had numerous great conversations with MB fans both on and offline

So, MusicBrainz had it’s birthday a few days ago – so here’s a belated toast to Rob, and everyone who’s made MusicBrainz what it is! Let’s make it even better

Work progressing on moderations, problems with add release…

Hi folks, seems we’re a bit overdue for an update!

Work has been progressing well, after we made some important design decisions – I’ve mainly been focusing on 2 areas at the moment. The first of these areas is the moderation queue.

I have this looking fairly complete – it just needs polishing. At the moment you can view a list of all open moderations, view specific moderations, add edit notes and vote on these edits. I’m using a similar system to the existing code base, whereby each moderation has a template to show how the details of the change should be displayed. So one major area I still need to do is write all these change-templates.

Along with these templates, I still need to devise a system to filter edits (much like our current advanced edit search) – but I’d like to try and get something a little bit cleaner than the code we currently use. At the moment, once the user submits the search form, we hand craft a SQL query – which results in quite a lot of complex code. I’m considering using a chained system to formulate the query; something along the lines of

$c->model('Moderation')->filtered->
                         edit_type('MOD_EDIT_RELEASE')->
                         status('any')->
                         automod(0);

I’m hoping this will make for cleaner code in the controllers, but also the option to make use of specific moderation-filters in other places (maybe a per user dashboard, where users can decide what edits they want to see?)

Moving away from moderation, I’m working on the add release form, and I’m currently finding this one of the hardest challenges of the project. It’s also probably one of the most (if not the most) important forms we have on the site – so getting this working correctly is highly important. However, interactions with this form are very tricky to model.

The largest problem, is that the path is very much non-linear, for non-trivial edits. Let’s consider a 5 track, various artists release, with 2 release events. This would require the following steps:

1. Specify that we need 5 tracks
2. Enter the details of the release
3. For each track, display a search form for the user to select the artist (my branch will not have AJAX until everything can be done without it).
4. Search for the label for the 2 release events, allowing the user to select them.
5. Enter a moderation note, and insert the moderation into the queue.

However, this can be become even more complex – consider the case where the artists/labels do not exist, so the user needs to be redirected to an add artist form. Or maybe the user wants to change an artist they have already confirmed… see how this is starting to become a little bit more complicated, with a very non-predictable path?

While each of these stages can be done on their own, the problem I am having is to work out a solution to keep track of the user progress, and make all of these stages transparently tie together.

If anyone has any experience working with muilt-page forms, I’m certainly curious to hear how you’re doing it. I’ve had feedback from some people pointing me towards a staleless RESTful solution, and others saying I should save the results in the session.

Hello from university!

Hello Brainz

Another week down, and this one has been a bit more tricky than the first! Not in terms of workload, simply in terms of finding time to do work! I moved back to uni about a week ago, and with everyone moving in and meeting all the new guys in the flat, it sure has been tought to set aside 2-3 hours at a time… But nontheless, I have a status update for you!

I decided to spend the last week working on pushing out as many new moderation forms as possible. To shoot through the list: move track, merge labels, label editing, label creation, relate to url, remove release, subscribe/unsubscribe to labels & artists, add cc license, edit release attributes… and I’m sure I missed a few.

But more importantly, I made a few more refactorings to make the process of working with these forms simpler and clearer… I looked at the Form::Processor plugin, and while I don’t like all of it (a bit too much magic, even I agree there Lukas!) – I do like the helper methods for loading forms. This change also made me split the forms up and make use of some more inheritance making the actual code that little bit cleaner.

I made my first multiple step forms as well; these were pretty tricky, and have taken a fair bit of thought. There are still no forms that use many steps (add release, import, etc) – but this has been the start. MB-Catalyst now has a nice elegent way to do those “search for artist/label” forms (like when you move a release to another artist) – so that’s one more little generic component down.

So, that’s last week – what about this week? I started the day working on convert to mulitple artists/single artist, and have them nearly down, but I think I’m going down the wrong route. Lukas reminded me that there is still a lot of ugly code in the background. Now could be a sensible time to start putting some beauty cream on those warts. My work is certainly on schedule, almost all ‘common’ edits can now be done.

To leave you all with a question, where do you think I should head to go next? The moderation system is non-existant (and obviously critical) and some of the backend work is really crucial too… I also have some work for you lot if you’re interested, to hopefully encourage some light community participation (that’s what this branch is all about!)… more on that tomorrow

Last week in review

Ok, that’s a week of employment official over… and I’m certainly not regretting it As an added bonus of more mb_server hackage, you guys will also be seeing an increase in blog posts – with at least one a week of this nature (i’m hoping to do Sundays, but was too busy yesterday).

Starting back at Monday not too much interesting stuff happened – I just spent time fixing some minor issues in the code and getting it up and running on my new dev system. This had a nice bonus of allowing me to refresh myself with the code, and make a mental roadmap, if you will, of what I need to be doing.

Tuesday saw some changes to make the codebase more consistant in some areas, which makes certain things in templates much easier (like [% track.artist.name %]).

Wednesday saw the first new feature additions, and added: verification of user registration via emails, conflicting artists are detected by the add/edit artist form, and the data quality changing form was created; this is modular, so I should be able to support other entity types using it easily; and finally I quickly added some code for artist subscriptions (no links to unsubscribe, but the code is still there!)

On Thursday I reworked the authentication system and thus removed one more wrapper module. Now Catalyst’s authentication system interacts directly with our MusicBrainz user model – again, cleaner code. I added the remove track form, along with merge artists (which was more complex than I expected – as I had to write code for searching).

After working with the form system (which I’m generally really happy with), I decided it needed a bit of refactoring. I re-wrote the system for compound fields first. Compound fields are parts of forms that require multiple user inputs, but should be treated as one data value. Examples include dates, and I also wrote tracks as compound inputs (track number, track title and duration). This allowed me to create the Add Non Album track form quickly, and then I made a final change to the form presentation layer (ooo, look at how businessy I sound) – just something a little more modular.

So, I’m happy with the weeks progress, and have no large concerns about moving forward. The immediate next steps will be to continue working through the huge amount of forms to implement – but I’m aiming to get 2-3 a day done.

In lighter, more general news – check out the up-and-coming flamewar in mb-devel I joke of course, but choosing a new VCS (if that’s what we do) is a tricky topic – so if you are a developer, or are interesting in becoming one, we could really do with your feedback!

Oliver Charles, professional computer programming consultant, signing off!

Updates!

Ok, time for an update I think!

Since my last entry I have removed facades from my codebase (hurrah!) – this means that internal we now through one object around, which reduces complexity quite a bit (no more checking if something was exported to the template or not). Because I’ve been enjoying working with objects like this so much, I’m slowly changing a few subroutines that returned “derived data” (eg, a track returning artist name) to return that data through the correct object. Let’s return to the track, for a better example.

In the old code, if you needed to find the artist for a track, you’d do $track->artist_name. This stinks though, because there’s more than just the artist name, so now we do $track->artist->name – which is better, because it’s what we actually meant (and thought about in our heads). Furthermore, this makes it dead easy to get other artist information:

$track->artist->LoadFromID; # Load everything we can from the database about artists.
my $sort_name = $track->artist->sort_name;

I’ve considered making that LoadFromID implicit (that is, if something is undef, then the artist will load itself and try and again) – but I think I like this explicit form more. Thoughts?

In other news… I’ve introduced some slight changes to the templates, mainly exporting a few “prettifying” subroutines that map integers to names (ie, the release type and status names) – but still have some thoughts about how to make this a bit more organised.

Now, as for the future. I’m going to make a few more changes to make our system act more like a relational system should (ala the above changes), and then I can finally resume work on the editing interfaces! This work should be pretty fast – I’ve already worked out the details with the edit artist page, so I just need to extend that over different edits.

And… there’s some quite exciting news that you’ll hear about soon… but I’m not quite ready to mention it yet Stay tuned!

Most important bits of refactoring complete!

Finally, after what seems like an eternity, I have completed renaming a load of functions to use a nicer more Perl-ish form. It’s taken 3 attempts, but I’ve finally got there and hopefully I haven’t broken anything. I’ll just take a moment to explain how I’ve done the refactoring, so hopefully some veteran coders can tell me how I should have done it

I worked through modules to decide which functions to rename – let’s start in MusicBrainz::Server::Artist for example. I want to rename “sub GetResolution” and “sub SetResolution” into just “sub resolution.” Firstly, I did:

grep -r ’sub [GS]etResolution’ .

This showed only the getter and setter I was interested in. If it showed more complicated functions (i.e. sub GetResolutionInReverse) then I would rename those first. Next, I would do the bulk of the renaming:

find lib -name ‘*.pm’ | xargs perl -pi -e ’s/[GS]etResolution/resolution/g’

And finally, I went back into Artist.pm and rewrote sub resolution to look like:

sub resolution
{
    my ($self, $new_resolution) = @_;

    if (defined $new_resolution) { $self->{resolution} = $new_resolution; }
    return $self->{resolution};
}

I did this over just about every function that my facades are using, so I can now merge this refactoring and make a new branch to delete those facades (never liked ‘em in the first place!). And just to give an idea how much code this touched:

musicbrainz@doughnut ~/TemplateToolkit/lib $ git diff --shortstat master..HEAD
120 files changed, 2549 insertions(+), 2584 deletions(-)

And one final statistic…

musicbrainz@doughnut ~/TemplateToolkit/lib $ grep -r 'sub [GS]et' . | wc -l
255

255 functions still have Get or Set in front of them – at some point these need to be changed! But this will likely happen after facades are removed – one step at a time.