Bingo Node

Much has happened since I last posted here. At $work, a shift has occurred in programming responsibilities and whilst I stilll have lead on the internal LIMs, much of the last few months has been working with Nodejs to produce an analysis framework.

Node uses the v8 engine to run javascript server side, and it's asynchronous nature, and easy use of sockets, makes it perfect to produce an analysis framework for small packet based analysis.

Take info, wrap in JSON packet, pass through socket to new process, process asynchronously enhances packet and pushes through new socket to next new process...

This has been a steep learning curve for me, as I have been primarily a Perl developer with some JS on the client side. Writing purely asynchronous code is very much a change of mind set.

And it works both ways, I was recently helping a colleague with a Perl script which actually would have benefited from being Asynchronous, and I spent a day trying to do that in Perl. (Mind you, I lost count of the amount of times I tried to do synchronous stuff in Node :) Long live the promise module)

Also, we discovered the node-webkit library. This was just fantastic, as we need to be able to deploy a user interface on multiple platforms, and this allows just that, but with everything in done in Node. A standalone application is produced, compiled for Mac, Linux and Windows (one for each).

Now, as we all know, mastership of a skill takes 10,000 hours, and that ain't only gonna happen at work. So I needed a pet project to play with.

Bring in a rewrite of my bingo caller. I have full love for Perl, and the Catalyst/Moose frameworks, but the bingo-caller was a bit slow and clunky. I needed a server and SQL-Lite running in the background. It really needed a rewrite.

I also was thinking of setting myself up as a freelance Bingo caller, as I now have a couple of regular gigs, but the Catalyst app version just won't cut the mustard any more;

So, in a weekend, with a substantially reduced codebase, I got a bingo caller running in Node with node-webkit. It compiles, is now a double click app (my first ever!), and is much much faster. Even putting in the traditional bingo calls hasn't worried the memory usage. There are clear advantages to writing amd running it all in javascript. So, to all involved in NodeJS, and it's many libraries (especially node-webkit), I salute you all.

The code is hosted on GitHub. Versions 0.1/0.2 are under the MIT License, v1.0 is copyright me. I would ask for you to ask permission if you wish to use the caller, but feel free to peruse the code and take hints from it.

github.com/setitesuk/node_webkit_bingo_caller

On a comparison note, I now have 2 real tools for server side programming now, and whilst Perl was my hammer, I now don't need to look at everything as a nail. NodeJS is a great screwdriver, and I can certainly start to identify screws.

If you are interested in hiring a Bingo Caller

https://www.facebook.com/setitesukbingo
twitter.com/setitesukbingo
setitesukbingo@gmail.com

Stumped by range

Today at $work we can across the following in someone's perl script

  if (/\Q$start/x../\Q$end/x) {print q{match};}

This rather stumped all of us from $boss down. Between us, we have over 25 years worth of perl experience, and no-one had seen this before. What is the range operator doing between 2 regexes?

It turns out, that we can use this to loop over (for example) lines in a file, and when $start is seen, start printing out until $end is seen. Then reset and finish looping.

A google search brought up the following page

http://perlmeme.org/faqs/file_io/middle_of_a_file.html

Interestingly (although it is probably the right way to do it), it only keeps a track of whether you have seen $start once, and $end once (before a 'reset') so if $start is there twice before $end, it only finds the first $end, and stops 'matching'. ie:

 foo       no_match
 $start    match
 foo       match
 foo       match
 $start    match
 foo       match
 $end      match
 foo       no_match
 $end      no_match
 $start    match
 foo       match
 foo       match
 $end      match
 foo       no_match



Whilst we worked this out (quicker with a hint from a comment), I think we universally decided that it wasn't particularly readable. However, I think my first ever production script probably would have been a bit shorter had I used it back then :)

I'll leave the merits of this down to the reader. Personally, I don't know how useful I'd find it, but it might help someone out there. Hello to you if you are that person!

(Hopefully, I'll remember this post next time I see this!)

A very handsontable(.js)

At $work, everyone loves spreadsheets (well, almost everyone) and they like to use them at any and all opportunities they can.

This has meant the ability to upload and download data from the LIMs as xls(x), including downloadable template files.

However, people don't seem to like data entry (well, ok, who does - my first non-supermarket job I spent most of my first afternoon, past shift end, data entering), no matter how you construct the web form to do it.

User: Can I enter my data directly into LIMs, modifying many rows of data at a time, in a style like Excel, because I know Excel?

Developer(AKA me): Yes, you can download the data as xls, change it and batch upload the changed sheet. Expected time delay of less than 5 minutes, (generally less than 30s).

User: But I don't like the idea of a time lag between my upload and seeing the data.

Developer: So what would you like?

User: Excel table entry, but within the web application.

Developer: (silent _scream - AAAHHHHH!) I'll see what I can do, but it will take a bit of time.

User: It is quite urgent that the interface is changed.

Developer: (thinks - funny that, user needs something doing urgently) I can give you a ball park estimate of a possibility it can be done in 2 weeks, with a rough usable prototype system in a further 2 weeks. (Starts searching Google)

User: Thankyou (I do on the whole have nice users)

So, I need a Tabular style of entry, for rows of data editing, that can save, and do much as Excel does.

Well, I could do an html table (yes, this thought did originally cross my mind, I'm not proud), with each box being a field in a form. However, it is ugly and you can't do cut and paste, value dragging...

So, I looked around for alternatives using JavaScript libraries (preferably jQuery) as I really didn't want to roll my own.

Luckily, after a quick search, I found it handsontable.js which is jQuery based, and the examples on the website did very much like I wanted it to.

So, after pointing User to the website, I got a good thumbs up, and started to put together a prototype app with some real data to test it out.

The great thing about handsontable.js is that the API is very compact. You only have to provide a set of data, and a few other params (helpfully named) to get a working table up, and it provides the functions to dumpout the data to be directly sent via AJAX.

So, I was able to get the working prototype out within those first 2 weeks easily. Once I had had theat tested by User, it was quite simple to import it into the LIMs, so within the allotted time I had the feature working.

I had some interesting Bugs to then deal with. Perhaps Bugs is too strong a word. The first was that a button to remove table entry (hide) and then bring it back didn't revert unsaved changes. Luckily, handsontables loadData function saw to helping me there. Really simple. Then column sizes and scrollability/change col width. Both again very simple to implement (although there does seem to be a finite size to a column, which I unfortunately hit)

The website has all the features you need, including the css files, and has some of the best online tutorial/examples I've seen in a very long time.

So, big up thanks to Marcin Warpechowski for providing a brilliant solution. I can thoroughly recommend this as a solution for table style entry in a web interface.