Monday, 25 February 2008

Mocking out to retrieve an email

So, we are using MIME::Lite to generate emails from our scripts/modules. The problem I came across in
testing was:

How do I test the $msg->send() to see that the email I have generated is what was expected?

In Ruby on Rails, the test framework automatically allows for email to be appended to an array, which
you can then test against.

I want something like this in PERL!

Well, it turns out that I can.

After the module using MIME::Lite has been loaded, and my object created, do the following before running a method calling $msg->send(); ($model is an object created from containing the method you want to call $msg->send() from)

my $sub = sub {
my $msg = shift;
push @{$model->{emails}}, $msg->as_string;
MIME::Lite->send('sub', $sub);

Now, if you run the method containing $msg->send(), you can then access an array, where each element contain the MIME::Lite output of the email generated (and no emails would have been sent, thus not spamming people at random:).

If you model method only creates a MIME::Lite object, and you are testing what a script might do with it, then an alternative $sub is

my $sub = sub {
my $msg = shift;
return $msg->as_string();
MIME::Lite->send('sub', $sub);

Then yous can happily do:

my $email = $msg->send();

and test $email to your hearts content, including sending it through MIME::Parser to obtain the headers, attachments, body etc.

I hope this may be of help for someone.


Friday, 8 February 2008

Contacting Authors and creating SQL

So yesterday I was having a problem. I wanted to send an email using MIME::Lite from a PERL
but such that auto responders wouldn't send back.

I trawled through the RFC to see what things they suggested. It is a lot of boring documentation, but I found 3 useful things

1) Subjects should have autoreply in them
2) The auto-submitted header should be set
3) Auto responders should ignore anything that comes with a Precendence: list header

The first two I could programmatically cope with (if (x) { ignore in some way }).

The third needed me to change my outgoing posts. So, off to the documentation for MIME::Lite to find how to set the Precendence header.

Unfortunately, I couldn't find this, and just using MIME::Lite->new(Precendence => 'list'); wasn't working. Give up I did not! I mailed the very nice man who wrote MIME::Lite as to whether a) it did it, or b) if it would be put in a future release.

Bingo, he mailed me back. Simply do MIME::Lite->new('Precendence:' => 'list'); Hey presto,it worked - brilliant. (My biggest thanks to eryq for his quick informative response!)

In fact, he mentioned you can set any of the non-standard MIME::Lite setting headers in this way. A very useful thing to know indeed.

So, one quick change and a test later, and we have this in svn!

On to the next issue - generating advanced queries in SQL programmatically.

So, I have been set for this sprint to write an advanced query page. I have taken this slowly and steadily, as I want to get it right! The first task is just to get it self generating some multi-table queries, that also use join tables, performing a simple one statement select.

Easy or hard? That is the question.

The way I have looked at it is that I, the wonderful software guru, know the database tables. After all, why wouldn't I? In a search model, I create 4 new methods.

1) advanced_search - this is to generate the SQL, perform it and return the results
2) search_for - this returns a hash of all the required fields and corresponding tables based on what the user has requested.
3) search_conditions - this returns a hash of what fields are being selected on, and corresponding tables for those. Here, I can write some additional extra WHERE statements to group together more than one field lookup if needed (which for loader was)
4) table_links - this returns a hash of all tables that have foreign keys, and the table for that foreign key (all keys have the format "id_").

With this and some looped code, I have managed to generate some rather complex single SELECT statements in the format of

SELECT DISTINCT tablea.id_tablea,, tableb.id_tableb, tablec.comments
FROM tablea, tableb, tablec, tabled
WHERE tabela.position IN (x,y,z)
AND tablea.id_tabled = tabled.id_tabled
AND tabled.id_tablec = tablec.id_tablec
AND tablec.comment LIKE '%good%'
AND tablec.id_tableb = tableb.id_tableb

I have chosen to put DISTINCT in as a default, as sometimes, when x,y,z are all chosen, it can return multiple rows that are the same.

This seems to be going quite fast, however my ever faithful friend Test::Perl::Critic says that method advanced_search has a high complexity score, so it definitely needs a bit of refactoring.

However, it seems to be going fairly well. So, refactoring can wait until next week.

By for now

Tuesday, 5 February 2008

It's been a (rather busy) while

So, a little while since I last posted. However, much has been done.

We had another release last week, some of which has needed some tweaking since for the next release. GD doesn't help make pages go quickly! However, a bit of ajaxing later and we have a page which loads quickly, but allows the graphs to be opened on the page afterwards.


Also, I have been experimenting with creating a script ot parse a MIME format email, to then store the body as an annotation and get the id from the subject.

Fairly easy. I am using MIME::Lite and MIME::Parser to do the hard work, and the api for the application to contact the main models, which in turn handle saving the annotation. You got to love well constructed and documented APIs.

I have written extensive tests over the code for the main part. I wrote most within a module, so that this could be tested easily and reused, with a small script to handle receiving the email, and if anything croaks, doing something with the croak. (An email back, unless it is an Out of Office reply - why do people need to use them? I think we should all start immediately spamming anyone who uses them so an end such that perhaps they won't in future*).

Anyway, I've done a fair bit over the past couple of weeks towards new features, or bettering existing ones. The learning curve keeps going up. I am now also looking at trying to dynamically create SQL in order to be able to do advanced searching. The problem, however, is how to store the foreign key pattern from across the database. I need to look into if the models can provide the information they possess, or if I need to store it within a hash (as current).

Must dash for now. More soon.

*This is not a serious suggestion. I do not condone spamming in any way. A polite request should be used instead.