So, we are using MIME::Lite to generate emails from our scripts/modules. The problem I came across in
testing was:
How do I test the $msg->send() to see that the email I have generated is what was expected?
In Ruby on Rails, the test framework automatically allows for email to be appended to an array, which
you can then test against.
I want something like this in PERL!
Well, it turns out that I can.
After the module using MIME::Lite has been loaded, and my object created, do the following before running a method calling $msg->send(); ($model is an object created from xx.pm containing the method you want to call $msg->send() from)
my $sub = sub {
my $msg = shift;
push @{$model->{emails}}, $msg->as_string;
return;
};
MIME::Lite->send('sub', $sub);
Now, if you run the method containing $msg->send(), you can then access an array, where each element contain the MIME::Lite output of the email generated (and no emails would have been sent, thus not spamming people at random:).
If you model method only creates a MIME::Lite object, and you are testing what a script might do with it, then an alternative $sub is
my $sub = sub {
my $msg = shift;
return $msg->as_string();
};
MIME::Lite->send('sub', $sub);
Then yous can happily do:
my $email = $msg->send();
and test $email to your hearts content, including sending it through MIME::Parser to obtain the headers, attachments, body etc.
I hope this may be of help for someone.
Andy
Monday, 25 February 2008
Friday, 8 February 2008
Contacting Authors and creating SQL
So yesterday I was having a problem. I wanted to send an email using MIME::Lite from a PERL
script,
but such that auto responders wouldn't send back.
I trawled through the RFC to see what things they suggested. It is a lot of boring documentation, but I found 3 useful things
1) Subjects should have autoreply in them
2) The auto-submitted header should be set
3) Auto responders should ignore anything that comes with a Precendence: list header
The first two I could programmatically cope with (if (x) { ignore in some way }).
The third needed me to change my outgoing posts. So, off to the documentation for MIME::Lite to find how to set the Precendence header.
Unfortunately, I couldn't find this, and just using MIME::Lite->new(Precendence => 'list'); wasn't working. Give up I did not! I mailed the very nice man who wrote MIME::Lite as to whether a) it did it, or b) if it would be put in a future release.
Bingo, he mailed me back. Simply do MIME::Lite->new('Precendence:' => 'list'); Hey presto,it worked - brilliant. (My biggest thanks to eryq for his quick informative response!)
In fact, he mentioned you can set any of the non-standard MIME::Lite setting headers in this way. A very useful thing to know indeed.
So, one quick change and a test later, and we have this in svn!
On to the next issue - generating advanced queries in SQL programmatically.
So, I have been set for this sprint to write an advanced query page. I have taken this slowly and steadily, as I want to get it right! The first task is just to get it self generating some multi-table queries, that also use join tables, performing a simple one statement select.
Easy or hard? That is the question.
The way I have looked at it is that I, the wonderful software guru, know the database tables. After all, why wouldn't I? In a search model, I create 4 new methods.
1) advanced_search - this is to generate the SQL, perform it and return the results
2) search_for - this returns a hash of all the required fields and corresponding tables based on what the user has requested.
3) search_conditions - this returns a hash of what fields are being selected on, and corresponding tables for those. Here, I can write some additional extra WHERE statements to group together more than one field lookup if needed (which for loader was)
4) table_links - this returns a hash of all tables that have foreign keys, and the table for that foreign key (all keys have the format "id_").
With this and some looped code, I have managed to generate some rather complex single SELECT statements in the format of
SELECT DISTINCT tablea.id_tablea, tablea.name, tableb.id_tableb, tablec.comments
FROM tablea, tableb, tablec, tabled
WHERE tabela.position IN (x,y,z)
AND tablea.id_tabled = tabled.id_tabled
AND tabled.id_tablec = tablec.id_tablec
AND tablec.comment LIKE '%good%'
AND tablec.id_tableb = tableb.id_tableb
I have chosen to put DISTINCT in as a default, as sometimes, when x,y,z are all chosen, it can return multiple rows that are the same.
This seems to be going quite fast, however my ever faithful friend Test::Perl::Critic says that method advanced_search has a high complexity score, so it definitely needs a bit of refactoring.
However, it seems to be going fairly well. So, refactoring can wait until next week.
By for now
script,
but such that auto responders wouldn't send back.
I trawled through the RFC to see what things they suggested. It is a lot of boring documentation, but I found 3 useful things
1) Subjects should have autoreply in them
2) The auto-submitted header should be set
3) Auto responders should ignore anything that comes with a Precendence: list header
The first two I could programmatically cope with (if (x) { ignore in some way }).
The third needed me to change my outgoing posts. So, off to the documentation for MIME::Lite to find how to set the Precendence header.
Unfortunately, I couldn't find this, and just using MIME::Lite->new(Precendence => 'list'); wasn't working. Give up I did not! I mailed the very nice man who wrote MIME::Lite as to whether a) it did it, or b) if it would be put in a future release.
Bingo, he mailed me back. Simply do MIME::Lite->new('Precendence:' => 'list'); Hey presto,it worked - brilliant. (My biggest thanks to eryq for his quick informative response!)
In fact, he mentioned you can set any of the non-standard MIME::Lite setting headers in this way. A very useful thing to know indeed.
So, one quick change and a test later, and we have this in svn!
On to the next issue - generating advanced queries in SQL programmatically.
So, I have been set for this sprint to write an advanced query page. I have taken this slowly and steadily, as I want to get it right! The first task is just to get it self generating some multi-table queries, that also use join tables, performing a simple one statement select.
Easy or hard? That is the question.
The way I have looked at it is that I, the wonderful software guru, know the database tables. After all, why wouldn't I? In a search model, I create 4 new methods.
1) advanced_search - this is to generate the SQL, perform it and return the results
2) search_for - this returns a hash of all the required fields and corresponding tables based on what the user has requested.
3) search_conditions - this returns a hash of what fields are being selected on, and corresponding tables for those. Here, I can write some additional extra WHERE statements to group together more than one field lookup if needed (which for loader was)
4) table_links - this returns a hash of all tables that have foreign keys, and the table for that foreign key (all keys have the format "id_
With this and some looped code, I have managed to generate some rather complex single SELECT statements in the format of
SELECT DISTINCT tablea.id_tablea, tablea.name, tableb.id_tableb, tablec.comments
FROM tablea, tableb, tablec, tabled
WHERE tabela.position IN (x,y,z)
AND tablea.id_tabled = tabled.id_tabled
AND tabled.id_tablec = tablec.id_tablec
AND tablec.comment LIKE '%good%'
AND tablec.id_tableb = tableb.id_tableb
I have chosen to put DISTINCT in as a default, as sometimes, when x,y,z are all chosen, it can return multiple rows that are the same.
This seems to be going quite fast, however my ever faithful friend Test::Perl::Critic says that method advanced_search has a high complexity score, so it definitely needs a bit of refactoring.
However, it seems to be going fairly well. So, refactoring can wait until next week.
By for now
Tuesday, 5 February 2008
It's been a (rather busy) while
So, a little while since I last posted. However, much has been done.
We had another release last week, some of which has needed some tweaking since for the next release. GD doesn't help make pages go quickly! However, a bit of ajaxing later and we have a page which loads quickly, but allows the graphs to be opened on the page afterwards.
Nice!
Also, I have been experimenting with creating a script ot parse a MIME format email, to then store the body as an annotation and get the id from the subject.
Fairly easy. I am using MIME::Lite and MIME::Parser to do the hard work, and the api for the application to contact the main models, which in turn handle saving the annotation. You got to love well constructed and documented APIs.
I have written extensive tests over the code for the main part. I wrote most within a module, so that this could be tested easily and reused, with a small script to handle receiving the email, and if anything croaks, doing something with the croak. (An email back, unless it is an Out of Office reply - why do people need to use them? I think we should all start immediately spamming anyone who uses them so an end such that perhaps they won't in future*).
Anyway, I've done a fair bit over the past couple of weeks towards new features, or bettering existing ones. The learning curve keeps going up. I am now also looking at trying to dynamically create SQL in order to be able to do advanced searching. The problem, however, is how to store the foreign key pattern from across the database. I need to look into if the models can provide the information they possess, or if I need to store it within a hash (as current).
Must dash for now. More soon.
*This is not a serious suggestion. I do not condone spamming in any way. A polite request should be used instead.
We had another release last week, some of which has needed some tweaking since for the next release. GD doesn't help make pages go quickly! However, a bit of ajaxing later and we have a page which loads quickly, but allows the graphs to be opened on the page afterwards.
Nice!
Also, I have been experimenting with creating a script ot parse a MIME format email, to then store the body as an annotation and get the id from the subject.
Fairly easy. I am using MIME::Lite and MIME::Parser to do the hard work, and the api for the application to contact the main models, which in turn handle saving the annotation. You got to love well constructed and documented APIs.
I have written extensive tests over the code for the main part. I wrote most within a module, so that this could be tested easily and reused, with a small script to handle receiving the email, and if anything croaks, doing something with the croak. (An email back, unless it is an Out of Office reply - why do people need to use them? I think we should all start immediately spamming anyone who uses them so an end such that perhaps they won't in future*).
Anyway, I've done a fair bit over the past couple of weeks towards new features, or bettering existing ones. The learning curve keeps going up. I am now also looking at trying to dynamically create SQL in order to be able to do advanced searching. The problem, however, is how to store the foreign key pattern from across the database. I need to look into if the models can provide the information they possess, or if I need to store it within a hash (as current).
Must dash for now. More soon.
*This is not a serious suggestion. I do not condone spamming in any way. A polite request should be used instead.
Wednesday, 23 January 2008
Adding a graph
So, Hands up all those that have 'use(d) GD;'?
Now that includes me!
So, I needed to show a table of numbers in a bar chart format. Quite simply, I had an arrayref of hash refs, each hash ref using keys date and percentage.
First off, the GD plot needs it as an arrayref containing 2 arraysrefs, the first of the x-axis (in this case the date) and the second of the percentages. ($cmap is an object of in-house stuff to work out appropriate colours to use).
Easy
my $plot = [[],[]];
foreach my $href (@{$arrayref}) {
push $plot->[0], $href->{date};
push $plot->[1], $href->{percentage};
}
my $graph = GD::Graph::bars->new(1000,400);
$graph->set(
'dclrs' => [
map {
GD::Graph::colour::add_colour(q(#).$cmap->hex_by_name($_));
} (qw(red purple orange blue green yellow magenta cyan))],
'fgclr' => 'black',
'boxclr' => 'white',
'accentclr' => 'black',
'shadowclr' => 'black',
'y_long_ticks' => 1,
'x_label' => 'Date',
'y_label' => 'Percentage Activity'
);
return $graph->plot($plot)->png();
See, I said it was easy. :)
So, of course now I have said that, everyone wants their data in a graph format. Luckily, GD provides lots of graph formats. I just wish that we could submit to them as array of hrefs, rather than array of arrays, which I personlly think is
1) messier
2) relies on the end user to know the order the arrays should be in
3) Sends no additional information for legends (should this be needed), whereas the key can be the legend.
Oh well. I shouldn't complain really. I didn't have to write it. Don't you just love the CPAN!
(GD and it's suite of components by Lincoln D. Stein and searchable on CPAN. There are lots of additional stuff to link into Template::Toolkit, etc written by others. Have a look!)
FooMongers today. Don't yet know what we'll discuss.
Andy
Now that includes me!
So, I needed to show a table of numbers in a bar chart format. Quite simply, I had an arrayref of hash refs, each hash ref using keys date and percentage.
First off, the GD plot needs it as an arrayref containing 2 arraysrefs, the first of the x-axis (in this case the date) and the second of the percentages. ($cmap is an object of in-house stuff to work out appropriate colours to use).
Easy
my $plot = [[],[]];
foreach my $href (@{$arrayref}) {
push $plot->[0], $href->{date};
push $plot->[1], $href->{percentage};
}
my $graph = GD::Graph::bars->new(1000,400);
$graph->set(
'dclrs' => [
map {
GD::Graph::colour::add_colour(q(#).$cmap->hex_by_name($_));
} (qw(red purple orange blue green yellow magenta cyan))],
'fgclr' => 'black',
'boxclr' => 'white',
'accentclr' => 'black',
'shadowclr' => 'black',
'y_long_ticks' => 1,
'x_label' => 'Date',
'y_label' => 'Percentage Activity'
);
return $graph->plot($plot)->png();
See, I said it was easy. :)
So, of course now I have said that, everyone wants their data in a graph format. Luckily, GD provides lots of graph formats. I just wish that we could submit to them as array of hrefs, rather than array of arrays, which I personlly think is
1) messier
2) relies on the end user to know the order the arrays should be in
3) Sends no additional information for legends (should this be needed), whereas the key can be the legend.
Oh well. I shouldn't complain really. I didn't have to write it. Don't you just love the CPAN!
(GD and it's suite of components by Lincoln D. Stein and searchable on CPAN. There are lots of additional stuff to link into Template::Toolkit, etc written by others. Have a look!)
FooMongers today. Don't yet know what we'll discuss.
Andy
Friday, 18 January 2008
Release of Acme::Hardware::Light::Bulb
OK, so it itsn't the biggest thing ever, but now Acme::Hardware::Light::Bulb is in sourceforge svn
http://sourceforge.net/projects/acmehardwarelig/
If you SVN Browse, you can see a file trunk.tar, which should download you the tarball of the trunk. Else you could just get an svn co of the trunk.
I take no responsibility for anything that happens when you use these modules!
The coverage of tests though is 97,2%, and all the programming is to the standards supported
by Test::Perl::Critic.
You will need Carp and Class::Std.
The modules are
Acme/Hardware/Light/Bulb.pm - which generates a Light Bulb object
Acme/Hardware/Light/Bulb/Change.pm - which generates a Change Light Bulb object with Bulb.pm as the base class.
Anyway. Yesterday was moving methods from the View to the Model, as that is where they should reside, and then today was tests. Coverage of two modules from 48 to 97% Hurrah!
Only 3 modules below 50% now, and total is 86% over the main project. We are getting there.
next week - creating a bar chart on a web page!
Andy
http://sourceforge.net/projects/acmehardwarelig/
If you SVN Browse, you can see a file trunk.tar, which should download you the tarball of the trunk. Else you could just get an svn co of the trunk.
I take no responsibility for anything that happens when you use these modules!
The coverage of tests though is 97,2%, and all the programming is to the standards supported
by Test::Perl::Critic.
You will need Carp and Class::Std.
The modules are
Acme/Hardware/Light/Bulb.pm - which generates a Light Bulb object
Acme/Hardware/Light/Bulb/Change.pm - which generates a Change Light Bulb object with Bulb.pm as the base class.
Anyway. Yesterday was moving methods from the View to the Model, as that is where they should reside, and then today was tests. Coverage of two modules from 48 to 97% Hurrah!
Only 3 modules below 50% now, and total is 86% over the main project. We are getting there.
next week - creating a bar chart on a web page!
Andy
Thursday, 17 January 2008
Sprint ended
So, after a frantic week of finishing off the bits, and getting an extra feature to need to add in this release,
we deployed yesterday to massive applause.
Well, OK, so that was a bit of a lie. We deployed. I then needed to fix a bug which hadn't shown up in my
test version of the database, which then crashed the server, which then needed another fix.
However, it is done,and we added many new features. Some are ready for a need to change what group people belong to and the permissions of that group. Some are already there. Some need a new feature change in the next sprint to move from a table to a bar chart.
On other things, I was writing Acme::Hardware::Light::Bulb and Acme::Hardware::Light::Bulb::Change, since at PerlMongers last week, Michael and I couldn't find any modules relating to changing a light bulb.
We started it last week, and then I finished it over the week. Complete with96.7% test coverage, and completely happy with Test::Perl::Critic, and POD/distribution test coverage.
I tried to tarball it and mail it around, but for some reason the tar files would unpack.
I then went to upload to sourceforge via SVN, and ...
... accidently deleted the whole directory --AAHHH!
Exactly what svn/cvs is supposed to help prevent, I did whilst trying to put it in there. It was due to my own stupidity though. I did a mv instead of a cp within my dev area, and then deleting that moved directory to try again did me in. Note to self: NEVER mv a whole directory.
Anyway, I have resurrected the modules and all the critic stuff. I just need to rewrite the tests. However, it is all available on sourceforge should you wish to check it out.
Once completed, I will look to adding it to CPAN.
Enough for now. Back to my main job.
Andy
we deployed yesterday to massive applause.
Well, OK, so that was a bit of a lie. We deployed. I then needed to fix a bug which hadn't shown up in my
test version of the database, which then crashed the server, which then needed another fix.
However, it is done,and we added many new features. Some are ready for a need to change what group people belong to and the permissions of that group. Some are already there. Some need a new feature change in the next sprint to move from a table to a bar chart.
On other things, I was writing Acme::Hardware::Light::Bulb and Acme::Hardware::Light::Bulb::Change, since at PerlMongers last week, Michael and I couldn't find any modules relating to changing a light bulb.
We started it last week, and then I finished it over the week. Complete with96.7% test coverage, and completely happy with Test::Perl::Critic, and POD/distribution test coverage.
I tried to tarball it and mail it around, but for some reason the tar files would unpack.
I then went to upload to sourceforge via SVN, and ...
... accidently deleted the whole directory --AAHHH!
Exactly what svn/cvs is supposed to help prevent, I did whilst trying to put it in there. It was due to my own stupidity though. I did a mv instead of a cp within my dev area, and then deleting that moved directory to try again did me in. Note to self: NEVER mv a whole directory.
Anyway, I have resurrected the modules and all the critic stuff. I just need to rewrite the tests. However, it is all available on sourceforge should you wish to check it out.
Once completed, I will look to adding it to CPAN.
Enough for now. Back to my main job.
Andy
Wednesday, 9 January 2008
1 week left of the sprint to 7.0
So, 1 week left of the current sprint. Deployment of 7.0 is scheduled for next weds.
Manipulating to increase the level of the loader was not difficult. Putting it into the profile of the user, just a submit button to "promote", with the next level as a hidden field. Then in the tt2 template, a wrapper to ensure that the button isn't generated if the user him/herself is the viewer (we don't want users promoting themselves), or that they are already at "Gold" standard.
So, on to writing tests to exercise the code. We aren't doing to much test-driven development (SHOCK HORROR!), but rather writing the tests afterwards and using Devel::Cover to ensure it is as full as possible. (When FireWatir is improved, I may start looking at that for testing the web interface, but at the moment, it is just Test::More unit tests)
It is quite amazing how quickly the time can go still when writing tests. It isn't the sexiest of topics, but it is good to know that the code we are writing is able to be checked, in case something down the line changes it.
I am opting for writing new test units for each of the additions I have written, rather than adding them in to previous units. This may mean an additional time overhead when running the tests, but I have found it easier when they fail to see via the name of a unit what the problem is (which is reported at the end of the make test output) as well as via the name of the test itself (assuming you have verbose switched on).
So, over the last 3 working days, I have enabled a loader to be assigned a level, and tested that the level can be found for a user who is in the loading group, and that it is displayed in the right places.
Just need to tidy up a couple of extra options and we should be ready for deployment.
Bye for now
Manipulating to increase the level of the loader was not difficult. Putting it into the profile of the user, just a submit button to "promote", with the next level as a hidden field. Then in the tt2 template, a wrapper to ensure that the button isn't generated if the user him/herself is the viewer (we don't want users promoting themselves), or that they are already at "Gold" standard.
So, on to writing tests to exercise the code. We aren't doing to much test-driven development (SHOCK HORROR!), but rather writing the tests afterwards and using Devel::Cover to ensure it is as full as possible. (When FireWatir is improved, I may start looking at that for testing the web interface, but at the moment, it is just Test::More unit tests)
It is quite amazing how quickly the time can go still when writing tests. It isn't the sexiest of topics, but it is good to know that the code we are writing is able to be checked, in case something down the line changes it.
I am opting for writing new test units for each of the additions I have written, rather than adding them in to previous units. This may mean an additional time overhead when running the tests, but I have found it easier when they fail to see via the name of a unit what the problem is (which is reported at the end of the make test output) as well as via the name of the test itself (assuming you have verbose switched on).
So, over the last 3 working days, I have enabled a loader to be assigned a level, and tested that the level can be found for a user who is in the loading group, and that it is displayed in the right places.
Just need to tidy up a couple of extra options and we should be ready for deployment.
Bye for now
Subscribe to:
Posts (Atom)
