This is not a slander of Perl::Critic, but more a need to ensure that I remember the following.
Things break when there are no tests!
perl critic threw a paddy over
print STDERR q{Hello World!} or carp;
Because STDERR is not preceded by a *
So in my program I preceded to be a good little programmer, and put a * in front of all my STDERRs. (Before people ask, this is in a help statement to be outputted to the user if they didn't provide a run id, not really Hello World!, and was actually a quickie script to fix an external bug initially).
In PBP, it mentions the good practice of surrounding *STDERR with braces, but in my case, Perl::Critic didn't throw for this.
Now, here is my problem (my fault). I didn't have a test which at least compiled this script.
Can you guess what happens next? It goes into production, is launched and fails to compile, since
print *STDERR q{Hello World!} or carp;
isn't allowed. Here it is in a one liner:
>perl -e 'print *STDERR qq{hello world};'
syntax error at -e line 1, near "*STDERR qq{hello world}"
Execution of -e aborted due to compilation errors.
AAAGGHHH! It is a basic schoolboy error, I'm sure, but certainly teaches a lesson after around an hour of digging through logs to try to work out exactly why it worked last week, and not this.
All braced and ready to fly.
Thursday, 26 November 2009
Wednesday, 25 November 2009
Cloning via the command line
I have update MooseX::AttributeCloner today on CPAN - v0.17
It had a few revisions over the last few days. Some due to the CPAN testers firing me back errors in my tests, some due to development requirements.
The last new thing about it though is the method 'attributes_as_command_options'.
This method goes through all of your built non-private attributes which have an init_arg, and builds a string of them for using in a command line.
The default is to render them as follows
--Boolean --attr1 val1 --hash_attr key1=val1 --hash_attr key2=val2
but you can request that the values be quoted, an = sign be placed between arg and val, and that only a single dash is used.
You can also exclude some attributes by supplying an arrayref of 'init_arg's to be excluded.
For more info, see the CPAN page
or once installed, perldoc MooseX::AttributeCloner
Unlike the new_from_cloned_attributes, you cannot supply additional key/value pairs to be added, but then, if you are generating a command line you can always add these yourself.
I hope that this proves of use to some people. It is certainly helping to reduce code and increase the flexibility of the pluggable pipeline system I have been developing.
It had a few revisions over the last few days. Some due to the CPAN testers firing me back errors in my tests, some due to development requirements.
The last new thing about it though is the method 'attributes_as_command_options'.
This method goes through all of your built non-private attributes which have an init_arg, and builds a string of them for using in a command line.
The default is to render them as follows
--Boolean --attr1 val1 --hash_attr key1=val1 --hash_attr key2=val2
but you can request that the values be quoted, an = sign be placed between arg and val, and that only a single dash is used.
You can also exclude some attributes by supplying an arrayref of 'init_arg's to be excluded.
For more info, see the CPAN page
or once installed, perldoc MooseX::AttributeCloner
Unlike the new_from_cloned_attributes, you cannot supply additional key/value pairs to be added, but then, if you are generating a command line you can always add these yourself.
I hope that this proves of use to some people. It is certainly helping to reduce code and increase the flexibility of the pluggable pipeline system I have been developing.
Tuesday, 17 November 2009
Refactoring into Roles
So, the pipeline moves along, and then the bossman says:
'I want to be able to run it outside of the normal pipeline locations - how do I do it?'
After a discussion as to why you want to do this (since the normal person responsible for testing out new Illumina pipelines often does it in the usual place, moving softlinks as necessary), it is that he wanted to run some stuff to dump the output into his home directory.
OK, if that is what you want.
'Oh yes, also, can we make it more generic. A lot of things we have done imply the setup we have here (most of which is generated by the Illumina pipeline) and I would like it to move away from that'.
This is a major refactor then. Separate Business logic from Generic logic, and allow everything to be user defined, should the user want to.
Looking at it, it seems an obvious thing really, but at the time I have to say, I panicked a bit. The pipeline has been written to be pluggable, so new 'components' can be switched in or out quickly, but I hadn't really planned it to be used outside of the current setup. The principle is easy enough to apply elsewhere, and should not take long to set up, but the components were specialised to apply to what they represent.
So, the first thing I did - go and get a drink. I don't drink Tea or Coffee, but popping off for a break seemed like the right thing to do. This break lasted a while, as I thought about strategies to take. Should I go for all new objects, passing them around. How about pushing them from the command line? MooseX::Getopt seemed to give an option to put any attributes onto the command line, but what about subsequent component objects from the pipeline?
In the end, I wondered about using Moose Roles. The Manual seemed to suggest that this could be a good way forward. A 'class' that doesn't need to be instantiated or extended, but instead is consumed to become part of the class that you want. So, everything could have all the same attributes, in exactly the same way.
Now, I have to say, I could see an immediate danger here. If everything can be given to everything, then what makes anything different to be a component, or the pipeline 'flag waver'. So, I needed to be sensible, in that the pipelines need to match the components that are going to be launched from them, but not others.
First job: Sort out the pipelines. This was quite simple, and needed doing anyway, since the flag waver had a component launcher for every possible component, but most components are only used for 1 of 3 pipelines, so separate out the 3 pipeline components into subclasses. This had the added advantage of naming the 3 pipeline flag wavers as well.
OK, so what's the next thing.
Go through all components looking for common attributes and methods, and labelling them as generic, or specific. Once I had done this, I created some roles which I refactored these into. Sometimes leaving them in the class if they were unnecessary to be available in multiple classes, otherwise putting them in a role which described if they were generic/business logic and which described the type of feature they give.
After that, I just needed to apply roles to classes which meant that the flag waver had the same attributes as any component classes it would launch. This enables me to have the attribute value in the flag waver, and pass it to anything launched from it.
This left a headache in that new instantiation would need me to loop through all attributes and pass them through. So, I put together another role to do this (see http://vampiresoftware.blogspot.com/2009/11/moosexattributecloner.html for details on this).
So, now I'm left with the final problem, how to allow the user to run a pipeline with the options they want. A quick solution to this was already being used by another of my team, MooseX::Getopt. This is a great Role to apply to a class, which then enables any attribute to instantly become a command line option. Since the the Roles created above give attributes to a class, then they become command line options. Hurrah, problem solved.
So, after I initially thought that Roles would not really be worth it much, considering how much subclassing I have normally done and have been used to, now I'm convinced that this is a very useful technology.
Result of all this refactoring - over 900 lines of code lost. The value according to sloccount if $35,000 less. The code is now more maintainable, and more useful for users who want to define their own parameters.
Sorry Class::Std, you have just had another nail forcefully bashed into your coffin.
'I want to be able to run it outside of the normal pipeline locations - how do I do it?'
After a discussion as to why you want to do this (since the normal person responsible for testing out new Illumina pipelines often does it in the usual place, moving softlinks as necessary), it is that he wanted to run some stuff to dump the output into his home directory.
OK, if that is what you want.
'Oh yes, also, can we make it more generic. A lot of things we have done imply the setup we have here (most of which is generated by the Illumina pipeline) and I would like it to move away from that'.
This is a major refactor then. Separate Business logic from Generic logic, and allow everything to be user defined, should the user want to.
Looking at it, it seems an obvious thing really, but at the time I have to say, I panicked a bit. The pipeline has been written to be pluggable, so new 'components' can be switched in or out quickly, but I hadn't really planned it to be used outside of the current setup. The principle is easy enough to apply elsewhere, and should not take long to set up, but the components were specialised to apply to what they represent.
So, the first thing I did - go and get a drink. I don't drink Tea or Coffee, but popping off for a break seemed like the right thing to do. This break lasted a while, as I thought about strategies to take. Should I go for all new objects, passing them around. How about pushing them from the command line? MooseX::Getopt seemed to give an option to put any attributes onto the command line, but what about subsequent component objects from the pipeline?
In the end, I wondered about using Moose Roles. The Manual seemed to suggest that this could be a good way forward. A 'class' that doesn't need to be instantiated or extended, but instead is consumed to become part of the class that you want. So, everything could have all the same attributes, in exactly the same way.
Now, I have to say, I could see an immediate danger here. If everything can be given to everything, then what makes anything different to be a component, or the pipeline 'flag waver'. So, I needed to be sensible, in that the pipelines need to match the components that are going to be launched from them, but not others.
First job: Sort out the pipelines. This was quite simple, and needed doing anyway, since the flag waver had a component launcher for every possible component, but most components are only used for 1 of 3 pipelines, so separate out the 3 pipeline components into subclasses. This had the added advantage of naming the 3 pipeline flag wavers as well.
OK, so what's the next thing.
Go through all components looking for common attributes and methods, and labelling them as generic, or specific. Once I had done this, I created some roles which I refactored these into. Sometimes leaving them in the class if they were unnecessary to be available in multiple classes, otherwise putting them in a role which described if they were generic/business logic and which described the type of feature they give.
After that, I just needed to apply roles to classes which meant that the flag waver had the same attributes as any component classes it would launch. This enables me to have the attribute value in the flag waver, and pass it to anything launched from it.
This left a headache in that new instantiation would need me to loop through all attributes and pass them through. So, I put together another role to do this (see http://vampiresoftware.blogspot.com/2009/11/moosexattributecloner.html for details on this).
So, now I'm left with the final problem, how to allow the user to run a pipeline with the options they want. A quick solution to this was already being used by another of my team, MooseX::Getopt. This is a great Role to apply to a class, which then enables any attribute to instantly become a command line option. Since the the Roles created above give attributes to a class, then they become command line options. Hurrah, problem solved.
So, after I initially thought that Roles would not really be worth it much, considering how much subclassing I have normally done and have been used to, now I'm convinced that this is a very useful technology.
Result of all this refactoring - over 900 lines of code lost. The value according to sloccount if $35,000 less. The code is now more maintainable, and more useful for users who want to define their own parameters.
Sorry Class::Std, you have just had another nail forcefully bashed into your coffin.
Sunday, 8 November 2009
MooseX::AttributeCloner
I have just released a MooseX::AttributeCloner to the CPAN. The purpose of this role is to go pass all built attribute values from one class object to a newly instantiated object.
The reason for the creation of this role is for our pluggable pipeline system, where we want to be able to pass any parameters from the command line to any of the objects which are responsible for submitting jobs.
The Role inspects all the attributes the class has via meta, and checks that the attribute is defined. If it is, it passes the value into a hash, which it then uses in the object instantiation of the new object.
my $object_with_attributes = $self->new_with_cloned_attributes('Package::Name');
This will take $self's built attributes, and uses them in a hash_ref to create a new object instance of 'Package::Name'.
It does check the init_arg of the attribute in $self, and uses that as the key for the value. If you have a Ref in the attribute, then it will be the same Ref in the new object. (So, in this case, it isn't strictly cloned, please don't be pedantic).
If you wish to pass through additional information, or use a different value for an attribute that you already have a value for in the current object, then a hash_ref can also be provided with key/value pairs so that these can be used/added as as well
my $object_with_attributes = $self->new_with_cloned_attributes('Package::Name',{
attr1 => q{val1},...
});
There are also two additional methods provided by this role, 'attributes_as_json' and attributes_as_escaped_json'. With these methods, you can get a JSON string with the values of all built attributes, which you could then use elsewhere, such as just building a data hash or pushing into a commandline. More info can be found in the Documentation, but it is worth noting, objects are not stringified, throughout the whole data structure. In the case where this would be in an array, a null will be found instead.
I hope that this will be found to be useful. Please give me any feedback for this that you wish.
The reason for the creation of this role is for our pluggable pipeline system, where we want to be able to pass any parameters from the command line to any of the objects which are responsible for submitting jobs.
The Role inspects all the attributes the class has via meta, and checks that the attribute is defined. If it is, it passes the value into a hash, which it then uses in the object instantiation of the new object.
my $object_with_attributes = $self->new_with_cloned_attributes('Package::Name');
This will take $self's built attributes, and uses them in a hash_ref to create a new object instance of 'Package::Name'.
It does check the init_arg of the attribute in $self, and uses that as the key for the value. If you have a Ref in the attribute, then it will be the same Ref in the new object. (So, in this case, it isn't strictly cloned, please don't be pedantic).
If you wish to pass through additional information, or use a different value for an attribute that you already have a value for in the current object, then a hash_ref can also be provided with key/value pairs so that these can be used/added as as well
my $object_with_attributes = $self->new_with_cloned_attributes('Package::Name',{
attr1 => q{val1},...
});
There are also two additional methods provided by this role, 'attributes_as_json' and attributes_as_escaped_json'. With these methods, you can get a JSON string with the values of all built attributes, which you could then use elsewhere, such as just building a data hash or pushing into a commandline. More info can be found in the Documentation, but it is worth noting, objects are not stringified, throughout the whole data structure. In the case where this would be in an array, a null will be found instead.
I hope that this will be found to be useful. Please give me any feedback for this that you wish.
Friday, 6 November 2009
Extending a Role attribute
Not so much a blog post, more a blog question.
When you are 'extend'ing a Moose base class, Attributes can be extended
package Super::Hero;
use Moose;
has q{super_abilitity} => (isa => q{Str}, is => q{rw});
...
package My::Hero;
use Moose;
extends qw{Super::Hero};
has q{+super_abilitity} => (required => 1);
...
However, in this case, I would like Super::Hero to be a Role to be consumed by some class.
package Super::Hero;
use Moose::Role;
has q{super_abilitity} => (isa => q{Str}, is => q{rw});
...
However, the following doesn't work:
package My::Hero;
use Moose;
with qw{Super::Hero};
has q{+super_abilitity} => (required => 1);
...
The required 'extension' is just ignored. I have to actually declare the whole attribute again.
package My::Hero;
use Moose;
with qw{Super::Hero};
has q{super_abilitity} => (isa => q{Str}, is => q{ro}, required => 1);
...
I can't find anything in CPAN Documentation to confirm or deny that this the deliberate design. Does anyone have any suggestions as to any solutions to this?
Any help appreciated.
When you are 'extend'ing a Moose base class, Attributes can be extended
package Super::Hero;
use Moose;
has q{super_abilitity} => (isa => q{Str}, is => q{rw});
...
package My::Hero;
use Moose;
extends qw{Super::Hero};
has q{+super_abilitity} => (required => 1);
...
However, in this case, I would like Super::Hero to be a Role to be consumed by some class.
package Super::Hero;
use Moose::Role;
has q{super_abilitity} => (isa => q{Str}, is => q{rw});
...
However, the following doesn't work:
package My::Hero;
use Moose;
with qw{Super::Hero};
has q{+super_abilitity} => (required => 1);
...
The required 'extension' is just ignored. I have to actually declare the whole attribute again.
package My::Hero;
use Moose;
with qw{Super::Hero};
has q{super_abilitity} => (isa => q{Str}, is => q{ro}, required => 1);
...
I can't find anything in CPAN Documentation to confirm or deny that this the deliberate design. Does anyone have any suggestions as to any solutions to this?
Any help appreciated.
Subscribe to:
Posts (Atom)
