Thursday 25 February 2010

Readonly::Scalar, $VERSION and Module::Build/Install

As I blogged before, I have just rebuilt my development area. Within this, I installed the latest version of Module::Build (0.3603).

Now, two things have happened since this. The first, I mentioned in my post on the latest release of MooseX::AttributeCloner, about the fact that 'passthrough' is being deprecated as an option for dist producing a Makefile.PL.

I won't go into that one now, but the second is potentially more disastrous.

Before I go any further, I would like to make it clear that this is not an attempt to slag off the people who write/maintain Module::Build. The tool is extremely useful, and I got a very helpful response from David Golden about my RT ticket. I would like to say Thankyou for producing the tool. The below is possibly more our abuse of it than their failings with it.

We use Test::Perl::Critic to monitor our coding standards. (OK, I accept that it is a set of guidelines, and is in no way compulsory, but it is a start.) Perlcritic wants $VERSION to be a constant, and to do this within the code, we use (as for all constants)

Readonly::Scalar our $VERSION => do { my ($r) = q$LastChangedRevision: 8362 $ =~ /(\d+)/mxs; $r; };

(Again, yes, taking the version number from a version control system is supposedly 'not good', but it works for us, and others I know)

When I run perl Makefile.PL, I get the following:
(note, I use the passthrough or small Makefile.PL and perl Makefile.PL to run Build.PL)

Creating new 'MYMETA.yml' with configuration results
Error evaling version line 'BEGIN { q# Hide from _packages_inside()
#; package Module::Build::ModuleInfo::_version::p56;
use Module::Build::Version;
no strict;

local $VERSION;
$VERSION=undef;
$vsub = sub {
Readonly::Scalar our $VERSION => do { my ($r) = q$LastChangedRevision: 8362 $ =~ /(\d+)/mxs; $r; };;
$VERSION
};
}' in /this/package/lib/module.pm: syntax error at (eval 92) line 9, near "Readonly::Scalar our "
BEGIN not safe after errors--compilation aborted at (eval 92) line 11, line 36.

failed to build version sub for /this/package/lib/module.pm at /Users/ajb/dev/perl/5.10.1/lib/5.10.1/Module/Build/ModuleInfo.pm line 332, line 36.

WARNING: Possible missing or corrupt 'MANIFEST' file.
Nothing to enter for 'provides' field in metafile.
Creating new 'Build' script for 'my_project' version '8391.'

OK, in this case, it's a warning, but the Build file is created, and I can do what I need to.

However, updating the Build.PL 'requires' to include some other package libs, the problem becomes more serious.

Error evaling version line 'BEGIN { q# Hide from _packages_inside()
#; package Module::Build::ModuleInfo::_version::p5;
use Module::Build::Version;
no strict;

local $VERSION;
$VERSION=undef;
$vsub = sub {
Readonly::Scalar our $VERSION => do { my ($r) = q$LastChangedRevision: 8212 $ =~ /(\d+)/mxs; $r; };;
$VERSION
};
}' in /another/package/lib/module.pm: syntax error at (eval 28) line 9, near "Readonly::Scalar our "
BEGIN not safe after errors--compilation aborted at (eval 28) line 11, line 19.

failed to build version sub for /another/package/lib/module.pm at /Users/ajb/dev/perl/5.10.1/lib/5.10.1/Module/Build/ModuleInfo.pm line 332, line 19.
Couldn't run Build.PL: No such file or directory at /Users/ajb/dev/perl/5.10.1/lib/5.10.1/Module/Build/Compat.pm line 335.

Here, because it can't identify the version of the 'external' module, it has croaked out.

I submitted a bug report, and David Golden (Big thanks to him for responding) suggested that before the line could perhaps be made

use Readonly; Readonly::Scalar our $VERSION => do { my ($r) = q$LastChangedRevision: 8212 $ =~ /(\d+)/mxs; $r; };

since the problem is in the eval block.

This is fine to do for internal modules, but a problem for anything we install centrally from CPAN (maintenance of code, root access, etc).

I tried using Module::Install as an alternative. This carps errors in both cases, but in both cases doesn't cause the Makefile not to be created.

This post therefore comes down to 2 things.

1) Information to anyone who really cares or reads this blog.
2) Are we the only people who use Readonly::Scalar to declare 'our $VERSION'? (in which case, MooseX::AttributeCloner is possibly the only module on CPAN which does this)

Thoughts and comments welcome, although I would appreciate people not calling us Idiots (or stronger) for using Readonly::Scalar to 'constant'ify the variable (or at least not without good reason). We have looked at use version;, but it doesn't seem to be right with using a version control value.

Again, thanks to the authors/maintainers of Module::Build and Module::Install for these wonderful tools.

Friday 19 February 2010

MooseX::AttributeCloner v0.2

Yesterday I released v0.2 of MooseX::AttributeCloner. This is just a bugfix release, thanks to those lovely people over at CPANTs.

The problem was that I missed a file in my MANIFEST, so when I built my distribution package, it left it out. Upshot - tests failed.

This has now been fixed, however, I since discovered a deprecation on Module::Build, which I have fixed.

I had initially set up my Build.PL file to use

  create_makefile_pl => 'passthrough',

which generated a Makefile.PL, which loaded Module::Build if not installed.

However, this feature is deprecated, and may be removed, since newer versions of CPAN.pm/CPANPLUS and 5.10.1 accept the 'configure_requires' option. So, I have converted to using

  create_makefile_pl => 'small',
  configure_requires => { 'Module::Build' => 0.3603 }

in Build.PL. This is the new way to do it. It is mentioned in the POD and README.

The new version can be found on CPAN here

http://tinyurl.com/ylztfvz

Cheers

Andy

Wednesday 17 February 2010

Generated code from DBIx::Class and Test::Perl::Critic

We have hit an interesting thing in our ever changing code base.

We want to start managing the database schema and use the power of DBIx::Class to keep the code and database in sync by auto generation.

However, we also want to keep to coding standards by running Test::Perl::Critic at maximum severity over the code base.

Problem - the auto generated code fails tests, but if we modify above the comment line

#DO NOT MODIFY ANYTHING ABOVE THIS LINE

we lose the ability to run regeneration of the code if we make any database changes.

One solution is to say, don't run Perl::Critic over the resultset files. However, this means we need to separate out any manually written code, which should be here (running Catalyst, etc), or we never Test::Perl::Critic our own code. It also means additional maintenance to our auto test each time we add more files.

Another is never to be able to auto-regenerate and manually keep the database and code in sync.

So, has anyone come across any solutions to this problem? All comments gratefully received.

BTW - we understand that

1) TIMTOADY - some people write good maintainable code which doesn't follow Perl::Critic standards.
2) It would be very difficult for maintainers of code which generates code to constantly keep up with the latest Perl::Critic.

Tuesday 16 February 2010

New release of MooseX::AttributeCloner

I released a new version of MooseX:AttributeCloner yesterday to CPAN.

Here are the changes:

1) BugFix - CPANTs put in a bug report that MooseX::Getopt was not in the dependencies list in the Build.PL module

2) You can now do

  my $NewObject = new::object->new_with_cloned_attributes($CurrentObject);

instead of only

  my $NewObject = $CurrentObject->new_with_cloned_attributes(q{new::object);

However, to do this, both objects need to use the MooseX:AttributeCloner role. This is on my TODO list that $CurrentObject would only need to be a Moose object, and not have to utilise the MooseX:AttributeCloner role.

It's out there now. Any feedback appreciated.

Andy

It's amazing - how many more CPAN dependencies

It's true, if you need something done, someone may well have done it before and released a CPAN module. However, there are some that I think might have not been necessary.

After the initial install of my dev area (see last post), I checkout of svn and git my projects again, and started work. However, over the last week, I found a number of other lacking CPAN modules, which I then needed to download.

LibXSLT:

Download Latest version from ftp://xmlsoft.org/libxslt/

./configure --prefix=$HOME/dev
make
make install

Further CPAN modules:

i Cache::Memcached
i XML::Generator
i File::Type
i Math::Round
i Data::ICal
i Date::ICal
i Statistics::Lite
i MIME::Parser
i Perl6::Slurp
i Sys::Filesystem::MountPoint
i XSLT::Cache
i Net::Stomp
i Net::Stomp::Receipt
i GD::Graph::bars3d
i IO::Prompt
i Parallel::ForkManager
i HTML::Tidy
i SQL::Translator
i http://search.cpan.org/CPAN/authors/id/D/DR/DROLSKY/List-AllUtils-0.02.tar.gz (dodgy md5, so use full url)
i Fey::ORM
i Fey
i Fey::DBIManager
i Fey::Loader

I'm not really sure why we particularly needed List-AllUtils, since List-Utils and List-MoreUtils are already being used, but someone has done so. Also, I am not sure why this (and List-MoreUtils before it) had dodgy md5sums. (I should probably put in an RT ticket).

Anyway, just an update.

Monday 8 February 2010

Rebuilding my development area

I thought it was about time to get around to rebuilding my development area, for a few reasons:

1) Housekeeping - My dev area was getting a lot of junk floating around, and rather than just go through and delete, I thought it better to restart

2) About time I upgraded to perl-5.10.1

3) I need to start up a VM soon, and thought it a good opportunity to set make notes about what was needed

4) cpan/cpanplus wasn't working for me

5) I've never had GD working properly, and I could be about to lose my desktop at work

So plenty of reasons. I also wanted to try to structure how I setup various apps, so that it should (in theory) be easier to upgrade an of them. Getting further on, I think this might not be so worthwhile, but at least it's a try.

Here is 'What I Have Done' so far

INITIAL SETUP:

In $HOME

mkdir dev
cd dev

This gives me a base dev directory to use

PERL:

mkdir perl

Download the version of perl you want to install

mkdir perl/ (i.e. mkdir perl/5.10.1)

Unarchive the download and go into the directory for created from unarchiving and do the following

./Configure -des -Dprefix=$HOME/dev/perl/
make
make test (go away and make a cup of tea)
make install

This will now give you in $HOME/dev/perl/ the bin/, lib/ and man/ directories

Once you have done this, symlink this version to $HOME/dev/perl/current, and add $HOME/dev/perl/current/bin to $PATH

This should make your default perl $HOME/dev/perl/current/bin/perl

Since you have done this, if you now want to download and try another perl, then you can do the same, and just switch the current softlink


LIBGD:

You need to download
freetype-2.3.11.tar.gz
jpegsrc.v8.tar.gz
libpng-1.2.23.tar.gz
zlib-1.2.3.tar.gz
gd-2.0.35.tar.gz

unpack and install

freetype

cd freetype-2.3.11
./configure --prefix=$HOME/dev
make install
cd ..

jpeg-8
(need to look at)
cd jpeg-8
./configure --prefix=$HOME/dev --enable-shared --enable-static
make
make install
cd ..

zlib-1.2.3

cd zlib-1.2.3
./configure --prefix=$HOME/dev
make
make install
cd ..

[edit] libpng-1.2.x

cd libpng-1.2.x
CFLAGS="-I$HOME/dev/include" LDFLAGS="-L$HOME/dev/lib" ./configure --prefix=$HOME/dev
make
make install
cd ..

[edit] gd-2.0.35

cd gd-2.0.35
CFLAGS="-I$HOME/dev/include" LDFLAGS="-L$HOME/dev/lib" ./configure --prefix=$HOME/dev --with-png=$HOME/dev --with-freetype=$HOME/dev --with-jpeg=$HOME/dev
make INCLUDEDIRS="-I. -I$HOME/dev" LIBDIRS="-L$HOME/dev" LIBS="-lgd -lpng -lz -lm" CFLAGS="-O -DHAVE_LIBPNG"
make install
cd..

For these, I chose not to create individual versions of them. You will also note that libpng is 1.2 and jpeg is V8 but says need to look at, since this doesn't seem to work with this version of gd. However, since I mostly create png images, I'm not too concerned at this time. Must sort it though eventually.

Graphviz:

This is needed for installation of some CPAN modules
http://www.graphviz.org

Follow instructions on how to install, using $HOME/dev as the prefix

Again, no version specific route taken

SLEEPYCAT libdb-4:

Download from Oracle

unarchive latest version and install

cd build_unix/
../dist/configure --prefix=$HOME/dev
make
make install

Again, no version specific route, and needed for some CPAN modules

CPAN modules:

cpanp is the recommended method to download and install modules from cpan

type

cpanp

and the interactive shell will be launched

If this is the first time, then enter the following

s conf prereqs 1; s save

This will save some of the hassle of needing to confirm installation of required modules

These are chosen because I need to set up a webserver, I work in a Bio place, and some are personal choice. Obviously, if you need others, or not some of these, then pick and choose. They are also loaded in this order for convenience and dependencies.

To install a cpan module, just type

i

i Bundle::LWP
i LWP::Parallel::UserAgent
i YAML::Tiny
i Module::Build
i Module::PortablePath
i Task::Moose (select all the optional loads)
i IO::Stringy
i Calendar::Simple
i List::MoreUtils (This had a checksum error, so manually downloaded)
i DateTime
i DateTime::Format::ICal
i iCal::Parser
i Digest::SHA1
i Class::Std
i Crypt::CBC
i Crypt::Blowfish
i MIME::Lite
i DBI
i DBD::mysql # force install if you've no test database available, also requires the mysql client development headers - mysql_config needs to be in your $PATH (probably ~/dev/bin).
i DBD::SQLite
i Tie::IxHash
i XML::XPathEngine
i XML::Parser (again, I got a dodgy md5)
i XML::XPath
i HTML::TreeBuilder
i XML::SAX
i XML::Simple
i XML::Handler::YAWriter
i XML::Filter::BufferText
i MLDBM
i Jcode
i Spreadsheet::WriteExcel
i Unicode::Map
i Apache::DBI
i Readonly (another dodgy md5)
i XML::FeedLite (causes lots of prereqs to be installed - would suggest a cup of tea if you have selected auto download of prereqs)
i Chart::OFC
i YAML
i Digest::SHA
i Ace # force install if fails to make as it may have problems connecting to Ace database during tests
i Bio::ASN1::EntrezGene # force install if fails as it looks as though for tests it needs a non-existent CPAN module
i Bundle::BioPerl
i GD (Why has this failed tests?)
i B/BI/BIRNEY/bioperl-1.4.tar.gz # Requires sleepycat libdb-4 to pass tests
i Bio::Das
i Bio::Das::Lite
i App::Ack

manually download and install DB_File - as you need to Change config.in to point at dev/lib and dev/include
manually download and install BerkeleyDB - as you need to Change config.in as above.

APACHE and MOD-PERL:

Apache = httpd-2.2.14;

http://httpd.apache.org/download.cgi

in dev, mkdir -p apache/2.2.14
cd apache
ln -s 2.2.14/ current

This gives space to install this version of apache into, and a softlink to the version we want to use (similar to perl above)

export LD_LIBRARY_PATH=$HOME/dev/lib
./configure --prefix=$HOME/dev/apache/2.2.14 LDFLAGS="-L/$HOME/dev/lib"

add $HOME/dev/apache/current/bin to $PATH

mod_perl 2.0:

http://perl.apache.org/download/index.html

Get latest version of 2.0
$HOME/dev/bin/perl Makefile.PL
# follow instructions, e.g. apxs is at $HOME/dev/apache/current/bin/apxs

make
make install

Change/create the $HOME/dev/apache/current/conf/httpd.conf and $HOME/dev/apache/current/conf/perlconfig.ini as you need to.

Catalyst:

Now the biggie. Catalyst has lots of dependencies. It will take some time, plus it is interactive.
Just install everything - except the extra DBD supports.
You can do them in your own time, but they may make the Install fall over now, which you don't want.

cpanp
i Task::Catalyst

If you have got through this, then congrats.

I have also downloaded into my dev area subversion and git, and have tried to do ImageMagick (although this is erroring that my C compiler won't compile executables, even though it has done svn and git).

subversion:

Retrieve the latest version and dependency from http://subversion.apache.org/source-code.html
unpack both, the dependency folder should end up in the same directory, and will then be installed with svn

mkdir -p $HOME/dev/subversion/
cd $HOME/dev/subversion
ln -s current

cd into unpacked folder

./configure --prefix=$HOME/dev/subversion/ --eprefix=$HOME/dev/subversion/
make
make install

add $HOME/dev/subversion/current/bin to your $PATH

This will enable you to have/try multiple versions of svn in the same way as Perl and Apache above

git:

Retrieve the latest version from http://git-scm.com/
unpack

mkdir -p $HOME/dev/git/
cd $HOME/dev/git
ln -s current

make configure
./configure --prefix=$HOME/dev/git/
make
make install (had to do as root)

ImageMagick:

problem with my gcc version at this time

mkdir -p $HOME/dev/imageMagick/
cd $HOME/dev/imageMagick
ln -s current

./configure PREFIX=/Users/ajb/dev/imageMagick/6.5.9 EXEC-PREFIX=/Users/ajb/dev/imageMagick/6.5.9 LIBS=-l/Users/ajb/dev/lib --enable-shared --disable-static

After this, you should have a nice fairly 'clean' version of a dev area. If you want ot install other stuff, then I would recommend the method suggested for versioning the download you have. (I can also recommend the MOCA installation idea for mysql, although I choose to not have that in my dev area).

Once inside this, I then create folders for my projects, using svn or git to version control within those folders, just adding the directories to my path as I need to.

Note: I am using MAC OSX Leopard. At times for the make install, I have needed to sudo make install. I accept no liability for anything that happens should you follow these instructions on any system, but hope that they might be useful for anyone who would like to set up a dev/test area, but are not sure how to go about it.

Cheers