Finally, I get somewhere, I hope

We can't expect things to stay static, but you would hope that when they don't, you find out with time to make changes, and that they tell you!

However, this doesn't always happen, and sometimes the changes are quite major. As any regular readers know, I work at a high throughput sequencing facility, with a major analysis pipeline running all the time. We need to be adaptable, and have become more so, but something changed which hit us hard.

All of a sudden, something that previously expected 3 inputs, suddenly wanted 2. Also, those two inputs are 1 and 3. And the description needed the middle part stripped out. And the files needed changing both name and internally. And ...

You get the picture.

The problem occurs because we are expecting certain outputs, in a certain way to produce the final files that our researchers are expecting. This was no longer going to be what we would get. AAHHH!

So, a think about it, a look around the code producing the files, and we come up with a strategy.

1) Rename the 3 files such that the middle file only psuedo exists to the next part of the pipeline.

2) Run the next part of the pipeline with only those two files.

3) Rename the files back to the original expected names, moving in the middle file again.

4) Keep our fingers crossed :)

So, first things first, rename the files. Now at this point, we think there are only 3 input files to 'Stage 2' as I'll call it. By stripping out the middle descriptor, and renaming, we run the pipeline.

Error...

The error, 'Stage 2' believes there to be a middle descriptor still. It turns out two files are created before 'Stage 1', in prep for 'Stage 2', still with the middle descriptor in it. 'Stage 2' then reads this file, and submits the middle descriptor to the scripts it kicks off. Pants.

So, change these created files. Unfortunately, this can't be done from the config file we create, so after creating the other files, go in and physically change them

Result! But then...

We thought it wanted 3 files, reduced to 2. Actually, it wants 6 reduced to 4. Another set of 3 files, are required.

So, rename those and Result.

I have now successfully run 'Stage 2'.

The next step is to get the stuff back to how we expect for the production of our own output files. Wish me luck, I'm going in.