issue writing to pipe with bio sequoia

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

issue writing to pipe with bio sequoia

Stephane Plaisance | VIB |
Dear,
I try to bgzip my fasta outpmut fbut have an issue


my $seq_out;
if ( defined($zipit) ) {
        my $bgzip = `which bgzip`;
        chomp($bgzip);
        die "No bgzip command available\n" unless ( $bgzip );
        $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => "  | $bgzip -c >  $outfile\.gz");
} else {
        $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => ">$outfile" );
}

the code fails although bgzip is in my path and the  $bgzip variable sets it right
Any help is very welcome

Stephane
_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: issue writing to pipe with bio sequoia

Roy Chaudhuri-3
Hi Stephane,

According to the SeqIO docs your way should work, so this is probably a
bug. However, Bio::SeqIO can accept a filehandle instead using the -fh
option, so as a workaround you could try something like this:

open my $fh, '|$bgzip -c > $outfile.gz' or die $!;
$seq_out=Bio::SeqIO->newFh(-format=>'fasta', -fh=>$fh);

Cheers,
Roy.

On 31/05/2016 15:06, Stephane Plaisance | VIB | wrote:

> Dear,
> I try to bgzip my fasta outpmut fbut have an issue
>
>
> my $seq_out;
> if ( defined($zipit) ) {
> my $bgzip = `which bgzip`;
> chomp($bgzip);
> die "No bgzip command available\n" unless ( $bgzip );
> $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => "  | $bgzip -c >  $outfile\.gz");
> } else {
> $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => ">$outfile" );
> }
>
> the code fails although bgzip is in my path and the  $bgzip variable sets it right
> Any help is very welcome
>
> Stephane
> _______________________________________________
> Bioperl-l mailing list
> [hidden email]
> http://mailman.open-bio.org/mailman/listinfo/bioperl-l
>
_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: issue writing to pipe with bio sequoia

Mark A. Jensen

Stephane, Roy
Can we open issue at github, bioperl-live ?
Mark

On Tue, May 31, 2016 at 10:53 AM, Roy Chaudhuri <[hidden email]> wrote:

Hi Stephane,

According to the SeqIO docs your way should work, so this is probably a
bug. However, Bio::SeqIO can accept a filehandle instead using the -fh
option, so as a workaround you could try something like this:

open my $fh, '|$bgzip -c > $outfile.gz' or die $!;
$seq_out=Bio::SeqIO->newFh(-format=>'fasta', -fh=>$fh);

Cheers,
Roy.

On 31/05/2016 15:06, Stephane Plaisance | VIB | wrote:
> Dear,
> I try to bgzip my fasta outpmut fbut have an issue
>
>
> my $seq_out;
> if ( defined($zipit) ) {
> my $bgzip = `which bgzip`;
> chomp($bgzip);
> die "No bgzip command available\n" unless ( $bgzip );
> $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => "  | $bgzip -c >  $outfile\.gz");
> } else {
> $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => ">$outfile" );
> }
>
> the code fails although bgzip is in my path and the  $bgzip variable sets it right
> Any help is very welcome
>
> Stephane
> _______________________________________________
> Bioperl-l mailing list
> [hidden email]
> http://mailman.open-bio.org/mailman/listinfo/bioperl-l
>
_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l


_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: issue writing to pipe with bio sequoia

Roy Chaudhuri-3
In reply to this post by Roy Chaudhuri-3
Hi Stephane,

Please remember to copy in the mailing list on replies. In my tests, a
leading space doesn't make any difference to the issue - Bio::SeqIO
still tries to read from the pipe rather than writing to it.

Mark - I have opened an issue on github:
https://github.com/bioperl/bioperl-live/issues/153

I think the issue is the regex in Bio::Root::IO->cleanfile not
recognising a leading pipe character as indicating a "write" filehandle.

Cheers,
Roy.

On 31/05/2016 15:58, Stephane Plaisance | VIB | wrote:

> Thanks a lot Roy,
>
> I found that a leading space before the pipe and possibly the space directly after it were guilty, leading to the executable not found although fully provided with path ??? what works is
>>> -file => " |$bgzip -c >  $outfile\.gz")
>
> but you fh version is precious and I will record it for future use.
>
> Thanks!
> stephane
>
>> On 31 May 2016, at 16:45, Roy Chaudhuri <[hidden email]> wrote:
>>
>> Hi Stephane,
>>
>> According to the SeqIO docs your way should work, so this is probably a bug. However, Bio::SeqIO can accept a filehandle instead using the -fh option, so as a workaround you could try something like this:
>>
>> open my $fh, '|$bgzip -c > $outfile.gz' or die $!;
>> $seq_out=Bio::SeqIO->newFh(-format=>'fasta', -fh=>$fh);
>>
>> Cheers,
>> Roy.
>>
>> On 31/05/2016 15:06, Stephane Plaisance | VIB | wrote:
>>> Dear,
>>> I try to bgzip my fasta outpmut fbut have an issue
>>>
>>>
>>> my $seq_out;
>>> if ( defined($zipit) ) {
>>> my $bgzip = `which bgzip`;
>>> chomp($bgzip);
>>> die "No bgzip command available\n" unless ( $bgzip );
>>> $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => "  | $bgzip -c >  $outfile\.gz");
>>> } else {
>>> $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => ">$outfile" );
>>> }
>>>
>>> the code fails although bgzip is in my path and the  $bgzip variable sets it right
>>> Any help is very welcome
>>>
>>> Stephane
>>> _______________________________________________
>>> Bioperl-l mailing list
>>> [hidden email]
>>> http://mailman.open-bio.org/mailman/listinfo/bioperl-l
>>>
>
_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: issue writing to pipe with bio sequoia

Mark A. Jensen

Awesome Roy, thanks !

On Tue, May 31, 2016 at 11:39 AM, Roy Chaudhuri <[hidden email]> wrote:

Hi Stephane,

Please remember to copy in the mailing list on replies. In my tests, a
leading space doesn't make any difference to the issue - Bio::SeqIO
still tries to read from the pipe rather than writing to it.

Mark - I have opened an issue on github:
https://github.com/bioperl/bioperl-live/issues/153

I think the issue is the regex in Bio::Root::IO->cleanfile not
recognising a leading pipe character as indicating a "write" filehandle.

Cheers,
Roy.

On 31/05/2016 15:58, Stephane Plaisance | VIB | wrote:
> Thanks a lot Roy,
>
> I found that a leading space before the pipe and possibly the space directly after it were guilty, leading to the executable not found although fully provided with path ??? what works is
>>> -file => " |$bgzip -c >  $outfile\.gz")
>
> but you fh version is precious and I will record it for future use.
>
> Thanks!
> stephane
>
>> On 31 May 2016, at 16:45, Roy Chaudhuri <[hidden email]> wrote:
>>
>> Hi Stephane,
>>
>> According to the SeqIO docs your way should work, so this is probably a bug. However, Bio::SeqIO can accept a filehandle instead using the -fh option, so as a workaround you could try something like this:
>>
>> open my $fh, '|$bgzip -c > $outfile.gz' or die $!;
>> $seq_out=Bio::SeqIO->newFh(-format=>'fasta', -fh=>$fh);
>>
>> Cheers,
>> Roy.
>>
>> On 31/05/2016 15:06, Stephane Plaisance | VIB | wrote:
>>> Dear,
>>> I try to bgzip my fasta outpmut fbut have an issue
>>>
>>>
>>> my $seq_out;
>>> if ( defined($zipit) ) {
>>> my $bgzip = `which bgzip`;
>>> chomp($bgzip);
>>> die "No bgzip command available\n" unless ( $bgzip );
>>> $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => "  | $bgzip -c >  $outfile\.gz");
>>> } else {
>>> $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => ">$outfile" );
>>> }
>>>
>>> the code fails although bgzip is in my path and the  $bgzip variable sets it right
>>> Any help is very welcome
>>>
>>> Stephane
>>> _______________________________________________
>>> Bioperl-l mailing list
>>> [hidden email]
>>> http://mailman.open-bio.org/mailman/listinfo/bioperl-l
>>>
>


_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: issue writing to pipe with bio sequoia

Fields, Christopher J
In reply to this post by Stephane Plaisance | VIB |
If you are using a piped command you should use a file handle as input, not a file.  Something like:

open(my $fh,  "  | $bgzip -c >  $outfile\.gz”);
$seq_out = Bio::SeqIO->new( -format => 'Fasta', -fh => $fh);

The newFh() method is used for *returning* a tied filehandle for I/O; IMHO I don't recommend this, as it’s pretty confusing and hard to debug (I avoid most anything related to tied variables for the same reason unless it’s absolutely necessary).  It is not used for dealing with a file handle input.

chris

> On May 31, 2016, at 9:06 AM, Stephane Plaisance | VIB | <[hidden email]> wrote:
>
> Dear,
> I try to bgzip my fasta outpmut fbut have an issue
>
>
> my $seq_out;
> if ( defined($zipit) ) {
> my $bgzip = `which bgzip`;
> chomp($bgzip);
> die "No bgzip command available\n" unless ( $bgzip );
> $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => "  | $bgzip -c >  $outfile\.gz");
> } else {
> $seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => ">$outfile" );
> }
>
> the code fails although bgzip is in my path and the  $bgzip variable sets it right
> Any help is very welcome
>
> Stephane
> _______________________________________________
> Bioperl-l mailing list
> [hidden email]
> http://mailman.open-bio.org/mailman/listinfo/bioperl-l


_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: issue writing to pipe with bio sequoia

Stephane Plaisance | VIB |
Thanks a lot Chris,

Your code example is much more instructive than the text I found on the IO man pages.
If someone has access to (http://bioperl.org/howtos/SeqIO_HOWTO.html) and could edit some examples for saving to bgzip it would probably help others.
My code now works like a charm and I generate smaller compressed and tabix index-able bgzip Fasta instead of huge plain files.

THANKS to ALL who helped me

Stephane

On 31 May 2016, at 20:59, Fields, Christopher J <[hidden email]> wrote:

If you are using a piped command you should use a file handle as input, not a file.  Something like:

open(my $fh,  "  | $bgzip -c >  $outfile\.gz”);
$seq_out = Bio::SeqIO->new( -format => 'Fasta', -fh => $fh);

The newFh() method is used for *returning* a tied filehandle for I/O; IMHO I don't recommend this, as it’s pretty confusing and hard to debug (I avoid most anything related to tied variables for the same reason unless it’s absolutely necessary).  It is not used for dealing with a file handle input.

chris

On May 31, 2016, at 9:06 AM, Stephane Plaisance | VIB | <[hidden email]> wrote:

Dear,
I try to bgzip my fasta outpmut fbut have an issue


my $seq_out;
if ( defined($zipit) ) {
my $bgzip = `which bgzip`;
chomp($bgzip);
die "No bgzip command available\n" unless ( $bgzip );
$seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => "  | $bgzip -c >  $outfile\.gz");
} else {
$seq_out = Bio::SeqIO -> newFh( -format => 'Fasta', -file => ">$outfile" );
}

the code fails although bgzip is in my path and the  $bgzip variable sets it right
Any help is very welcome

Stephane
_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l



_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: issue writing to pipe with bio sequoia

Peter Cock
Hi Stephane,

The website source for that page is now here:

https://github.com/bioperl/bioperl.github.io/blob/master/howtos/SeqIO_HOWTO.md

You can edit it within the GitHub web interface and make a pull request (remember to check the preview).

Chris etc: Can you apply something like this to add an edit button/link to each page?


Regards,

Peter

On Wed, Jun 1, 2016 at 9:28 AM, Stephane Plaisance | VIB | <[hidden email]> wrote:
Thanks a lot Chris,

Your code example is much more instructive than the text I found on the IO man pages.
If someone has access to (http://bioperl.org/howtos/SeqIO_HOWTO.html) and could edit some examples for saving to bgzip it would probably help others.
My code now works like a charm and I generate smaller compressed and tabix index-able bgzip Fasta instead of huge plain files.

THANKS to ALL who helped me

Stephane


_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: issue writing to pipe with bio sequoia

Fields, Christopher J
Sure, should be easy enough to add.

-c

On Jun 1, 2016, at 11:12 AM, Peter Cock <[hidden email]> wrote:

Hi Stephane,

The website source for that page is now here:

https://github.com/bioperl/bioperl.github.io/blob/master/howtos/SeqIO_HOWTO.md

You can edit it within the GitHub web interface and make a pull request (remember to check the preview).

Chris etc: Can you apply something like this to add an edit button/link to each page?


Regards,

Peter

On Wed, Jun 1, 2016 at 9:28 AM, Stephane Plaisance | VIB | <[hidden email]> wrote:
Thanks a lot Chris,

Your code example is much more instructive than the text I found on the IO man pages.
If someone has access to (http://bioperl.org/howtos/SeqIO_HOWTO.html) and could edit some examples for saving to bgzip it would probably help others.
My code now works like a charm and I generate smaller compressed and tabix index-able bgzip Fasta instead of huge plain files.

THANKS to ALL who helped me

Stephane



_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: issue writing to pipe with bio sequoia

Stephane Plaisance | VIB |
In reply to this post by Peter Cock
I will do
maybe some adjustments should be done in order for people to find that new page, I did not find it in google when I searched for info. ( a link in the old page???)

Best
;-)

Nucleomics Core      Stephane Plaisance, Ph.D

Address: Herestraat 49, O&N4, Post Box 816, Room nr. 404-24 / 08.428, 3000 Leuven - Belgium
Tel: +32 (0)16 373126
Tel (lync): +32 (0)16 320060
Fax: +32 (0)16 373129

Web: www.nucleomics.be

On 01 Jun 2016, at 18:12, Peter Cock <[hidden email]> wrote:

Hi Stephane,

The website source for that page is now here:

https://github.com/bioperl/bioperl.github.io/blob/master/howtos/SeqIO_HOWTO.md

You can edit it within the GitHub web interface and make a pull request (remember to check the preview).

Chris etc: Can you apply something like this to add an edit button/link to each page?


Regards,

Peter

On Wed, Jun 1, 2016 at 9:28 AM, Stephane Plaisance | VIB | <[hidden email]> wrote:
Thanks a lot Chris,

Your code example is much more instructive than the text I found on the IO man pages.
If someone has access to (http://bioperl.org/howtos/SeqIO_HOWTO.html) and could edit some examples for saving to bgzip it would probably help others.
My code now works like a charm and I generate smaller compressed and tabix index-able bgzip Fasta instead of huge plain files.

THANKS to ALL who helped me

Stephane



_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l