memory leak in Bio::Species

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

memory leak in Bio::Species

Dmitry Karasik
Dear all,

I've hit a memory leak issue that OOMs our daemons once in a while, and what is
worse, I don't have the BioPerl expertise to fix it (I would send a patch
otherwise ). The problem is that Bio::Species and/or the modules it uses forms
cyclic references, which are never killed by perl automatically. I guess
either some internal data structure has to be reworked, or there should be some
strategic placing of Scalar::Util::weaken, but I have no idea where (or rather,
I could devise a hack that hammers weaken() instantly, but I don't think
this is the right approach).

It's very simple to reproduce, f.ex. by this:

        use Devel::Cycle;
        use Bio::Species;
        find_cycle(Bio::Species->new(-classification => ['A']));

which outputs

        Cycle (1):
           $Bio::Species::A->{'taxon'} => \%Bio::Taxon::B              
         $Bio::Taxon::B->{'_ancestor'} => \%Bio::Taxon::C              
             $Bio::Taxon::C->{'_desc'} => \%D                          
                             $D->{'1'} => \%Bio::Taxon::B              

        Cycle (2):
            $Bio::Species::A->{'tree'} => \%Bio::Tree::Tree::E          
        $Bio::Tree::Tree::E->{'_rootnode'} => \%Bio::Taxon::C              
             $Bio::Taxon::C->{'_desc'} => \%D                          
                             $D->{'1'} => \%Bio::Taxon::B              
         $Bio::Taxon::B->{'_ancestor'} => \%Bio::Taxon::C    

whereas I would expect it would print nothing.

I should really much like to ask the devs for a closer look. It's here on
github: https://github.com/bioperl/bioperl-live/issues/81

Thank you in advance!

--
Sincerely,
        Dmitry Karasik
_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: memory leak in Bio::Species

Mark A. Jensen

Looks like a 'self-as-descendant' idea in the data structure. If the algorithms depend on that, I'm sure the right person could add a strategic weaken.


On Fri, Sep 12, 2014 at 2:22 PM, Dmitry Karasik <[hidden email]> wrote:

Dear all,

I've hit a memory leak issue that OOMs our daemons once in a while, and what is
worse, I don't have the BioPerl expertise to fix it (I would send a patch
otherwise ). The problem is that Bio::Species and/or the modules it uses forms
cyclic references, which are never killed by perl automatically. I guess
either some internal data structure has to be reworked, or there should be some
strategic placing of Scalar::Util::weaken, but I have no idea where (or rather,
I could devise a hack that hammers weaken() instantly, but I don't think
this is the right approach).

It's very simple to reproduce, f.ex. by this:

use Devel::Cycle;
use Bio::Species;
find_cycle(Bio::Species->new(-classification => ['A']));

which outputs

Cycle (1):
           $Bio::Species::A->{'taxon'} => \%Bio::Taxon::B              
         $Bio::Taxon::B->{'_ancestor'} => \%Bio::Taxon::C              
             $Bio::Taxon::C->{'_desc'} => \%D                          
                             $D->{'1'} => \%Bio::Taxon::B              

Cycle (2):
            $Bio::Species::A->{'tree'} => \%Bio::Tree::Tree::E         
        $Bio::Tree::Tree::E->{'_rootnode'} => \%Bio::Taxon::C              
             $Bio::Taxon::C->{'_desc'} => \%D                          
                             $D->{'1'} => \%Bio::Taxon::B              
         $Bio::Taxon::B->{'_ancestor'} => \%Bio::Taxon::C    

whereas I would expect it would print nothing.

I should really much like to ask the devs for a closer look. It's here on
github: https://github.com/bioperl/bioperl-live/issues/81

Thank you in advance!

--
Sincerely,
Dmitry Karasik
_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l


_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: memory leak in Bio::Species

Mark A. Jensen
In reply to this post by Dmitry Karasik

Looks like a 'self-as-descendant' idea in the data structure. If the algorithms depend on that, I'm sure the right person could add a strategic weaken.


On Fri, Sep 12, 2014 at 2:22 PM, Dmitry Karasik <[hidden email]> wrote:

Dear all,

I've hit a memory leak issue that OOMs our daemons once in a while, and what is
worse, I don't have the BioPerl expertise to fix it (I would send a patch
otherwise ). The problem is that Bio::Species and/or the modules it uses forms
cyclic references, which are never killed by perl automatically. I guess
either some internal data structure has to be reworked, or there should be some
strategic placing of Scalar::Util::weaken, but I have no idea where (or rather,
I could devise a hack that hammers weaken() instantly, but I don't think
this is the right approach).

It's very simple to reproduce, f.ex. by this:

use Devel::Cycle;
use Bio::Species;
find_cycle(Bio::Species->new(-classification => ['A']));

which outputs

Cycle (1):
           $Bio::Species::A->{'taxon'} => \%Bio::Taxon::B              
         $Bio::Taxon::B->{'_ancestor'} => \%Bio::Taxon::C              
             $Bio::Taxon::C->{'_desc'} => \%D                          
                             $D->{'1'} => \%Bio::Taxon::B              

Cycle (2):
            $Bio::Species::A->{'tree'} => \%Bio::Tree::Tree::E         
        $Bio::Tree::Tree::E->{'_rootnode'} => \%Bio::Taxon::C              
             $Bio::Taxon::C->{'_desc'} => \%D                          
                             $D->{'1'} => \%Bio::Taxon::B              
         $Bio::Taxon::B->{'_ancestor'} => \%Bio::Taxon::C    

whereas I would expect it would print nothing.

I should really much like to ask the devs for a closer look. It's here on
github: https://github.com/bioperl/bioperl-live/issues/81

Thank you in advance!

--
Sincerely,
Dmitry Karasik
_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l


_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Loading...