Spankin' new (alpha) build system for Bioperl-Run

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Spankin' new (alpha) build system for Bioperl-Run

Mark A. Jensen
All (esp. George)-

My work on Issue #11 (https://github.com/bioperl/bioperl-run/issues/11)
has metastasized.

The proximate problem was tests that fail because of once-local
prerequisites. The ultimate
problems are

- Why should I have to install every single wrapper when I only want X?
- Why should I care about any test that doesn't deal with X?
- Why doesn't X bring along its own prereq metadata (including Bio
prereqs),
   rather than tag along with the distro and hope for the best?

(And I think these are the ultimate problems across BioPerl in terms of
decentralized
distribution.)

My solution was

- Add to the distro real, manually prepared metadata on prerequisites
for all
   the tools
- Add an interactive selector that allows a user to pick their desired
tools at
   perl Build.PL-time
- Have Module::Build check only (and ALL) the prereqs of the desired
tools, and
   inform user of missing ones at perl Build.PL-time
- Make use of the persistence of the config information to skip/run .t
files as
   appropriate
- Update ALL the tests to check whether to skip based on user selection
- Make M::B install only the relevant distro modules and documentation,
not everything,
   at ./Build install-time

This is ready for brave alpha-testers at
https://github.com/bioperl/bioperl-run/tree/topic/issue11.
Just do 'perl Build.PL'.

Pod below has some more details-- comments very welcome

MAJ

NAME
     Bio::Tools::Run::Build - Instrument the build for features

SYNOPSIS

...

DESCRIPTION
     Bio::Tools::Run::Build is a subclass of Module::Build that allows
an
     author to offer users the ability to select and install
pre-configured
     subsets of modules that are packaged in a single large M::B-based
     distribution.

     Grouping and selection of distro modules is driven by the optional
     features concept as defined in CPAN::Meta::Spec and used by
     Module::Build.

     The subclass provides the following:

     *   Author specification of features and their prereqs

         The build author develops metadata files in json that follow
         "optional_features" in CPAN::Meta::Spec to group distribution
         modules and dependencies as selectable features.

     *   Interactive user selection of features

         The user can be presented with an interactive selector during
         Build.PL runs.

     *   Prereq checking of user selected features only

         M::B only checks for the presence of selected feature
dependencies.

     *   Build-persistent recording of user selections

         The build object records the selection of features in the
         $build->feature field. This can be used in test files to
determine
         whether tests should be skipped (and not failed). See
         Bio::Tools::Run::Build::Test.

     *   Installation only of selected feature modules

         Bio::Tools::Run::Build adds a build action, "deselect", which
runs
         after the "code" and "docs" actions. "deselect" removes
unselected
         modules from the blib/lib directory and unneeded documentation
from
         the blib/libdoc directory. This keeps the "install" action from
         installing unwanted files.

MOTIVATION
     The BioPerl-Run distribution contains a large variety of wrappers
and
     parsers that handle the execution and output of many different
     bioinformatics tools. It has been provided as a large distro that
     installs and attempts to test all of its modules. Many users need
only a
     small fraction of the functionality BioPerl-Run provides, relevant
only
     to the tools they have installed. On the other hand, managing many
     different packages is unwieldy and uninviting for volunteer
maintainers.

     The system described here is a compromise that enables a user to
select,
     test and install only those modules that meet the need, yet reduces
the
     maintenance effort to the management of a set of metadata files in
a
     single distribution.

...
_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: Spankin' new (alpha) build system for Bioperl-Run

Mark A. Jensen
(minus damn linebreaks)

All (esp. George)-

My work on Issue #11 (https://github.com/bioperl/bioperl-run/issues/11) has metastasized.

The proximate problem was tests that fail because of once-local prerequisites. The ultimate problems are

- Why should I have to install every single wrapper when I only want X?

- Why should I care about any test that doesn't deal with X?

- Why doesn't X bring along its own prereq metadata (including Bio prereqs), rather than tag along with the distro and hope for the best?

(And I think these are the ultimate problems across BioPerl in terms of decentralized distribution.)

My solution was

- Add to the distro real, manually prepared metadata on prerequisites for all the tools

- Add an interactive selector that allows a user to pick their desired tools at perl Build.PL-time

- Have Module::Build check only (and ALL) the prereqs of the desired tools, and inform user of missing ones at perl Build.PL-time

- Make use of the persistence of the config information to skip/run .t files as appropriate

- Update ALL the tests to check whether to skip based on user selection

- Make M::B install only the relevant distro modules and documentation, not everything, at ./Build install-time

This is ready for brave alpha-testers at https://github.com/bioperl/bioperl-run/tree/topic/issue11. Just do 'perl Build.PL'.

Pod below has some more details-- comments very welcome

MAJ

NAME
Bio::Tools::Run::Build - Instrument the build for features

SYNOPSIS

...

DESCRIPTION

Bio::Tools::Run::Build is a subclass of Module::Build that allows an author to offer users the ability to select and install pre-configured subsets of modules that are packaged in a single large M::B-based distribution.

Grouping and selection of distro modules is driven by the optional features concept as defined in CPAN::Meta::Spec and used by Module::Build.

The subclass provides the following:

* Author specification of features and their prereqs

The build author develops metadata files in json that follow "optional_features" in CPAN::Meta::Spec to group distribution modules and dependencies as selectable features.

* Interactive user selection of features

The user can be presented with an interactive selector during Build.PL runs.

* Prereq checking of user selected features only

M::B only checks for the presence of selected feature dependencies.

* Build-persistent recording of user selections

The build object records the selection of features in the $build->feature field. This can be used in test files to determine whether tests should be skipped (and not failed). See Bio::Tools::Run::Build::Test.

* Installation only of selected feature modules

Bio::Tools::Run::Build adds a build action, "deselect", which runs after the "code" and "docs" actions. "deselect" removes unselected modules from the blib/lib directory and unneeded documentation from the blib/libdoc directory. This keeps the "install" action from installing unwanted files.

MOTIVATION

The BioPerl-Run distribution contains a large variety of wrappers and parsers that handle the execution and output of many different bioinformatics tools. It has been provided as a large distro that installs and attempts to test all of its modules. Many users need only a small fraction of the functionality BioPerl-Run provides, relevant only to the tools they have installed. On the other hand, managing many different packages is unwieldy and uninviting for volunteer maintainers.

The system described here is a compromise that enables a user to select, test and install only those modules that meet the need, yet reduces the maintenance effort to the management of a set of metadata files in a single distribution.
On 2014-09-29 23:41, Mark A. Jensen wrote: > All (esp. George)- >
All (esp. George)-

My work on Issue #11 (https://github.com/bioperl/bioperl-run/issues/11) has metastasized.

The proximate problem was tests that fail because of once-local prerequisites. The ultimate
problems are

- Why should I have to install every single wrapper when I only want X?
- Why should I care about any test that doesn't deal with X?
- Why doesn't X bring along its own prereq metadata (including Bio prereqs),
  rather than tag along with the distro and hope for the best?

(And I think these are the ultimate problems across BioPerl in terms of decentralized
distribution.)

My solution was

- Add to the distro real, manually prepared metadata on prerequisites for all
  the tools
- Add an interactive selector that allows a user to pick their desired tools at
  perl Build.PL-time
- Have Module::Build check only (and ALL) the prereqs of the desired tools, and
  inform user of missing ones at perl Build.PL-time
- Make use of the persistence of the config information to skip/run .t files as
  appropriate
- Update ALL the tests to check whether to skip based on user selection
- Make M::B install only the relevant distro modules and documentation, not everything,
  at ./Build install-time

This is ready for brave alpha-testers at https://github.com/bioperl/bioperl-run/tree/topic/issue11.
Just do 'perl Build.PL'.

Pod below has some more details-- comments very welcome

MAJ

NAME
    Bio::Tools::Run::Build - Instrument the build for features

SYNOPSIS

...

DESCRIPTION
    Bio::Tools::Run::Build is a subclass of Module::Build that allows an
    author to offer users the ability to select and install pre-configured
    subsets of modules that are packaged in a single large M::B-based
    distribution.

    Grouping and selection of distro modules is driven by the optional
    features concept as defined in CPAN::Meta::Spec and used by
    Module::Build.

    The subclass provides the following:

    *   Author specification of features and their prereqs

        The build author develops metadata files in json that follow
        "optional_features" in CPAN::Meta::Spec to group distribution
        modules and dependencies as selectable features.

    *   Interactive user selection of features

        The user can be presented with an interactive selector during
        Build.PL runs.

    *   Prereq checking of user selected features only

        M::B only checks for the presence of selected feature dependencies.

    *   Build-persistent recording of user selections

        The build object records the selection of features in the
        $build->feature field. This can be used in test files to determine
        whether tests should be skipped (and not failed). See
        Bio::Tools::Run::Build::Test.

    *   Installation only of selected feature modules

        Bio::Tools::Run::Build adds a build action, "deselect", which runs
        after the "code" and "docs" actions. "deselect" removes unselected
        modules from the blib/lib directory and unneeded documentation from
        the blib/libdoc directory. This keeps the "install" action from
        installing unwanted files.

MOTIVATION
    The BioPerl-Run distribution contains a large variety of wrappers and
    parsers that handle the execution and output of many different
    bioinformatics tools. It has been provided as a large distro that
    installs and attempts to test all of its modules. Many users need only a
    small fraction of the functionality BioPerl-Run provides, relevant only
    to the tools they have installed. On the other hand, managing many
    different packages is unwieldy and uninviting for volunteer maintainers.

    The system described here is a compromise that enables a user to select,
    test and install only those modules that meet the need, yet reduces the
    maintenance effort to the management of a set of metadata files in a
    single distribution.

...

_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l
Reply | Threaded
Open this post in threaded view
|

Re: Spankin' new (alpha) build system for Bioperl-Run

Dave Messina-4
Very cool! Nice work Mark!!




On Mon, Sep 29, 2014 at 11:30 PM, Mark A. Jensen <[hidden email]> wrote:
(minus damn linebreaks)


All (esp. George)-

My work on Issue #11 (https://github.com/bioperl/bioperl-run/issues/11) has metastasized.

The proximate problem was tests that fail because of once-local prerequisites. The ultimate problems are

- Why should I have to install every single wrapper when I only want X?

- Why should I care about any test that doesn't deal with X?

- Why doesn't X bring along its own prereq metadata (including Bio prereqs), rather than tag along with the distro and hope for the best?

(And I think these are the ultimate problems across BioPerl in terms of decentralized distribution.)

My solution was

- Add to the distro real, manually prepared metadata on prerequisites for all the tools

- Add an interactive selector that allows a user to pick their desired tools at perl Build.PL-time

- Have Module::Build check only (and ALL) the prereqs of the desired tools, and inform user of missing ones at perl Build.PL-time

- Make use of the persistence of the config information to skip/run .t files as appropriate

- Update ALL the tests to check whether to skip based on user selection

- Make M::B install only the relevant distro modules and documentation, not everything, at ./Build install-time

This is ready for brave alpha-testers at https://github.com/bioperl/bioperl-run/tree/topic/issue11. Just do 'perl Build.PL'.

Pod below has some more details-- comments very welcome

MAJ

NAME
Bio::Tools::Run::Build - Instrument the build for features

SYNOPSIS

...

DESCRIPTION

Bio::Tools::Run::Build is a subclass of Module::Build that allows an author to offer users the ability to select and install pre-configured subsets of modules that are packaged in a single large M::B-based distribution.

Grouping and selection of distro modules is driven by the optional features concept as defined in CPAN::Meta::Spec and used by Module::Build.

The subclass provides the following:

* Author specification of features and their prereqs

The build author develops metadata files in json that follow "optional_features" in CPAN::Meta::Spec to group distribution modules and dependencies as selectable features.

* Interactive user selection of features

The user can be presented with an interactive selector during Build.PL runs.

* Prereq checking of user selected features only

M::B only checks for the presence of selected feature dependencies.

* Build-persistent recording of user selections

The build object records the selection of features in the $build->feature field. This can be used in test files to determine whether tests should be skipped (and not failed). See Bio::Tools::Run::Build::Test.

* Installation only of selected feature modules

Bio::Tools::Run::Build adds a build action, "deselect", which runs after the "code" and "docs" actions. "deselect" removes unselected modules from the blib/lib directory and unneeded documentation from the blib/libdoc directory. This keeps the "install" action from installing unwanted files.

MOTIVATION

The BioPerl-Run distribution contains a large variety of wrappers and parsers that handle the execution and output of many different bioinformatics tools. It has been provided as a large distro that installs and attempts to test all of its modules. Many users need only a small fraction of the functionality BioPerl-Run provides, relevant only to the tools they have installed. On the other hand, managing many different packages is unwieldy and uninviting for volunteer maintainers.

The system described here is a compromise that enables a user to select, test and install only those modules that meet the need, yet reduces the maintenance effort to the management of a set of metadata files in a single distribution.
On 2014-09-29 23:41, Mark A. Jensen wrote: > All (esp. George)- >
All (esp. George)-

My work on Issue #11 (https://github.com/bioperl/bioperl-run/issues/11) has metastasized.

The proximate problem was tests that fail because of once-local prerequisites. The ultimate
problems are

- Why should I have to install every single wrapper when I only want X?
- Why should I care about any test that doesn't deal with X?
- Why doesn't X bring along its own prereq metadata (including Bio prereqs),
  rather than tag along with the distro and hope for the best?

(And I think these are the ultimate problems across BioPerl in terms of decentralized
distribution.)

My solution was

- Add to the distro real, manually prepared metadata on prerequisites for all
  the tools
- Add an interactive selector that allows a user to pick their desired tools at
  perl Build.PL-time
- Have Module::Build check only (and ALL) the prereqs of the desired tools, and
  inform user of missing ones at perl Build.PL-time
- Make use of the persistence of the config information to skip/run .t files as
  appropriate
- Update ALL the tests to check whether to skip based on user selection
- Make M::B install only the relevant distro modules and documentation, not everything,
  at ./Build install-time

This is ready for brave alpha-testers at https://github.com/bioperl/bioperl-run/tree/topic/issue11.
Just do 'perl Build.PL'.

Pod below has some more details-- comments very welcome

MAJ

NAME
    Bio::Tools::Run::Build - Instrument the build for features

SYNOPSIS

...

DESCRIPTION
    Bio::Tools::Run::Build is a subclass of Module::Build that allows an
    author to offer users the ability to select and install pre-configured
    subsets of modules that are packaged in a single large M::B-based
    distribution.

    Grouping and selection of distro modules is driven by the optional
    features concept as defined in CPAN::Meta::Spec and used by
    Module::Build.

    The subclass provides the following:

    *   Author specification of features and their prereqs

        The build author develops metadata files in json that follow
        "optional_features" in CPAN::Meta::Spec to group distribution
        modules and dependencies as selectable features.

    *   Interactive user selection of features

        The user can be presented with an interactive selector during
        Build.PL runs.

    *   Prereq checking of user selected features only

        M::B only checks for the presence of selected feature dependencies.

    *   Build-persistent recording of user selections

        The build object records the selection of features in the
        $build->feature field. This can be used in test files to determine
        whether tests should be skipped (and not failed). See
        Bio::Tools::Run::Build::Test.

    *   Installation only of selected feature modules

        Bio::Tools::Run::Build adds a build action, "deselect", which runs
        after the "code" and "docs" actions. "deselect" removes unselected
        modules from the blib/lib directory and unneeded documentation from
        the blib/libdoc directory. This keeps the "install" action from
        installing unwanted files.

MOTIVATION
    The BioPerl-Run distribution contains a large variety of wrappers and
    parsers that handle the execution and output of many different
    bioinformatics tools. It has been provided as a large distro that
    installs and attempts to test all of its modules. Many users need only a
    small fraction of the functionality BioPerl-Run provides, relevant only
    to the tools they have installed. On the other hand, managing many
    different packages is unwieldy and uninviting for volunteer maintainers.

    The system described here is a compromise that enables a user to select,
    test and install only those modules that meet the need, yet reduces the
    maintenance effort to the management of a set of metadata files in a
    single distribution.

...
Message 1 of 152

_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l


_______________________________________________
Bioperl-l mailing list
[hidden email]
http://mailman.open-bio.org/mailman/listinfo/bioperl-l