Multiplexer¶
avocado_varianter_yaml_to_mux.mux
Multiplexer
or simply Mux
is an abstract concept, which was
the basic idea behind the tree-like params structure with the support
to produce all possible variants. There is a core implementation of
basic building blocks that can be used when creating a custom plugin.
There is a demonstration version of plugin using this concept in
avocado_varianter_yaml_to_mux
which adds a parser and then
uses this multiplexer concept to define an Avocado plugin to produce
variants from yaml
(or json
) files.
Multiplexer concept¶
As mentioned earlier, this is an in-core implementation of building blocks intended for writing Varianter plugins based on a tree with Multiplex domains defined. The available blocks are:
MuxTree - Object which represents a part of the tree and handles the multiplexation, which means producing all possible variants from a tree-like object.
MuxPlugin - Base class to build Varianter plugins
MuxTreeNode
- Inherits from TreeNode and adds the support for control flags (MuxTreeNode.ctrl
) and multiplex domains (MuxTreeNode.multiplex
).
And some support classes and methods eg. for filtering and so on.
Multiplex domains¶
A default avocado-params
tree with variables could look like this:
Multiplex tree representation:
┣━━ paths
┃ → tmp: /var/tmp
┃ → qemu: /usr/libexec/qemu-kvm
┗━━ environ
→ debug: False
The multiplexer wants to produce similar structure, but also to be able to define not just one variant, but to define all possible combinations and then report the slices as variants. We use the term Multiplex domains to define that children of this node are not just different paths, but they are different values and we only want one at a time. In the representation we use double-line to visibly distinguish between normal relation and multiplexed relation. Let’s modify our example a bit:
Multiplex tree representation:
┣━━ paths
┃ → tmp: /var/tmp
┃ → qemu: /usr/libexec/qemu-kvm
┗━━ environ
╠══ production
║ → debug: False
╚══ debug
→ debug: True
The difference is that environ
is now a multiplex
node and it’s
children will be yielded one at a time producing two variants:
Variant 1:
┣━━ paths
┃ → tmp: /var/tmp
┃ → qemu: /usr/libexec/qemu-kvm
┗━━ environ
┗━━ production
→ debug: False
Variant 2:
┣━━ paths
┃ → tmp: /var/tmp
┃ → qemu: /usr/libexec/qemu-kvm
┗━━ environ
┗━━ debug
→ debug: False
Note that the multiplex
is only about direct children, therefore
the number of leaves in variants might differ:
Multiplex tree representation:
┣━━ paths
┃ → tmp: /var/tmp
┃ → qemu: /usr/libexec/qemu-kvm
┗━━ environ
╠══ production
║ → debug: False
╚══ debug
┣━━ system
┃ → debug: False
┗━━ program
→ debug: True
Produces one variant with /paths
and /environ/production
and
other variant with /paths
, /environ/debug/system
and
/environ/debug/program
.
As mentioned earlier the power is not in producing one variant, but in defining huge scenarios with all possible variants. By using tree-structure with multiplex domains you can avoid most of the ugly filters you might know from Jenkins sparse matrix jobs. For comparison let’s have a look at the same example in Avocado:
Multiplex tree representation:
┗━━ os
┣━━ distro
┃ ┗━━ redhat
┃ ╠══ fedora
┃ ║ ┣━━ version
┃ ║ ┃ ╠══ 20
┃ ║ ┃ ╚══ 21
┃ ║ ┗━━ flavor
┃ ║ ╠══ workstation
┃ ║ ╚══ cloud
┃ ╚══ rhel
┃ ╠══ 5
┃ ╚══ 6
┗━━ arch
╠══ i386
╚══ x86_64
Which produces:
Variant 1: /os/distro/redhat/fedora/version/20, /os/distro/redhat/fedora/flavor/workstation, /os/arch/i386
Variant 2: /os/distro/redhat/fedora/version/20, /os/distro/redhat/fedora/flavor/workstation, /os/arch/x86_64
Variant 3: /os/distro/redhat/fedora/version/20, /os/distro/redhat/fedora/flavor/cloud, /os/arch/i386
Variant 4: /os/distro/redhat/fedora/version/20, /os/distro/redhat/fedora/flavor/cloud, /os/arch/x86_64
Variant 5: /os/distro/redhat/fedora/version/21, /os/distro/redhat/fedora/flavor/workstation, /os/arch/i386
Variant 6: /os/distro/redhat/fedora/version/21, /os/distro/redhat/fedora/flavor/workstation, /os/arch/x86_64
Variant 7: /os/distro/redhat/fedora/version/21, /os/distro/redhat/fedora/flavor/cloud, /os/arch/i386
Variant 8: /os/distro/redhat/fedora/version/21, /os/distro/redhat/fedora/flavor/cloud, /os/arch/x86_64
Variant 9: /os/distro/redhat/rhel/5, /os/arch/i386
Variant 10: /os/distro/redhat/rhel/5, /os/arch/x86_64
Variant 11: /os/distro/redhat/rhel/6, /os/arch/i386
Variant 12: /os/distro/redhat/rhel/6, /os/arch/x86_64
Versus Jenkins sparse matrix:
os_version = fedora20 fedora21 rhel5 rhel6
os_flavor = none workstation cloud
arch = i386 x86_64
filter = ((os_version == "rhel5").implies(os_flavor == "none") &&
(os_version == "rhel6").implies(os_flavor == "none")) &&
!(os_version == "fedora20" && os_flavor == "none") &&
!(os_version == "fedora21" && os_flavor == "none")
Which is still relatively simple example, but it grows dramatically with inner-dependencies.
MuxPlugin¶
avocado_varianter_yaml_to_mux.mux.MuxPlugin
Defines the full interface required by
avocado.core.plugin_interfaces.Varianter
. The plugin writer
should inherit from this MuxPlugin
, then from the Varianter
and call the:
self.initialize_mux(root, paths, debug)
Where:
root - is the root of your params tree (compound of TreeNode -like nodes)
paths - is the Parameter Paths to be used in test with all variants
debug - whether to use debug mode (requires the passed tree to be compound of
TreeNodeDebug
-like nodes which stores the origin of the variant/value/environment as the value for listing purposes and is __NOT__ intended for test execution.
This method must be called before the Varianter’s second stage. The MuxPlugin’s code will take care of the rest.
MuxTree¶
This is the core feature where the hard work happens. It walks the tree
and remembers all leaf nodes or uses list of MuxTrees
when another
multiplex domain is reached while searching for a leaf.
When it’s asked to report variants, it combines one variant of each
remembered item (leaf node always stays the same, but MuxTree
circles
through it’s values) which recursively produces all possible variants
of different multiplex domains.