Comparison with PhyLiNC, PhyloDAG, SNAQ, PhyloNET MPL, and PhyloNET ML on simulated data

[lutteropp]

The complete PhyLiNC output for a simulated 10 taxon 1 reticulation 2000 MSA sites dataset, on the PhD laptop. I set the maximum number of reticulations it should try to 2, and turns out that PhyLiNC overshooted and inferred a 2-reticulation network then. As they use the unlinked sites original NEPAL likelihood model, this is expected. We had the same problem with that model back then.
Also, PhyLiNC had some more issues and errors down the line.
phylinc_output.txt

[lutteropp]

The simulated dataset, the RAxML-NG best ML tree, the PhyLINC inferred network, and networks inferrred by several NetRAX variants
datasets_phylinc_exp_smaller.zip

[lutteropp]

PhyLINC result on the PhD laptop, with max_reticulations set to 2, start from RAxML-NG best ML tree:
Total inference runtime: 38365.49 seconds.
Inferred a network with 2 reticulations.
Printed multiple error messages (ERROR found on PhyLiNC for run 5 seed 17293:
│ RootMismatch: non-leaf node 22 had 0 children.
│ Could be a hybrid whose parents' direction conflicts with the root.
│ isChild1 and containRoot were updated for a subset of edges in the network only.)

---

NetRAX results on the PhD laptop for the simulated 10-taxon 1-reticulatiion dataset:

• Start from RAxML-NG best ML tree, with LikelihoodModel.AVERAGE:
  Total inference runtime: 3.0 seconds.
  Best inferred network has 1 reticulations, logl = -15123.43304, bic = 30643.00558
  0_0_single_average_result.txt

• Start from all unique trees with 10 random and 10 parsimony, with LikelihoodModel.AVERAGE:
  11 unique start tree topologies.
  Total inference runtime: 40.0 seconds.
  Best inferred network has 1 reticulations, logl = -15123.32165, bic = 30642.7828, logl = -727209.282, bic = 1499403.751
  0_0_multi_average_result.txt

• Start from RAxML-NG best ML tree, with LikelihoodModel.BEST
  Total inference runtime: 2.0 seconds.
  Best inferred network has 1 reticulations, logl = -15123.43304, bic = 30643.00558
  0_0_single_best_result.txt

• Start from all unique trees with 10 random and 10 parsimony, with LikelihoodModel.BEST:
  11 unique start tree topologies.
  Total inference runtime: 28.0 seconds.
  Best inferred network has 1 reticulations, logl = -15123.32165, bic = 30642.7828
  0_0_multi_best_result.txt

[lutteropp]

I am also including PhyloDAG in this comparison. Here the data to run PhyloDAG on the dataset:
data_for_phylodag.zip

[lutteropp]

The PhyloDAG inference already finished. It took 3.308089 mins, ran only single-threaded, and inferred this network, with 1 reticulation and loglikelihood -17771.85:

[lutteropp]

We need to also compare NetRAX and PhyloDAG on a larger dataset. Let's say 30 taxa, 3 reticulations. I am using the dataset from experiment D (the scrambling one) for it.

[lutteropp]

In this archive, we have:

• 0_0.nex: The input file for PhyloDAG for the 10 taxa 1 reticulation dataset
• D.nex: The input file for PhyloDAG for the 30 taxa 3 reticulations dataset
• simulated_0_0.R: The R script calling PhyloDAG on the 10 taxa 1 reticulation dataset
• D.R: The R script calling PhyloDAG on the 30 taxa 3 reticulations dataset

data_for_phylodag_2.zip

[lutteropp]

I aborted the 30 taxa 3 reticulations run on PhyloDAG since it kept running for ages. Trying with a newly simulated 20 taxa 2 reticulations 4k MSA sites dataset now:

phylodag_data_20t2r.zip

[lutteropp]

Very interesting! PhyloDAG on 20 taxa 2 reticulations dataset finished, and it's result sucks really hard:
Total runtime: 20.42073 mins

Inferred network picture:

[lutteropp]

NetRAX results on the PhD laptop for the simulated 20-taxon 2-reticulatiion dataset:

• Start from RAxML-NG best ML tree, with LikelihoodModel.AVERAGE:
  Total inference runtime: 63.0 seconds.
  Best inferred network has 2 reticulations, logl = -47915.4701, bic = 96756.70232
  20t_2r_single_average_result.txt

• Start from all unique trees with 10 random and 10 parsimony, with LikelihoodModel.AVERAGE:
  16 unique start tree topologies.
  Total inference runtime: 1475.0 seconds.
  Best inferred network has 2 reticulations, logl = -47914.83028, bic = 96755.42267
  20t_2r_multi_average_result.txt

• Start from RAxML-NG best ML tree, with LikelihoodModel.BEST
  Total inference runtime: 51.0 seconds.
  Best inferred network has 2 reticulations, logl = -47915.54511, bic = 96756.85233
  20t_2r_single_best_result.txt

• Start from all unique trees with 10 random and 10 parsimony, with LikelihoodModel.BEST:
  16 unique start tree topologies.
  Total inference runtime: 1031.0 seconds.
  Best inferred network has 2 reticulations, logl = -47914.7023, bic = 96755.16672
  20t_2r_multi_best_result.txt

[lutteropp]

I retried PhyloDAG with their default parameters (before I used the parameters stated in their example file). This time, I got:

• For the 10 taxa, 1 reticulation, 2k MSA sites simulated dataset:
  Total runtime: 2.415105 mins
  Loglikelihood: -15794.61
  

• For the 20 taxa, 2 reticulations, 4k MSA sites simulated dataset:
  Total runtime: 16.88632 mins
  Loglikelihood: -60021.8
  

[lutteropp]

RAxML-NG best tree ML inference runtime (starting from 10 random + 10 parsimony trees) on the PhD laptop was:

• for 10 taxa, 1 reticulation: 5.688 seconds
• for 20 taxa, 2 reticulations: 49.811 seconds

[lutteropp]

I hand-wrote the Extended NEWICK for the PhyloDAG network, for the 10 taxa 1 reticulation dataset, using its default parameters:
phylodag_10t1r_inferred_network.txt

[lutteropp]

Re-running PhyloDAG with the same parameters gives me totally different networks every time.