For the third test case of Westfall-Young Single Step, we are starting right off with 10 blocks, a harmonic mean of 100, a sample size of 10 instead of 1,000 with the permutation size of 10,000. The power definition is for individual power. We have made the sample size to be 1,00 to see if there is any change in memory usage, elapsed time, iterations etc. The results are looking good with elapsed time being 2.42 minutes, with usage of 2GB and a total of 5 iterations. Westfall-Young SD and Westfall-Young SS has taken the biggest share of time consumption for power calculation overtaking the multtest function for the other MTP adjustments.

For the first test case of Westfall-Young Single Step, we are starting right off with 10 blocks, a harmonic mean of 100, a sample size of 10 instead of 1,000 with the permutation size of 10,000. The power definition is for individual power. We have picked a sample size of 10 as we know it would take approximately 3 minutes for each call of Power on Westfall-Young Single Step with sample size of 1,000. With 10 samples, it took **11** iterations to get a satisfactory MDES which takes around **1.427** minutes. Approximately, **2.096** GB of memory is used up.

For the second test case of Holms, we are starting right off with 10 blocks, a harmonic mean of 100, a sample size of 1,000 with the permutation size of 10,000. The power definition is for 1-minimal power.The total elapsed time is **4.38** minutes and we cannot find an optimal MDES after 45 iterations.

For the second test case of Holms, we are starting right off with 10 blocks, a harmonic mean of 100, a sample size of 1,000 with the permutation size of 10,000. The power definition is for 1-minimal power.The total elapsed time is **41** seconds and we find the optimal MDES within 4 iterations.

For the first test case of Holms, we are starting right off with 10 blocks, a harmonic mean of 100, a sample size of 1,000 with the permutation size of 10,000. The power definition is for individual power. We reach to an optimal MDES in 5 iterations around **37.7** seconds.

In this test case, changed the definition of power from individual to 1-minimal. The time elapsed to get an estimate of the MDES within the range is **37.5** seconds after **5** iterations of binary search.

In test case 7-8-9, we changed the definition of power from min1 (1-minimal power) to complete power. We increased the number of iterations to search for the optimal MDES to a range between 25-45 iterations. My AWS instance ran out of memory at around 40 iterations but clearing the rest of open programs provide more RAM. The currently available RAM for computation is around 4GB. A search for an optimal MDES at complete power is unsuccessful up to 45 iterations in both BF & BH.

In test case 7-8-9, we changed the definition of power from min1 (1-minimal power) to complete power. We increased the number of iterations to search for the optimal MDES to a range between 25-45 iterations. My AWS instance ran out of memory at around 40 iterations but clearing the rest of open programs provide more RAM. The currently available RAM for computation is around 4GB. A search for an optimal MDES at complete power is unsuccessful up to 45 iterations in both BF & BH.

In test case 7-8-9, we changed the definition of power from min1 (1-minimal power) to complete power. We increased the number of iterations to search for the optimal MDES to a range between 25-45 iterations. My AWS instance ran out of memory at around 40 iterations but clearing the rest of open programs provide more RAM. The currently available RAM for computation is around 4GB. A search for an optimal MDES at complete power is unsuccessful up to 45 iterations in both BF & BH.

Here compared to test 4, we increase the actual impact of MDES to 10 from 3. The time elapsed actually goes back down by 1.5 seconds to **28.5** seconds.

Here compared to test 3, we decreased the statistical power to **0.5** from **0.75**. The time elapsed again inched up by approximately **1** seconds to **30** seconds.

Here compared to test case 2, we increase the number of samples to **1000**. The time elapsed increases by barely **1** second to **29.5** seconds.

Here we increase the harmoninc mean to 100. The elapsed time for computing MDES increases by 5 seconds to approximately **28** seconds.

In test case 1 for MDES of Benjamini Hocheberg, we have the baseline parameters of 5 outcomes, 10 blocks, a harmonic mean of 10. We have set the statistical power to be at 0.75 with the power definition set. The total computation time turns out to be at approximately **21.5** seconds.

In test case 4, we have increased some of the baseline parameters such as M, J and n.j. Additionally, we doubled the number of samples for Westfall-Young from 100 to 200, a doubling. With this, we are seeing a further increase in computation time of up to **37** seconds with a non-linear increase in Westfall-Young procedure of up to **16** seconds each. * M = 10 * MDES = 0.125 * Ai = 3 * J = 50 * n.j = 100 * R2.1 = 0.5 * p = 0.5 * alpha = 0.05 * numCovar.1 = 5 * numCovar.2 = NULL * ICC = NULL * tnum = 10000 * snum = 200 * updateProgress = NULL

Number of clusters is back to 2 but, the sample number for Westfall-Young is doubled further to the minimum a thousand that is required for a good run of Westfall-Young. All the other parameters remain the same. Doubling than the number of sample size leads to more than a doubling of computation time at **3** minutes. * M = 10 * MDES = 0.125 * Ai = 3 * J = 50 * n.j = 100 * R2.1 = 0.5 * p = 0.5 * alpha = 0.05 * numCovar.1 = 5 * numCovar.2 = NULL * ICC = NULL * tnum = 10000 * snum = 1000 * updateProgress = NULL

In test case 6, we change the number of clusters used to run the power calculation from 2 to 4. The goal was to see whether with increased cluster, we see a decline in computational time. On the contrary, we don't. The computational time increases in fact by **2** seconds to **88** seconds. More runs are needed to confirm whether there is a statistical fluke or not. * M = 10 * MDES = 0.125 * Ai = 3 * J = 50 * n.j = 100 * R2.1 = 0.5 * p = 0.5 * alpha = 0.05 * numCovar.1 = 5 * numCovar.2 = NULL * ICC = NULL * tnum = 10000 * snum = 500 * updateProgress = NULL

In test case 5, we have maintained the same parameters as before except the sample for Westfall-Young is increased from 200 to 500. With this, we see a significant increase in computation time of up to **86** seconds. The majority of the time elapsed increase could be contributed as before to Westfall-Young procedures each clocking **40** seconds. * M = 10 * MDES = 0.125 * Ai = 3 * J = 50 * n.j = 100 * R2.1 = 0.5 * p = 0.5 * alpha = 0.05 * numCovar.1 = 5 * numCovar.2 = NULL * ICC = NULL * tnum = 10000 * snum = 500 * updateProgress = NULL

In test case 3, we go back to the baseline conditions for all the parameters except the number of sample as it applies to Westfall-Young. The target number of samples stands at 1,000 but, currently, we have increased it by **10** times to **1,00** from **10**. This leads to a significant increase in the computation time up to **22** seconds from the original **5.4** seconds.We are seeing an increase of about **4** times. The allocation across programming pieces have also shifted towards Westfall-Young procedures taking the balk of it at **10** and **9** seconds each with not much in the multtest procedure. * M = 5 * MDES = 0.125 * Ai = 3 * J = 10 * n.j = 10 * R2.1 = 0.5 * p = 0.5 * alpha = 0.05 * numCovar.1 = 5 * numCovar.2 = NULL * ICC = NULL * tnum = 10000 * snum = 100 * updateProgress = NULL

Here the **baseline** parameters of the harmonic mean is made 10 times compared to in test case 1. The number of M outcomes is made twice the original set amount of 5, too. The total elapsed time for the computation is ~ 8 seconds. Again, the majority of the time is spent on apply function using multtest's built in function to find adjustments for all the MTPs except Westfall-Young at ~ 3 seconds. We see increases in both Westfall-Young SD and Westfall-Young SS, too with more time allocated towards WYSD. * M = 10 * MDES = 0.125 * Ai = 3 * J = 10 * n.j = 100 * R2.1 = 0.5 * p = 0.5 * alpha = 0.05 * numCovar.1 = 5 * numCovar.2 = NULL * ICC = NULL * tnum = 10000 * snum = 10 * updateProgress = NULL

An initial test case with a **baseline** parameters set as below. The total elapsed time for the computation is 5.4 seconds. The majority of the time is spent on apply function using multtest's built in function to find adjustments for all the MTPs except Westfall-Young at 2.2 seconds. The rest is split in over 1 seconds each with Westfall-Young SD and Westfall-Young Single Step. Sample number for Westfall-Young is set at 10. * M = 5 * MDES = 0.125 * Ai = 3 * J = 10 * n.j = 10 * R2.1 = 0.5 * p = 0.5 * alpha = 0.05 * numCovar.1 = 5 * numCovar.2 = NULL * ICC = NULL * tnum = 10000 * snum = 10 * updateProgress = NULL