Skip to content

Refactored scanpy.tools._sparse_nanmean to eliminate unnecessary data… #3570

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Reovirus wants to merge 21 commits into
base: main
Choose a base branch
from
+76 −18

[pre-commit.ci] auto fixes from pre-commit.com hooks

7e23b19
Select commit
Loading
Failed to load commit list.
Open

Refactored scanpy.tools._sparse_nanmean to eliminate unnecessary data… #3570

[pre-commit.ci] auto fixes from pre-commit.com hooks
7e23b19
Select commit
Loading
Failed to load commit list.
scverse-benchmark / benchmark succeeded Apr 30, 2025 in 1h 57m 52s

Benchmark

Benchmark run successful

Details

All benchmarks:

Change Before [813608e] After [7e23b19] Ratio Benchmark (Parameter)
738M 737M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('bmmc', 'counts')
738M 738M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('bmmc', 'counts-off-axis')
6.79G 6.79G 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('lung93k', 'counts')
6.79G 6.8G 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('lung93k', 'counts-off-axis')
443M 444M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc3k', 'counts')
443M 442M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc3k', 'counts-off-axis')
270M 269M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc68k_reduced', 'counts')
270M 270M 1.00 preprocessing_counts.FastSuite.peakmem_calculate_qc_metrics('pbmc68k_reduced', 'counts-off-axis')
738M 738M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('bmmc', 'counts')
739M 738M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('bmmc', 'counts-off-axis')
6.79G 6.79G 1.00 preprocessing_counts.FastSuite.peakmem_log1p('lung93k', 'counts')
6.79G 6.79G 1.00 preprocessing_counts.FastSuite.peakmem_log1p('lung93k', 'counts-off-axis')
444M 445M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('pbmc3k', 'counts')
446M 445M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('pbmc3k', 'counts-off-axis')
270M 270M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('pbmc68k_reduced', 'counts')
270M 270M 1.00 preprocessing_counts.FastSuite.peakmem_log1p('pbmc68k_reduced', 'counts-off-axis')
733M 738M 1.01 preprocessing_counts.FastSuite.peakmem_normalize_total('bmmc', 'counts')
733M 738M 1.01 preprocessing_counts.FastSuite.peakmem_normalize_total('bmmc', 'counts-off-axis')
7.63G 7.63G 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('lung93k', 'counts')
7.63G 7.63G 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('lung93k', 'counts-off-axis')
448M 447M 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc3k', 'counts')
443M 448M 1.01 preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc3k', 'counts-off-axis')
266M 270M 1.01 preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc68k_reduced', 'counts')
270M 270M 1.00 preprocessing_counts.FastSuite.peakmem_normalize_total('pbmc68k_reduced', 'counts-off-axis')
21.3±0.5ms 19.9±0.7ms 0.93 preprocessing_counts.FastSuite.time_calculate_qc_metrics('bmmc', 'counts')
13.1±1ms 13.8±2ms 1.05 preprocessing_counts.FastSuite.time_calculate_qc_metrics('bmmc', 'counts-off-axis')
1.73±0s 1.75±0.01s 1.01 preprocessing_counts.FastSuite.time_calculate_qc_metrics('lung93k', 'counts')
1.41±0s 1.42±0.01s 1.01 preprocessing_counts.FastSuite.time_calculate_qc_metrics('lung93k', 'counts-off-axis')
40.7±5ms 36.6±5ms ~0.90 preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc3k', 'counts')
28.7±3ms 34.5±1ms ~1.20 preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc3k', 'counts-off-axis')
5.51±0.04ms 6.54±0.5ms ~1.19 preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc68k_reduced', 'counts')
5.52±0.06ms 5.56±0.05ms 1.01 preprocessing_counts.FastSuite.time_calculate_qc_metrics('pbmc68k_reduced', 'counts-off-axis')
1.59±0.09ms 1.87±0.2ms ~1.18 preprocessing_counts.FastSuite.time_log1p('bmmc', 'counts')
1.77±0.2ms 1.63±0.02ms 0.92 preprocessing_counts.FastSuite.time_log1p('bmmc', 'counts-off-axis')
645±2ms 648±0.2ms 1.01 preprocessing_counts.FastSuite.time_log1p('lung93k', 'counts')
638±8ms 664±20ms 1.04 preprocessing_counts.FastSuite.time_log1p('lung93k', 'counts-off-axis')
10.7±0.1ms 10.5±0.2ms 0.99 preprocessing_counts.FastSuite.time_log1p('pbmc3k', 'counts')
- 10.8±0.2ms 8.80±0.4ms 0.81 preprocessing_counts.FastSuite.time_log1p('pbmc3k', 'counts-off-axis')
412±1μs 440±20μs 1.07 preprocessing_counts.FastSuite.time_log1p('pbmc68k_reduced', 'counts')
402±30μs 413±20μs 1.03 preprocessing_counts.FastSuite.time_log1p('pbmc68k_reduced', 'counts-off-axis')
1.78±0.3ms 1.80±0.03ms 1.01 preprocessing_counts.FastSuite.time_normalize_total('bmmc', 'counts')
5.58±0.5ms 5.77±0.7ms 1.03 preprocessing_counts.FastSuite.time_normalize_total('bmmc', 'counts-off-axis')
1.02±0.01s 1.05±0s 1.02 preprocessing_counts.FastSuite.time_normalize_total('lung93k', 'counts')
1.68±0s 1.66±0.01s 0.99 preprocessing_counts.FastSuite.time_normalize_total('lung93k', 'counts-off-axis')
17.4±0.7ms 16.4±0.4ms 0.94 preprocessing_counts.FastSuite.time_normalize_total('pbmc3k', 'counts')
29.8±0.6ms 29.4±3ms 0.99 preprocessing_counts.FastSuite.time_normalize_total('pbmc3k', 'counts-off-axis')
611±4μs 549±30μs ~0.90 preprocessing_counts.FastSuite.time_normalize_total('pbmc68k_reduced', 'counts')
419±2μs 428±30μs 1.02 preprocessing_counts.FastSuite.time_normalize_total('pbmc68k_reduced', 'counts-off-axis')
477M 473M 0.99 preprocessing_counts.peakmem_filter_cells('pbmc3k', 'counts')
481M 476M 0.99 preprocessing_counts.peakmem_filter_cells('pbmc3k', 'counts-off-axis')
277M 278M 1.00 preprocessing_counts.peakmem_filter_cells('pbmc68k_reduced', 'counts')
278M 278M 1.00 preprocessing_counts.peakmem_filter_cells('pbmc68k_reduced', 'counts-off-axis')
474M 479M 1.01 preprocessing_counts.peakmem_filter_genes('pbmc3k', 'counts')
474M 481M 1.01 preprocessing_counts.peakmem_filter_genes('pbmc3k', 'counts-off-axis')
278M 278M 1.00 preprocessing_counts.peakmem_filter_genes('pbmc68k_reduced', 'counts')
277M 277M 1.00 preprocessing_counts.peakmem_filter_genes('pbmc68k_reduced', 'counts-off-axis')
1.08G 1.08G 1.00 preprocessing_counts.peakmem_scrublet('pbmc3k', 'counts')
1.08G 1.07G 1.00 preprocessing_counts.peakmem_scrublet('pbmc3k', 'counts-off-axis')
397M 396M 1.00 preprocessing_counts.peakmem_scrublet('pbmc68k_reduced', 'counts')
390M 397M 1.02 preprocessing_counts.peakmem_scrublet('pbmc68k_reduced', 'counts-off-axis')
53.5±6ms 48.0±2ms ~0.90 preprocessing_counts.time_filter_cells('pbmc3k', 'counts')
62.7±1ms 51.7±5ms ~0.82 preprocessing_counts.time_filter_cells('pbmc3k', 'counts-off-axis')
11.1±1ms 10.6±1ms 0.96 preprocessing_counts.time_filter_cells('pbmc68k_reduced', 'counts')
9.39±0.8ms 9.54±1ms 1.02 preprocessing_counts.time_filter_cells('pbmc68k_reduced', 'counts-off-axis')
46.5±1ms 44.8±2ms 0.96 preprocessing_counts.time_filter_genes('pbmc3k', 'counts')
- 48.4±3ms 43.0±0.9ms 0.89 preprocessing_counts.time_filter_genes('pbmc3k', 'counts-off-axis')
10.0±2ms 8.18±0.7ms ~0.82 preprocessing_counts.time_filter_genes('pbmc68k_reduced', 'counts')
7.46±1ms 8.95±1ms ~1.20 preprocessing_counts.time_filter_genes('pbmc68k_reduced', 'counts-off-axis')
3.05±0.09s 3.09±0.04s 1.01 preprocessing_counts.time_scrublet('pbmc3k', 'counts')
3.08±0.04s 3.03±0.07s 0.98 preprocessing_counts.time_scrublet('pbmc3k', 'counts-off-axis')
415±90ms 335±100ms ~0.81 preprocessing_counts.time_scrublet('pbmc68k_reduced', 'counts')
314±5ms 328±20ms 1.04 preprocessing_counts.time_scrublet('pbmc68k_reduced', 'counts-off-axis')
738M 733M 0.99 preprocessing_log.FastSuite.peakmem_mean_var('bmmc', 'off-axis')
737M 738M 1.00 preprocessing_log.FastSuite.peakmem_mean_var('bmmc', None)
6.79G 6.79G 1.00 preprocessing_log.FastSuite.peakmem_mean_var('lung93k', 'off-axis')
6.79G 6.79G 1.00 preprocessing_log.FastSuite.peakmem_mean_var('lung93k', None)
447M 441M 0.99 preprocessing_log.FastSuite.peakmem_mean_var('pbmc3k', 'off-axis')
441M 441M 1.00 preprocessing_log.FastSuite.peakmem_mean_var('pbmc3k', None)
266M 270M 1.02 preprocessing_log.FastSuite.peakmem_mean_var('pbmc68k_reduced', 'off-axis')
269M 270M 1.00 preprocessing_log.FastSuite.peakmem_mean_var('pbmc68k_reduced', None)
1.00±0.05ms 608±60μs ~0.60 preprocessing_log.FastSuite.time_mean_var('bmmc', 'off-axis')
6.69±2ms 7.36±1ms 1.10 preprocessing_log.FastSuite.time_mean_var('bmmc', None)
13.0±0.2ms 13.4±0.2ms 1.03 preprocessing_log.FastSuite.time_mean_var('lung93k', 'off-axis')
31.8±0.5ms 33.9±0.1ms 1.07 preprocessing_log.FastSuite.time_mean_var('lung93k', None)
922±100μs 955±20μs 1.04 preprocessing_log.FastSuite.time_mean_var('pbmc3k', 'off-axis')
3.05±0.4ms 3.35±0.2ms ~1.10 preprocessing_log.FastSuite.time_mean_var('pbmc3k', None)
1.17±0.1ms 1.32±0.07ms ~1.13 preprocessing_log.FastSuite.time_mean_var('pbmc68k_reduced', 'off-axis')
1.29±0.03ms 1.22±0.05ms 0.94 preprocessing_log.FastSuite.time_mean_var('pbmc68k_reduced', None)
443M 444M 1.00 preprocessing_log.peakmem_highly_variable_genes('pbmc3k', 'off-axis')
467M 469M 1.00 preprocessing_log.peakmem_highly_variable_genes('pbmc3k', None)
270M 266M 0.98 preprocessing_log.peakmem_highly_variable_genes('pbmc68k_reduced', 'off-axis')
272M 272M 1.00 preprocessing_log.peakmem_highly_variable_genes('pbmc68k_reduced', None)
572M 578M 1.01 preprocessing_log.peakmem_pca('pbmc3k', 'off-axis')
594M 588M 0.99 preprocessing_log.peakmem_pca('pbmc3k', None)
389M 389M 1.00 preprocessing_log.peakmem_pca('pbmc68k_reduced', 'off-axis')
391M 388M 0.99 preprocessing_log.peakmem_pca('pbmc68k_reduced', None)
n/a n/a n/a preprocessing_log.peakmem_regress_out('pbmc3k', 'off-axis')
n/a n/a n/a preprocessing_log.peakmem_regress_out('pbmc3k', None)
315M 319M 1.01 preprocessing_log.peakmem_regress_out('pbmc68k_reduced', 'off-axis')
324M 313M 0.97 preprocessing_log.peakmem_regress_out('pbmc68k_reduced', None)
961M 961M 1.00 preprocessing_log.peakmem_scale('pbmc3k', 'off-axis')
1.15G 1.15G 1.00 preprocessing_log.peakmem_scale('pbmc3k', None)
307M 306M 1.00 preprocessing_log.peakmem_scale('pbmc68k_reduced', 'off-axis')
308M 309M 1.01 preprocessing_log.peakmem_scale('pbmc68k_reduced', None)
21.9±0.05ms 24.8±3ms ~1.13 preprocessing_log.time_highly_variable_genes('pbmc3k', 'off-axis')
29.8±1ms 28.6±2ms 0.96 preprocessing_log.time_highly_variable_genes('pbmc3k', None)
9.16±0.06ms 9.38±0.6ms 1.02 preprocessing_log.time_highly_variable_genes('pbmc68k_reduced', 'off-axis')
10.4±1ms 11.4±0.4ms 1.10 preprocessing_log.time_highly_variable_genes('pbmc68k_reduced', None)
1.92±0.01s 1.92±0.02s 1.00 preprocessing_log.time_pca('pbmc3k', 'off-axis')
2.12±0s 2.12±0.01s 1.00 preprocessing_log.time_pca('pbmc3k', None)
31.3±0.3ms 31.3±0.3ms 1.00 preprocessing_log.time_pca('pbmc68k_reduced', 'off-axis')
35.2±3ms 32.2±0.8ms 0.92 preprocessing_log.time_pca('pbmc68k_reduced', None)
n/a n/a n/a preprocessing_log.time_regress_out('pbmc3k', 'off-axis')
n/a n/a n/a preprocessing_log.time_regress_out('pbmc3k', None)
5.56±0.5ms 5.89±0.6ms 1.06 preprocessing_log.time_regress_out('pbmc68k_reduced', 'off-axis')
5.00±0.5ms 5.05±0.6ms 1.01 preprocessing_log.time_regress_out('pbmc68k_reduced', None)
762±20ms 779±40ms 1.02 preprocessing_log.time_scale('pbmc3k', 'off-axis')
720±10ms 714±20ms 0.99 preprocessing_log.time_scale('pbmc3k', None)
4.30±0.2ms 4.47±0.03ms 1.04 preprocessing_log.time_scale('pbmc68k_reduced', 'off-axis')
4.44±0.6ms 4.50±0.3ms 1.01 preprocessing_log.time_scale('pbmc68k_reduced', None)
263M 259M 0.98 tools.peakmem_diffmap
269M 268M 1.00 tools.peakmem_leiden
328M 326M 0.99 tools.peakmem_rank_genes_groups
+ 261M 308M 1.18 tools.peakmem_score_genes
351M 351M 1.00 tools.peakmem_umap
15.0±0.4ms 15.0±0.8ms 1.00 tools.time_diffmap
14.6±0.06ms 14.5±0.03ms 0.99 tools.time_leiden
55.9±5ms 67.4±4ms ~1.21 tools.time_rank_genes_groups
21.5±0.2ms 20.8±0.7ms 0.97 tools.time_score_genes
1.02±0s 1.01±0s 1.00 tools.time_umap
View more details on scverse-benchmark