feat: allow lazy loading via vim.g.NvimTreeConfig

[perrin4869]

I was trying to improve my neovim startup time by lazy loading dependencies here, and nvim-tree was a good candidate to lazy load.
I noticed that it would be easy to implement this use case by adding the simple setup call in the wrap function.
It seems to work for my usecases but I did not do extensive testing

[alex-courtis]

NvimTree aims for ~0 startup cost and ~0 setup cost, see wiki

Do you have any new information on NvimTree startup or setup times? Has a regression been introduced? The preferable course of action is to resolve any problems as they will affect all users.

RE the vim.g.NvimTreeConfig pattern. Is there any prior art in other plugins? A quick look through my installed plugins doesn't reveal anything.

[perrin4869]

There hasn't been any regression that I am aware of, I was just looking at my neovim startup times and nvim-tree was one of the chunkier ones. Just looking for those last 10% optimizations

render-markdown.nvim offers something close to what I am doing here:https://github.com/MeanderingProgrammer/render-markdown.nvim/blob/main/plugin%2Frender-markdown.lua#L7

[alex-courtis]

There hasn't been any regression that I am aware of, I was just looking at my neovim startup times and nvim-tree was one of the chunkier ones. Just looking for those last 10% optimizations

That's no good... we need to fix that. What are the timings?

[gegoune]

I am somewhat against special casing as such. Let's fix underlying issue if there is one instead, benefitting everyone.

[perrin4869]

Here is the relevant section from the output of vim --startuptime:

050.600  000.048  000.048: require('nvim-tree.notify')
050.605  000.097  000.049: require('nvim-tree.events')
050.794  000.114  000.114: require('nvim-tree.utils')
050.839  000.044  000.044: require('nvim-tree.log')
050.854  000.249  000.091: require('nvim-tree.view')
050.855  000.410  000.064: require('nvim-tree.core')
051.054  000.052  000.052: require('nvim-tree.git.utils')
051.081  000.026  000.026: require('nvim-tree.renderer.components.devicons')
051.151  000.037  000.037: require('nvim-tree.classic')
051.154  000.072  000.036: require('nvim-tree.node')
051.174  000.230  000.080: require('nvim-tree.node.directory')
051.207  000.032  000.032: require('nvim-tree.iterators.node-iterator')
051.207  000.294  000.032: require('nvim-tree.actions.finders.find-file')
051.244  000.036  000.036: require('nvim-tree.actions.finders.search-node')
051.245  000.360  000.030: require('nvim-tree.actions.finders')
051.355  000.040  000.040: require('nvim-tree.node.file')
051.356  000.073  000.033: require('nvim-tree.actions.fs.create-file')
051.439  000.046  000.046: require('nvim-tree.lib')
051.502  000.031  000.031: require('nvim-tree.node.link')
051.505  000.065  000.034: require('nvim-tree.node.directory-link')
051.507  000.151  000.040: require('nvim-tree.actions.fs.remove-file')
051.563  000.056  000.056: require('nvim-tree.actions.fs.rename-file')
051.605  000.041  000.041: require('nvim-tree.actions.fs.trash')
051.607  000.362  000.041: require('nvim-tree.actions.fs')
051.727  000.060  000.060: require('nvim-tree.diagnostics')
051.730  000.093  000.034: require('nvim-tree.actions.moves.item')
051.769  000.039  000.039: require('nvim-tree.actions.moves.parent')
051.808  000.038  000.038: require('nvim-tree.actions.moves.sibling')
051.809  000.202  000.032: require('nvim-tree.actions.moves')
051.882  000.036  000.036: require('nvim-tree.actions.node.file-popup')
051.997  000.059  000.059: require('nvim-tree.renderer.components.full-name')
052.004  000.122  000.063: require('nvim-tree.actions.node.open-file')
052.036  000.032  000.032: require('nvim-tree.actions.node.run-command')
052.077  000.041  000.041: require('nvim-tree.actions.node.system-open')
052.110  000.032  000.032: require('nvim-tree.actions.node.buffer')
052.111  000.301  000.039: require('nvim-tree.actions.node')
052.178  000.039  000.039: require('nvim-tree.actions.root.change-dir')
052.205  000.027  000.027: require('nvim-tree.actions.root.dir-up')
052.206  000.094  000.028: require('nvim-tree.actions.root')
052.267  000.031  000.031: require('nvim-tree.actions.tree.find-file')
052.328  000.033  000.033: require('nvim-tree.actions.tree.modifiers.collapse')
052.375  000.046  000.046: require('nvim-tree.actions.tree.modifiers.expand')
052.375  000.108  000.029: require('nvim-tree.actions.tree.modifiers')
052.406  000.030  000.030: require('nvim-tree.actions.tree.open')
052.452  000.045  000.045: require('nvim-tree.actions.tree.toggle')
052.483  000.031  000.031: require('nvim-tree.actions.tree.resize')
052.484  000.277  000.032: require('nvim-tree.actions.tree')
052.485  001.629  000.032: require('nvim-tree.actions')
052.619  000.100  000.100: require('nvim-tree.appearance')
052.638  000.153  000.053: require('nvim-tree.appearance.hi-test')
052.724  000.045  000.045: require('nvim-tree.keymap')
052.729  000.090  000.045: require('nvim-tree.help')
052.759  000.029  000.029: require('nvim-tree.node.file-link')
052.881  000.122  000.122: require('nvim-tree.node.root')
053.019  000.082  000.082: require('nvim-tree.renderer.decorator')
053.020  000.138  000.057: require('nvim-tree.renderer.decorator.user')
053.144  002.777  000.206: require('nvim-tree.api')
053.576  000.431  000.431: require('nvim-tree')
053.661  000.076  000.076: require('nvim-tree.legacy')
053.723  000.007  000.007: sourcing nvim_exec2() called at /home/perrin4869/dotfiles/home/.config/nvim/plugin/tree.lua:0
053.734  000.002  000.002: sourcing nvim_exec2() called at /home/perrin4869/dotfiles/home/.config/nvim/plugin/tree.lua:0
054.693  000.335  000.335: require('nvim-tree.buffers')
054.946  000.159  000.159: require('nvim-tree.git.runner')
055.076  000.129  000.129: require('nvim-tree.watcher')
055.082  000.388  000.100: require('nvim-tree.git')
055.119  000.036  000.036: require('nvim-tree.node.factory')
055.211  000.016  000.016: require('nvim-tree.enum')
055.217  000.097  000.081: require('nvim-tree.explorer.filters')
055.247  000.029  000.029: require('nvim-tree.marks')
055.274  000.027  000.027: require('nvim-tree.explorer.live-filter')
055.301  000.027  000.027: require('nvim-tree.explorer.sorter')
055.330  000.028  000.028: require('nvim-tree.actions.fs.clipboard')
055.392  000.019  000.019: require('nvim-tree.renderer.decorator.bookmarks')
055.409  000.017  000.017: require('nvim-tree.renderer.decorator.copied')
055.426  000.016  000.016: require('nvim-tree.renderer.decorator.cut')
055.445  000.019  000.019: require('nvim-tree.renderer.decorator.diagnostics')
055.467  000.022  000.022: require('nvim-tree.renderer.decorator.git')
055.484  000.016  000.016: require('nvim-tree.renderer.decorator.hidden')
055.510  000.026  000.026: require('nvim-tree.renderer.decorator.modified')
055.528  000.017  000.017: require('nvim-tree.renderer.decorator.opened')
055.551  000.023  000.023: require('nvim-tree.renderer.components.padding')
055.566  000.216  000.042: require('nvim-tree.renderer.builder')
055.569  000.239  000.023: require('nvim-tree.renderer')
055.574  001.434  000.228: require('nvim-tree.explorer')
055.593  000.019  000.019: require('nvim-tree.explorer.watch')
055.615  000.018  000.018: require('nvim-tree.renderer.components')

So about 5msec. Complete load time is about 140msec, so about 3.5%. With the patch in this PR all those calls are deferred. Since I don't always use nvim-tree in my workflow (only when I want to visualize the project in a tree, which depends on the project), this is a net benefit.

NvimTree aims for ~0 startup cost and ~0 setup cost

Well, in this case I think the pattern in this PR should be encouraged, because it is going to be faster in all cases. Even if setup is optimized to not load all the files above, it is always going to need to load at the very least the lua file where setup is defined, which will always be less efficient than not loading it.

Somewhat relevant to this PR, I remembered this article I came across a few years ago: https://mrcjkb.dev/posts/2023-08-22-setup.html

[alex-courtis]

Fast PC:

: ; nvim --startuptime /tmp/startuptime
: ; grep "require('nvim-tree\." /tmp/startuptime | wc -l
84
: ; paste -sd+ <(grep "require('nvim-tree\." /tmp/startuptime | cut -d ' ' -f 5 | sed -E 's/:$//g') | bc
7.818

Slow remote server:

: ; paste -sd+ <(grep "require('nvim-tree\." /tmp/startuptime | cut -d ' ' -f 5 | sed -E 's/:$//g') | bc
11.252

7-11 is significant. We are loading almost all modules and classes when executing setup.

Without requiring nvim-tree modules or calling setup, we get just this one require, thanks to the great work of @przepompownia on #3210

: ; grep "require('nvim-tree\." /tmp/startuptime
046.341  000.085  000.085: require('nvim-tree.commands')

There is significant scope to improve this by removing unnecessary requires when executing setup. Removing most requires from nvim-tree.lua and commenting out the require/setup calls results in this minimal startup time when calling setup:

: ; grep "require('nvim-tree\." /tmp/startuptime
008.600  000.064  000.064: require('nvim-tree.log')
008.825  000.224  000.224: require('nvim-tree.utils')
010.094  000.060  000.060: require('nvim-tree.notify')
010.097  000.147  000.087: require('nvim-tree.legacy')
012.854  000.063  000.063: require('nvim-tree.commands')
: ; paste -sd+ <(grep "require('nvim-tree\." /tmp/startuptime | cut -d ' ' -f 5 | sed -E 's/:$//g') | bc
.498

[alex-courtis]

Created

• Performance: Load Only Necessary Modules On setup and requiring API #3231

[przepompownia]

I didn't experinence the case described here (at the startup) because of using a simple custom lazy loader which defers calling setup() while nvim-tree.api is not required.

https://github.com/przepompownia/nvim-arctgx/blob/57112c1b656ae2e2a9a2ef752ce908071443445d/lua/arctgx/pluginConfigs/nvim-tree.lua#L68-L70

[alex-courtis]

I didn't experinence the case described here (at the startup) because of using a simple custom lazy loader which defers calling setup() while nvim-tree.api is not required.

https://github.com/przepompownia/nvim-arctgx/blob/57112c1b656ae2e2a9a2ef752ce908071443445d/lua/arctgx/pluginConfigs/nvim-tree.lua#L68-L70

That's really nice - it sidesteps the module loading for both API and setup calls. Looks like the only module loading you get is plugin/commands, and your recent change made that fast.

[przepompownia]

That's really nice - it sidesteps the module loading for both API and setup calls. Looks like the only module loading you get is plugin/commands, and your recent change made that fast.

That prevents only from loading unnecessary modules (except command defs) before the first use. #3231 can be helpful afterwards.

[alex-courtis]

We have a good understanding of the genuine issue here: there are unnecessary, expensive modules loaded when requiring api and when executing setup. This goes against the ~0 startup cost and ~0 setup cost goal.

We know what needs to be done via #3231 which builds on the work already done via #3210

The vim.g.NvimTreeConfig pattern adds unnecessary complexity and risk. Once the above has been addressed, it won't be necessary as rudimentary lazy loading of modules / calling setup by the user will achieve the goal.

[perrin4869]

@przepompownia interesting, I was working on a very similar approach to your lazy module here, mine is called defer to differentiate from lazy 😅
I didn't know about package.loaders, that's very helpful! I guess the main problem with that approach is that it adds a call to setupAtFirstLoader on every single require call across the whole of neovim right? Do you know if it has a performance hit?

Without hijacking package.loaders, the only way to defer calling setup that I can tell is the approach that I tried to work on on my PR on my dotfiles repo, which I think is similar to what lazy.nvim is doing. Where I got stuck with that approach is that while I can easily defer the call to setup on a keymap call, it's hard to call setup before one of the NvimTree* cmds like NvimTreeOpen, without somehow hijacking all those commands manually... which is when I ended up coming up with this PR here, which solved all the problems.

I know you can optimize setup further, and I'm looking forward to that, but it will still require at least the 5 files that you listed on #3231 right? And there is no way to defer calling that without incurring a performance hit or hardcoding all the NvimTree* cmds? I realize the vim.g.NvimTreeConfig pattern introduces complexity and risk, but I'm not sure it's fair to call it "unnecessary", since it has performance benefits that can't be achieved otherwise (as far as I could tell).

[przepompownia]

I guess the main problem with that approach is that it adds a call to setupAtFirstLoader on every single require call across the whole of neovim right? Do you know if it has a performance hit?

No. Notice the index (2) of package.loaders, where the custom loader is inserted (so not called if the module is already loaded, because the first loader succeeded).

See https://www.lua.org/manual/5.1/manual.html#pdf-package.loaders

[przepompownia]

I didn't know about package.loaders, that's very helpful!

Initially I was interested in using lazy.nvim, but (with all due respect to the author's effort) it is shipped with lots of things I don't need, so I wrote something simple and enough for a few modules.

[perrin4869]

@przepompownia
yeah I like to manage my plugins as git submodules since I started my dotfiles, so I never went the lazy.nvim route either

I just tried to implement a similar functionality to what you have in my repo. My first instinct was to have a table of prefixes and a setup function for each prefix, for example, if the prefix is nvim-tree and modname matches ^${prefix}\.?.* then run the respective setup function, and remove the entry from the table... it ended up not working because nvim-tree.commands is lazy loaded and it matches the prefix, so immediately understood why you were matching against nvim-tree.api.
The reason I'd be reluctant to go this route is that it becomes dependent on nvim-tree internals instead of a public API...
I feel like this PR is the most pragmatic approach to be honest, it's also the way vim has operated since before neovim was ever a thing... all other methods are quite brittle and hacky...

Edit: by this approach being "risky", do you mean as in a security risk, where someone could hijack the config table to add arbitrary code on on_attach? Because if that is the case, no one can stop some other plugin from calling require("nvim-tree").setup({ on_attach = function() ... end }), maybe wrapped in pcall, and run arbitrary code either...