x/tools/gopls: memory/performance regression with eager Cursor construction

[findleyr]

Investigating a user report from slack, I discovered that our eager construction of the Cursor on parsego.File results in around 2-3x the allocation per parsed file, when rediagnosing the workspace.

Needs more profiling: our benchmarks did not show massive regression (10% regression in DiagnoseChange/kubernetes, perhaps).

Repro:

• checkout https://github.com/cockroachdb/cockroach
• open pkg/sql/sem/tree/expr.go, wait for gopls to finish indexing
• make an arbitrary change
• grab a heap profile while gopls is reindexing

Making the Cursor lazy mitigates this regression. I think our rationale for eagerness was that the cursor would be inevitably used, but that's not the case when the parsing is only for type checking.

CC @adonovan

[gabyhelp]

Related Issues

• x/tools/gopls: memory leak in parsing #57222 (closed)
• x/tools/gopls: reduce memory usage #36943 (closed)
• x/tools/gopls: optimize calculation of the workspace symbol index #52602 (closed)
• go/types: audit memory usage for go1.17 #45580 (closed)
• x/tools/gopls: don't pin all source text in memory #45528 (closed)
• x/tools/gopls: high memory consumption in certain workspaces when staticcheck is enabled #61178 (closed)
• x/tools/gopls: inconsistent performance on hashicorp/terraform-provider-aws #60621 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}