Add metric when validator stops validating because of low memory

[amsanghi]

Fixes: NIT-4281

[codecov]

Codecov Report

❌ Patch coverage is 0% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 32.91%. Comparing base (02566fc) to head (fb1ff97).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4196      +/-   ##
==========================================
- Coverage   33.41%   32.91%   -0.51%     
==========================================
  Files         461      461              
  Lines       55901    55907       +6     
==========================================
- Hits        18681    18400     -281     
- Misses      33921    34292     +371     
+ Partials     3299     3215      -84     

[github-actions]

❌ 3 Tests Failed:

Tests completed	Failed	Passed	Skipped
4441	3	4438	0

View the top 3 failed tests by shortest run time

TestEndToEnd_ManyEvilValidators

Stack Traces | -0.000s run time

... [CONTENT TRUNCATED: Keeping last 20 lines]
created by github.com/offchainlabs/nitro/bold/challenge/tracker.(*Tracker).openSubchallenge in goroutine 265550
	/home/runner/work/nitro/nitro/bold/challenge/tracker/tracker.go:830 +0x1575

goroutine 232116 [sleep]:
time.Sleep(0x1dcd6500)
	/opt/hostedtoolcache/go/1.25.5/x64/src/runtime/time.go:363 +0x165
github.com/offchainlabs/nitro/bold/testing/e2e.expectChallengeWinWithAllHonestEssentialEdgesConfirmed.func1(0xc003ad16c0)
	/home/runner/work/nitro/nitro/bold/testing/e2e/expectations.go:111 +0x953
testing.tRunner(0xc003ad16c0, 0xc00808f860)
	/opt/hostedtoolcache/go/1.25.5/x64/src/testing/testing.go:1934 +0xea
created by testing.(*T).Run in goroutine 232115
	/opt/hostedtoolcache/go/1.25.5/x64/src/testing/testing.go:1997 +0x465

goroutine 318318 [select]:
github.com/offchainlabs/nitro/bold/containers/events.(*Subscription[...]).Next(0x1fc3e60, {0x1fc3e60?, 0xc00808f720?})
	/home/runner/work/nitro/nitro/bold/containers/events/producer.go:131 +0xd0
github.com/offchainlabs/nitro/bold/challenge/tracker.(*Tracker).Spawn(0xc00561c960, {0x1fc3e60, 0xc00808f720})
	/home/runner/work/nitro/nitro/bold/challenge/tracker/tracker.go:205 +0x235
created by github.com/offchainlabs/nitro/bold/challenge/tracker.(*Tracker).Act in goroutine 295345
	/home/runner/work/nitro/nitro/bold/challenge/tracker/tracker.go:357 +0x10bf

TestEndToEnd_ManyEvilValidators/honest_essential_edges_confirmed_by_challenge_win

Stack Traces | -0.000s run time

... [CONTENT TRUNCATED: Keeping last 20 lines]
created by github.com/offchainlabs/nitro/bold/challenge/tracker.(*Tracker).openSubchallenge in goroutine 265550
	/home/runner/work/nitro/nitro/bold/challenge/tracker/tracker.go:830 +0x1575

goroutine 232116 [sleep]:
time.Sleep(0x1dcd6500)
	/opt/hostedtoolcache/go/1.25.5/x64/src/runtime/time.go:363 +0x165
github.com/offchainlabs/nitro/bold/testing/e2e.expectChallengeWinWithAllHonestEssentialEdgesConfirmed.func1(0xc003ad16c0)
	/home/runner/work/nitro/nitro/bold/testing/e2e/expectations.go:111 +0x953
testing.tRunner(0xc003ad16c0, 0xc00808f860)
	/opt/hostedtoolcache/go/1.25.5/x64/src/testing/testing.go:1934 +0xea
created by testing.(*T).Run in goroutine 232115
	/opt/hostedtoolcache/go/1.25.5/x64/src/testing/testing.go:1997 +0x465

goroutine 318318 [select]:
github.com/offchainlabs/nitro/bold/containers/events.(*Subscription[...]).Next(0x1fc3e60, {0x1fc3e60?, 0xc00808f720?})
	/home/runner/work/nitro/nitro/bold/containers/events/producer.go:131 +0xd0
github.com/offchainlabs/nitro/bold/challenge/tracker.(*Tracker).Spawn(0xc00561c960, {0x1fc3e60, 0xc00808f720})
	/home/runner/work/nitro/nitro/bold/challenge/tracker/tracker.go:205 +0x235
created by github.com/offchainlabs/nitro/bold/challenge/tracker.(*Tracker).Act in goroutine 295345
	/home/runner/work/nitro/nitro/bold/challenge/tracker/tracker.go:357 +0x10bf

TestAuctioneerFailoverMessageReprocessing

Stack Traces | 19.090s run time

... [CONTENT TRUNCATED: Keeping last 20 lines]
INFO [01-05|15:58:10.706] Stopping work on payload                 id=0x0329d5354c02db53                      reason=delivery
INFO [01-05|15:58:10.707] Imported new potential chain segment     number=23 hash=c87493..26b07e blocks=1 txs=1 mgas=0.092 elapsed="655.989µs" mgasps=140.138  triediffs=58.04KiB triedirty=0.00B
INFO [01-05|15:58:10.707] Chain head was updated                   number=23 hash=c87493..26b07e root=a3617e..0cb73b elapsed="31.669µs"
INFO [01-05|15:58:10.787] Submitted transaction                    hash=0x13f497c774d4191cd71392bdaca163140b09148e6355968c4f545402b5015f7a from=0x988671d86CBa6ca1577E9203F3fecf93Bb4ebA3d nonce=2  recipient=0x0C4291504d344537e071B28695d63De6fb0377FF value=0
INFO [01-05|15:58:10.790] Submitted transaction                    hash=0x9b2154a699d7da664ddba8b93476e60b82e7364f9956f2c765cd1be41da8a64b from=0xbefDE93c91483a016CE8b0dBFcb301cE380FcAcE nonce=2  recipient=0x7cd794fd1EE9E2479f4B4b70e0148fe289226c24 value=0
INFO [01-05|15:58:11.701] Starting work on payload                 id=0x03d25e60470029d5
INFO [01-05|15:58:11.701] Updated payload                          id=0x03d25e60470029d5                      number=24 hash=e7c584..3169fe txs=0 withdrawals=0 gas=0         fees=0            root=edee10..bf62cc elapsed="58.279µs"
INFO [01-05|15:58:11.701] Stopping work on payload                 id=0x03d25e60470029d5                      reason=delivery
INFO [01-05|15:58:11.701] Imported new potential chain segment     number=24 hash=e7c584..3169fe blocks=1 txs=0 mgas=0.000 elapsed="166.963µs" mgasps=0.000    triediffs=37.38KiB triedirty=0.00B
INFO [01-05|15:58:11.701] Chain head was updated                   number=24 hash=e7c584..3169fe root=edee10..bf62cc elapsed="27.832µs"
INFO [01-05|15:58:11.704] Starting work on payload                 id=0x03c46c01aea7fe03
INFO [01-05|15:58:11.704] Updated payload                          id=0x03c46c01aea7fe03                      number=24 hash=c1c2d5..0891bf txs=1 withdrawals=0 gas=91929     fees=9.1929e-08   root=ab02fc..77505f elapsed="415.814µs"
INFO [01-05|15:58:11.704] Stopping work on payload                 id=0x03c46c01aea7fe03                      reason=delivery
INFO [01-05|15:58:11.705] Imported new potential chain segment     number=24 hash=c1c2d5..0891bf blocks=1 txs=1 mgas=0.092 elapsed="522.701µs" mgasps=175.873  triediffs=61.62KiB triedirty=0.00B
INFO [01-05|15:58:11.705] Chain head was updated                   number=24 hash=c1c2d5..0891bf root=ab02fc..77505f elapsed="43.451µs"
INFO [01-05|15:58:11.705] Starting work on payload                 id=0x0347353b1a68cad4
INFO [01-05|15:58:11.706] Updated payload                          id=0x0347353b1a68cad4                      number=24 hash=4296e1..9ffa2b txs=1 withdrawals=0 gas=91929     fees=9.1929e-08   root=9f1466..99f6fa elapsed="486.6µs"
INFO [01-05|15:58:11.706] Stopping work on payload                 id=0x0347353b1a68cad4                      reason=delivery
INFO [01-05|15:58:11.706] Imported new potential chain segment     number=24 hash=4296e1..9ffa2b blocks=1 txs=1 mgas=0.092 elapsed="598.555µs" mgasps=153.585  triediffs=62.09KiB triedirty=0.00B
INFO [01-05|15:58:11.706] Chain head was updated                   number=24 hash=4296e1..9ffa2b root=9f1466..99f6fa elapsed="35.386µs"

📣 Thoughts on this report? Let Codecov know! | Powered by Codecov

[joshuacolvin0]

LGTM