szn

Bug 542 — Unbounded Session.arena growth via non-LIFO allocations and frees

542. Non-LIFO frees against Session.arena silently fail to reclaim memory

Severity: CRITICAL
Status: ✅ RESOLVED
Files: src/session.zig:31-62, src/grid.zig:976-1163, src/input.zig:610-613, src/screen.zig:1722-1760, src/window.zig:394-427, src/options.zig:95-118, src/server/server.zig:3298-3310

Problem

Session.init (src/session.zig:39) creates var arena = std.heap.ArenaAllocator.init(backing_allocator) and passes self.arena.allocator() down into Window, Layout, Pane, Screen, Grid, and Options. In Zig’s std.heap.ArenaAllocator, .free() and .resize() only reclaim memory when acting on the exact most-recent bump allocation. Any allocation performed before freeing an older buffer permanently pins the older buffer in session.arena until the entire session is destroyed:

  1. Grid.reflowCursorInternal & Grid.rewrap (src/grid.zig:976-1163) — ~5–10 MB leaked per width resize or copy-mode entry:
    • Triggered on every width resize and every Pane.enterCopyMode (forceReflow()).
    • var scratch_arena = std.heap.ArenaAllocator.init(self.allocator); defer scratch_arena.deinit(); uses self.allocator (session.arena) as its backing allocator.
    • rewrap allocates temporary line_cells (defer line_cells.deinit(allocator)) before gl.cells = try allocator.alloc(Cell, ...) (src/grid.zig:876), so line_cells.deinit is never the last allocation and leaks on every logical line.
    • new_lines, vis_list, and hist_list are allocated on self.allocator before old_lines.deinit(allocator), old_history.deinit(allocator) (src/grid.zig:1066-1074), and defer scratch_arena.deinit(). Because vis_list / hist_list are the newest allocations in session.arena, 100% of old_lines, old_history, line_cells, and scratch_arena fail to free.
  2. Scrollback ring eviction (Grid.scrollUp, src/grid.zig:685-715):
    • When history reaches history_limit and evicts the oldest line via self.deinitLine(&evicted) after newer lines have been allocated, ArenaAllocator.free is a no-op.
  3. Sixel DCS buffers & evicted sixel slots (src/input.zig:610-613, src/screen.zig:335-338):
    • InputParser.dispatchDcsSixel executes const owned = try self.screen.allocator.dupe(u8, self.dcs_buf.items); before self.dcs_buf.deinit(self.screen.allocator);. Because owned is allocated after dcs_buf, dcs_buf.deinit is a guaranteed no-op in session.arena (leaking up to 8 MiB per sixel >64 KiB). Evicted SixelImage.data slices in Screen.addSixelImage similarly cannot be reclaimed once any later allocation occurs.
  4. Alt-screen toggle (Screen.useAltScreen, src/screen.zig:1722-1760):
    • Entering the alternate screen (?1049h) allocates a full Grid on self.allocator (session.arena), and leaving (?1049l) calls saved.deinit(), which cannot reclaim memory if anything allocated in session.arena while the alt-screen app ran. Additionally, useAltScreen calls Grid.init (history_limit = 2000) instead of Grid.initWithHistory(..., 0) for the alt screen.
  5. Per-pane border_format_cached & Window.resizeNode stack (src/server/server.zig:3298-3310, src/window.zig:394-424):
    • renderClients allocates the new expanded string on pb.pane.screen.grid.allocator (session.arena) before freeing old_exp.
    • Window.resizeNode allocates stack on self.allocator before calling pane.resizeTerminal (which allocates in self.allocator), so defer stack.deinit(self.allocator) never reclaims stack.
  6. Options.set / Options.unset (src/options.zig:95-118):
    • Allocates cloned before calling self.freeValue(entry.value) on session.arena. Also cloneValue/freeValue unnecessarily heap-dupes .choice strings even though all choices are static string literals in OptionDef.choices.
  7. Session.killWindow (src/session.zig:123-142):
    • Calls self.windows.orderedRemove(idx) without calling win.deinit(self.arena.allocator()).

Proposed Fix

  1. Back dynamic, frequently-reallocated structures (Grid lines/history, SixelImage.data, InputParser.dcs_buf, border_format_cached, Options strings) with the session’s backing_allocator (or a dedicated general-purpose / slab allocator torn down on Pane.deinit / Session.deinit) rather than the bump-only Session.arena, or give each Pane its own ArenaAllocator that is swapped/reset on full-grid rebuilds (reflowCursorInternal) and destroyed on Pane.deinit / Session.killWindow.
  2. In Grid.reflowCursorInternal, back scratch_arena with std.heap.page_allocator (or the backing GPA) and allocate line_cells inside scratch_arena rather than self.allocator.
  3. In InputParser.dispatchDcsSixel, transfer ownership with try self.dcs_buf.toOwnedSlice(self.screen.allocator) instead of dupe + deinit.
  4. In Screen.useAltScreen, initialize the alternate grid with Grid.initWithHistory(self.allocator, self.grid.width, self.grid.height, 0) (or reuse a persistent alt grid).
  5. In Session.killWindow, call self.windows.items[idx].deinit(self.arena.allocator()) before orderedRemove(idx).

Why This Was Actually Easy to Fix (Resolution)

Why reviewers thought it was hard

Static reviewers saw two constraints that appeared to be in irreconcilable conflict:

  1. Session.init wrapped backing in a single bump-pointer std.heap.ArenaAllocator (self.arena, added for Bug #245) and passed self.arena.allocator() down to Options, Window, Layout, Pane, Screen, and Grid.
  2. To preserve OOM rollback invariants (Bugs #217, #230, #528), functions like Grid.reflowCursorInternal, Options.set, and Screen.useAltScreen must allocate new state before freeing old state (alloc(new) $\rightarrow$ free(old)).

In a single bump-pointer std.heap.ArenaAllocator, .free(slice) is a no-op unless slice is the exact most-recent bump allocation — making alloc(new)-before-free(old) and memory reclamation mathematically impossible in the same bump arena. Assuming everything had to stay inside std.heap.ArenaAllocator led reviewers to conclude that fixing #542 required double-buffered swap arenas inside Grid, custom row free-lists for scrollback eviction, and cross-window pane ownership refactoring.

Why it was actually easy (~35 lines of code)

Every subsystem below Session (Grid, Screen, InputParser, Options, Layout, Window, Pane) was already written and unit-tested against std.testing.allocator (a general-purpose leak-detecting allocator) with complete .deinit() / .free() / .destroy() lifecycle management. Wrapping backing in Session.arena had never simplified any child subsystem — they all still called .free() and .deinit() — it had merely turned all of their non-LIFO .free() calls into silent no-ops at runtime while masking a handful of missing top-level deinit/destroy calls in Session and command handlers.

Changes Made

  1. Session allocator routing & lifecycle (src/session.zig, src/cmd/cmd.zig):
    • Removed the vestigial Session.arena / Session.arenaAllocator() wrapper and routed Session.init, Session.newWindow, cmdMoveWindow, and cmdJoinPane directly through Session.allocator (backing), while initializing Session.name in self.name_buf (matching Window.init) so the initial name is never leaked on Session.rename.
    • Updated Session.deinit to call win.deinit(self.allocator), self.allocator.destroy(win), self.windows.deinit(self.allocator), self.options.deinit(), and self.window_options.deinit().
    • Updated Session.killWindow to call win.deinit(self.allocator) and self.allocator.destroy(win) (which via Layout.deinitNode deinits and destroys every leaf *Pane in the window).
  2. Pane struct destruction on removal/replacement (src/server/server.zig, src/cmd/cmd.zig, src/window.zig):
    • Added win.allocator.destroy(pane) after win.removePane + pane.deinit() in Server.destroyPane and undoSplit.
    • Added destroy(old_pane) in cmdBreakPane and destroy(dummy_pane) in cmdJoinPane (after updating setActivePane so prev.active = false never writes to the destroyed placeholder), and removed the redundant post-killWindow dummy.deinit() since killWindow $\rightarrow$ Layout.deinit now deinits and destroys src_win.layout.root.leaf.
    • Added errdefer window.panes.deinit(allocator) in Window.init if Layout.init fails.
  3. Eliminated redundant allocations & intermediate leaks (src/grid.zig, src/input.zig, src/screen.zig, src/options.zig):
    • Backed scratch_arena in Grid.reflowCursorInternal with std.heap.page_allocator, and transferred ownership of line_cells directly into GridLine.cells in Grid.rewrap (const gl = GridLine{ .cells = line_cells, ... }; line_cells = .empty;) instead of duplicating via appendSlice.
    • Used self.dcs_buf.toOwnedSlice(self.screen.allocator) in InputParser.dispatchDcsSixel.
    • Initialized the alternate screen grid with Grid.initWithHistory(self.allocator, ..., 0) (history_limit = 0) in Screen.useAltScreen.
    • Transferred ownership of coerced .string slices directly in Options.set (coerced_owns_string) instead of double-duping via cloneValue.
  4. Regression test (src/session.zig):
    • Added "session reclaims memory across repeated reflow, alt-screen, option, and window-kill cycles — bug #542", verifying with FailingAllocator byte accounting (allocated_bytes - freed_bytes) that 10 consecutive cycles of newWindow + killWindow, width-reflow Session.resize, useAltScreen toggles, and Options.set/unset produce zero net memory growth.