50 players closed alpha playtesting dev blog cover with session data visualization
Back to Dev Blog
lumenfall playtesting alpha studio

50 Players, 200 Hours: What Our Closed Alpha Taught Us

From November 2025 through March 2026 we ran a 50-player closed alpha. We sent invites to people who had signed up on the mailing list and a handful of players from communities that engage seriously with open-world games. No press, no influencers, just people willing to file bug reports and tolerate rough edges. Here is what they taught us.

What We Were Trying to Learn

Before the alpha started, we wrote down the three questions we most needed answered. Knowing what you are trying to learn sounds obvious but it shapes everything: which metrics you instrument, what you ask players to do, and how you interpret surprising results.

Question one: were players experiencing the world as genuinely different between runs, or were they mentally mapping equivalent structures and seeing through the surface variation? We had confident internal opinions about this but no external verification.

Question two: where in the terrain traversal did players stop moving? Exploration games have invisible walls that are not physical barriers. They are places where the world stops offering forward pull. Finding those friction points before ship is the entire point of a closed alpha.

Question three: how long before NPC interaction became repetitive? The character synthesis system was new. We expected it to hold up better than a scripted roster. We did not know by how much.

The Geometry Variation Finding

We instrumented a simple tile-visit tracker that logged every terrain zone a player entered, tagged to the world seed. After 10-hour sessions, the median player had entered 214 distinct zone configurations. The minimum across all participants was 187. That answered question one clearly: players were not cycling through a small effective vocabulary of terrain experiences. The variation is real and players are encountering it.

What we did not expect was the distribution of zone entry time. Players spent roughly three times as long in transition zones, the border regions between biome types, as in zone interiors. The transition zones in earlier builds were seam-mitigation measures, not primary content. Players had decided otherwise. They were treating the border crossings as the most interesting terrain to explore because that is where the geological variety concentrated. That realization directly shaped the Alpha 04 investment in the border negotiation system.

Where Players Stopped Moving

Four friction locations emerged clearly from the session data. We found them by looking at geographic clusters of session-end events and long stationary periods in the movement logs.

The first was elevation transitions. Specifically, the generation grammar produced cliff-like inclines that had no navigable path. Players would reach a high terrain feature, identify something interesting above or below, and then spend time searching for a traversal route that did not exist. Several participants noted this explicitly in their feedback. The grammar was generating dramatic terrain without generating the traversal affordances to match.

The second friction location was what we internally called the fog wall problem. Long-distance visibility renders correctly in Lumenfall, which means players can see interesting terrain features at significant distance. When they navigate toward those features and find a fog transition boundary mid-path, something feels broken even though the boundary is technically correct. Distance and accessibility were decoupled in a way the player could not read in advance.

The third was NPC density. Several large interior zones generated with very sparse NPC placement. Players moving through those zones described them as empty in a negative sense. Not serene. Just absent.

The fourth surprised us. Some players stopped moving voluntarily and stayed in a single zone for extended periods. They had found terrain that held their attention. That is the good case. But it meant our aggregate movement data looked alarming until we separated intentional dwelling from friction stops. Reading session telemetry requires context that the numbers alone do not supply.

NPC Interaction Depth

Question three got the most mixed results. The character synthesis system held up in terms of surface variation: players rarely described two NPCs as having the same personality. What they did describe was a shallowness cliff. Initial interactions felt fresh. By the third or fourth exchange with the same NPC, the generative limits became apparent. The NPC had a consistent personality but not a history. It could generate contextually relevant responses but could not accumulate the conversation into a coherent relationship arc.

We want to be clear about what this is and what it is not. The synthesis system is working as designed. It generates personality-consistent behavior from trait pools. What it does not have is memory architecture that lets an NPC treat each interaction as part of a continuing relationship. That is a scope problem, not a generation quality problem. Players interacted with the system long enough to find its current design boundary, which is exactly what you want from a closed alpha.

Tariq has been scoping a lightweight conversation history buffer since the alpha closed. The goal is not full persistent memory, which would require a data architecture we do not have budget to build right now. The goal is a short interaction log that lets the NPC reference the last two or three conversations and adapt its opening approach. Even a shallow continuity signal should meaningfully change how relationships feel.

Feedback Categories We Did Not Expect

Two themes appeared in participant written feedback that we had not instrumentally prepared for.

The first was a strong positive reaction to weather. Lumenfall has a procedurally generated weather layer that was mostly a visual atmospheric feature in our thinking. Players wrote about weather as a navigation and mood reference. They oriented using weather patterns. Several described enjoying how a coming storm felt like it was announcing itself before it arrived visually. That is not a feature we built intentionally; it is a property of having a coherent weather simulation that players found meaningful.

The second theme was a request for more ways to mark the world. Several participants wanted to leave some kind of trace in a zone: not a waypoint marker in a traditional game sense, but something that the world acknowledged. We do not have a response to this yet. It is sitting in our design backlog as a question about whether persistent player-world interaction belongs in Lumenfall's model. The game is specifically about worlds that generate fresh. Player-authored permanence is in tension with that. We are not saying no to it. We are saying we do not know what the right version of yes looks like.

What Changed in the Build After Alpha

The terrain traversal fix shipped first because it had a direct cause. We modified the grammar to require passable routes within a configurable distance of any elevation feature above a height threshold. The constraint is simple, and it works: playtest build 037 produced no elevation-blocked deadends in 15 hours of internal testing.

The fog wall issue is deeper. The render budget for loading adjacent zones at distance is a performance question as much as a design question. We changed the fog boundary to a graduated visibility fade rather than a sharp cutoff, which helps. But the real solution is loading adjacent terrain before a player reaches the boundary, which requires smarter lookahead in the zone resolver. That is on the Alpha 05 scope.

NPC density is now governed by a zone population floor. Any zone above a minimum area must contain at least two NPC slots. The grammar fills those slots with placement-appropriate character archetypes based on the zone type. The change is conservative but effective.

Thank you to everyone who participated in the closed alpha. The feedback was thorough and direct, exactly what we needed. The next alpha round will be announced through the mailing list when Alpha 05 is far enough along to be worth testing.

More from the Dev Blog

All posts
Alpha 04 terrain synthesis blog cover

Alpha 04: Terrain Synthesis Gets Smarter

Read post
LLMs for NPC personality generation blog cover

Using Language Models to Generate NPC Personality Profiles

Read post
Angel funding blog cover

Angel-Backed: What Changes and What Stays the Same

Read post