Summary

I’m currently building a world management system to handle the critical job of loading world data into the scene for player to actively see, hear, and interact with, then unload again when appropriate. This requires a good understanding of how to store the world and how to reference any part of it using a global coordinate system so it can be loaded or unloaded as needed.

The work of loading/unloading world data and tracking where player is at all times is important to ensure stability across multiple systems and to accurately save and restore your progress. This required substantial time to refine in Daggerfall Unity, and I’ll be taking the lessons I learned there forward into Project Arroa: Revenwood.

In this post, I’ll discuss how I’m planning to structure the world and some of the technical aspects and challenges of building open world games in general.

Divide and Conquer

Rather than a vast and mostly empty procedural overworld, the world of Project Arroa: Revenwood is divided into several distinct handcrafted open world maps. I’m aiming for a sweet spot where each map feels broad with room to explore and discover secrets while still feeling designed and not overtly copy-pasted. This approach also provides several benefits that will ease development through the project’s entire lifecycle.

The most immediate benefit is that I can partition development to fit my time and budget. I will start small while building baseline tools and systems, then expand world as those systems mature and I have more resources. Once I reach a stage where I can employ help with world design, I can have different people iterating over individual maps without conflicting with each other. And it means that modders have free reign to create new areas from whole cloth without risk of breaking story maps or overlapping with other mods.

Dividing world into multiple maps also enables a continuous, incremental release schedule and player feedback loop. I believe this approach is critical to getting the most out of a game on a tight budget. I’ve experienced this first-hand during the development of Daggerfall Unity and watched it succeed with several other games. So once Project Arroa: Revenwood is in a minimal playable state on a single map, I will start scheduling regular releases for the remainder of its lifecycle so the community can actively help the game evolve. The specifics of how this will work are still to be determined, but I wanted to let everyone know my intentions early on.

Map Size in Depth

World maps in Project Arroa: Revenwood are managed internally using cells of 125x125m dimension tiled up to a theoretical maximum of 20x20km (i.e. 400 square kilometres), although in practice a map will never be this large.

What’s with this very specific number for the maximum map size? You may have even noticed a similar maximum world dimension of 20x20km in something like Unreal Engine and wondered about it before. This size comes about due to a hardware limitation of single-precision (32-bit) floating point values still widely used across game engines today. Single-precision floats offer decent accuracy for gaming applications while being less demanding on memory bandwidth and more efficient for CPUs and GPUs, but it places a hard cap on how large a game world can become before precision limits are encountered.

For simplicity, let’s say single-precision values can have 7 digits with the decimal point placed somewhere in there. This means the values 123.4567 and 123456.7 both use 7 digits, but one has more decimal places than the other. If these numbers represent a unit like metres, then the former number is precise down to sub-millimetre resolution of 0.0001 metre increments, but the latter is only accurate down to 10cm resolution of 0.1 metre increments.

More precision means smoother movement and stable physics simulation. Losing precision results in jittery movement and a likely breakdown of simulation so that player falls out of world or clips through walls. Because single-precision numbers can be either large or precise (not both), it places a hard cap on the size of world you can smoothly move around without noticeable issues.

The minimum amount of precision desirable is at least millimetre increments, or 0.001 metres. To maintain this means the game world can be no larger than +/- 9999.999 units, i.e. less than 10km in any one direction on any axis. Because world origin is placed at 0,0,0 in X/Y/Z axes, this means 10km on the positive and negative side, resulting in 20km overall.

So that’s where the 20x20km theoretical cap comes from, but this is not the end of the story. In reality, cumulative precision loss can start to appear once player travels more than 999.9999 metres (1 kilometre) from origin in any direction and precision drops from sub-millimetre to millimetre resolution. The farther player travels from origin, the more necessary it becomes to mitigate these problems through various design and optimisation strategies.

One strategy is to use a floating origin where the active world area is moved back towards origin anytime player travels beyond a threshold distance. This is the approach Daggerfall Unity uses to enable smooth movement across its vast world. Every time the player moves beyond a world block’s distance from origin (approx. 820m thus always below 999.9999 sub-millimetre threshold) the entire world is shifted back towards origin and new terrain is loaded in front of player. This can result in a noticeable pause on lower-end systems or with modded games as all this work needs to take place in under a couple of frames to remain imperceptible to player.

A floating origin handily gets around the precision problem while creating significant challenges in other areas. Having a lot of experience with those challenges from my time with Daggerfall Unity, I have opted not to use a floating origin in Project Arroa: Revenwood. I will instead deploy a range of other mitigations. Without going into more tedious detail right now, these will include:

• Limiting playable area: A maximum playable area of about 16 square kilometres per map provides a satisfying open world experience while limiting potential for cumulative precision loss. The world outside of this space can still be used for distant terrain and other features to help the world feel much larger, as precision is less important for distant and non-interactive elements.
• Double-precision master coordinates: The single-precision limitation applies to what is currently being rendered or simulated by the physics system. In other words, all of the realtime elements being processed every frame. For everything else, Project Arroa: Revenwood uses an over-arching double-precision coordinate system for storage and tracking critical objects in and out of active scene space. This ensures a highly precise source of truth across all systems.
• Sensible unit alignment: Another strategy for minimising precision dramas is to standardise on a minimum unit alignment for all world objects. Project Arroa: Revenwood uses 1cm units for its layouts, meaning all modular kits will snap together in some increment of 1cm (0.01 metres). This ensures no more than two digits of precision are required for snapping together and the world continues tiling cleanly far outside the expected play area. An alignment value requiring more digits of precision (e.g. inches to metres conversion like Daggerfall Unity uses) risk cracks between tiling objects as precision loss accumulates.
• Other strategies: There are other strategies I can explore as world comes together if precision becomes a factor. Because I have complete control over the engine and its rendering pipeline, I can use coordinate offset tricks and precision management in shaders to further mitigate any issues. To avoid premature optimisation, I will only start to explore these if and when they are required. I’ll have a better sense of my engine’s real-world precision tolerance once these systems come fully online.

World Cells in Depth

I mentioned earlier that world maps are managed internally using cells of 125x125m dimension. Let’s explore what that means and how it pertains to managing scene memory.

A small world map of 8×8 cells will define an area of 1x1km (8 times 125m equals 1000m). Each of these cells is like a bucket for information and behaviours called “cell systems”. An individual cell system might draw terrain with ground textures, modify climate, play environment effects, and so on. Additional cell systems can be layered into cells at any time without impacting other systems as each cell system is generally independent of others, with a few exceptions.

All cells in a map will potentially contribute something to the world, but it’s important not to spend time processing cell systems outside of their useful range. For this reason, cell systems come in two variants, active and passive. An active cell system only does its work at close range to player and will switch off outside of active range. Passive cell systems do lighter work at any range. This isn’t very important right now as I’ll mostly be talking about terrain in the next few posts, but keep in mind the active/passive split exists for later.

This design allows for each world cell to receive only the resources it requires at any time. For example, distant passive terrain might have simple, low-resolution elevation data and textures as it will only be seen from a great distance. Whereas cells containing a key gameplay location might have higher resolution terrain and texture data, custom weather, special ambient audio, and more. Some of these systems might only trigger once player moves into active range and lie dormant or unload completely while player is elsewhere on map.

In short, every world cell is designed to load quickly and efficiently and spread loading across several frames to maintain smooth action. A cell only contains the systems it requires, and these systems can be loaded in priority order.

From a storage perspective, everything about a world map layout is contained within its data file and whatever external assets are referenced. When the player enters a map, the map file definition is read from storage and the game will start making decisions about how to load and manage the scene from the data file supplied.

The first thing the engine does is partition the world into however many cells are required. At this stage all cells are just empty buckets with no systems attached, and use practically no memory. Then engine will perform the following steps to build world around player:

1. Determine all cells within active range of player. If the active range is 375m or 3 cells, then engine needs to build 3×3 active cells around player position.
2. Load critical game assets referenced by priority active cell systems, e.g. heightmap data. Non-critical systems and their assets will be loaded in subsequent stages.
3. Priority build terrain collision data for whichever cell player is standing on so their controller can be placed during remaining load process.
4. Priority load world objects in active area. Start loading remaining active cell systems in priority order.
5. Continue loading world objects and any passive cell systems outside of active range.

Other than loading any heavy game assets, the critical elements of above layout process would ideally complete within a few frames. Remaining elements can be spread across several frames or threads wherever possible to ensure loading is smooth and does not starve frame generation or input sampling. I’d rather have some pop-in during loading than game to stall completely for any amount of time.

As the player starts to move and explore map, new cells will come into active range and old cells will fall out of active range. Active cell systems are instantiated as cells come into active range and fallback to only passive cell systems as they transition out of active range. Any active cell features that go out of range are suspended and potentially purged from memory after a few frames. Passive cell features might remain memory resident at all times depending on their intended function.

The first cell system to come online will be terrain. Terrain cells use continuous level of detail, so terrain resolution smoothly increases as player approaches and decreases as player moves away. In general, some kind of smoothing/fading/scaling will be used to minimise pop-in of all visual and audible elements.

The overall goal of using distributed, layered world cell systems is to ensure the engine loads only what is needed in priority order and not everything all at once. This helps smooth out loading/unloading of world areas, maintains predictable memory requirements, and prevents distant objects from using up valuable CPU or GPU time when no longer needed.

Coming Up

Thank you for reading! For the next post in this series, I will discuss how terrain elevation is constructed with a hierarchical stamp layout, how terrain cells test for visibility, and how terrain system maintains a continuous level of detail as player moves around world map.