Brian has posted a Dev Diary about the critical 13-day rewrite of the Ingress backend architecture following severe server failures during the Orion Anomalies on May 16. The database overhaul prioritised write performance and memory management to withstand the extreme load of anomaly events.
The development team migrated to a new database in March 2026. While everyday latency improved during early events like +Gamma Hyderabad and the Orion Global Op, the architecture failed under peak stress during the onsite anomalies in Sydney and Prague. Agent activity overwhelmed the auto-scalers, causing logjams and systemic collapse. In response, Niantic initiated an emergency code red, tasking the team with overhauling the infrastructure before the Orion Kure and Jersey City anomalies.
Pre-warming Anomaly Sites
While the team already manually spins up additional server capacity on Fridays before an Anomaly, specially during the Prague Anomalies the system still needed almost 40 minutes to analyze traffic and detect which data needed to be moved to be readily available.
The system now pre-loads specific, heavily requested data prior to the first measurement windows in addition to the usual increase in processing power.
Action Logjam Removal
Agent actions were previously recorded in a single table indexed by timestamp, however during anomalies, simultaneous writes to the same endpoint caused severe delays.
This index was removed. Administrative query speed has decreased slightly, but live write performance improved significantly.
Ghost Record Elimination
The portal index, previously sorted by physical location and last update time, generated outdated "ghost entries" with each state change. Anomaly traffic created a 100:1 ratio of ghost records to actual updates, choking the system.
The "last update time" ordering was removed. The database now scans and sorts nearby portals on-the-fly, adding a minor latency (under 1.0 millisecond) to everyday play but resolving the anomaly backlog.
In-Memory Batching
The most significant structural change involved moving portal updates from immediate database writes to an in-memory batching system. Game actions are now processed in temporary server memory before being written to the database in batches.
Performance Data and Results
The codebase overhaul required updating over 200 files, adding 22,000 lines of code, and removing 5,000 lines. The new infrastructure was deployed on May 30 for Orion Kure and Jersey City, operating without scanner instability or global login issues.
Currently, the in-memory architecture is active only during Anomaly weekends and disabled the following Monday to gather data and isolate bugs. Recent patches have resolved subsequent issues, such as frozen portals requiring a photo interaction to force a state update, and specific mission completion blockers. The development team intends to refine the system until the in-memory portal architecture can run continuously and during everyday gameplay, providing long-term stability for the XM Network.
Source: https://ingress.com/news/rewiring-xm-network
@IUENG
547 - 801
74+26+42 - 23+56+58
