The Architecture of Real-Time Dashboards: WebSockets, Polling, and Event Sourcing

The first version of Aura Finance's dashboard was a lie. It showed portfolio balances that were five seconds old, occasionally stale by thirty, and completely wrong whenever the WebSocket that fed the backend silently died. We noticed because a user tweeted a screenshot of their account showing a trade that never executed. The trade appeared because our polling layer had fetched a half-written aggregate, constructed a phantom position, and rendered it as fact.

I built Aura Finance as a personal wealth dashboard: it aggregates crypto, brokerage, and bank account balances, then renders a live net-worth chart and a list of recent transactions. For a finance app, "live" is not a nice-to-have. When a wire transfer clears, the user should see it clear. When a market order executes, the chart should update before the user refreshes their brokerage app. Achieving that meant rethinking the data path from the ground up, and the architecture we landed on uses a mix of WebSockets, event sourcing, and carefully managed projections. This is the detailed account of that transition.

Why We Started With Polling (and Regretted It)

The first implementation was embarrassingly simple: every five seconds, each open dashboard fired GET /api/positions and GET /api/transactions, replaced the React state with the new JSON, and let the chart re-render. It worked for 10 minutes and one user. Then I invited a dozen friends, and the Postgres CPU graph started looking like a stock market spike.

The math was brutal. Two hundred active clients, each polling two endpoints every 5 seconds, means 80 requests per second. Each request triggered a handful of queries against positions, trades, and balances, many of which hit the same hot rows. The database spent more time serving redundant reads than writing actual events. Worse, clients that got throttled by a mobile network would queue callbacks, then fire them all at once when connectivity returned, creating a thundering herd.

The user-visible problem was latency, but the real problem was architectural: polling makes the server answer questions the client already knows the answer to. Every poll re-sends the entire positions list, even if nothing changed. The only way to make polling acceptable is to add conditional requests with ETag headers or version counters, and you are still stuck with the worst-case latency of one poll interval.

WebSockets: The Honest Transport Layer

We replaced polling with a single WebSocket connection per dashboard. After the initial state snapshot is fetched over HTTP, the client subscribes to a stream of transaction and position events. The server pushes only what changed, and the client applies those deltas to its local state.

Why WebSockets over Server-Sent Events (SSE)? Because Aura Finance also lets you mark a transaction as "reviewed" or add a note, which changes the dashboard state. That requires the client to send messages. SSE is one-way and would demand a parallel HTTP channel for those actions. I am not dogmatic: if your dashboard is purely read-only, SSE is a lighter option with automatic reconnection built into the browser. For us, the bidirectional channel collapsed the negotiation down to one connection.

The protocol we settled on uses a monotonically increasing event ID. Every event pushed to the client carries an ID. On reconnect, the client sends {"type":"resync","lastEventId": 1042} and the server replays everything after 1042. Here is the TypeScript client class that handles the core of it:

type ServerEvent = {
  id: number;
  type: "position.updated" | "transaction.created" | "transaction.deleted";
  payload: Record<string, unknown>;
};

export class DashboardSocket {
  private socket: WebSocket | null = null;
  private lastEventId = 0;
  private queue: ServerEvent[] = [];
  private shouldReconnect = true;

  connect() {
    this.socket = new WebSocket(`wss://api.aura-finance.dev/dashboard`);
    this.socket.addEventListener("open", () => {
      this.socket!.send(JSON.stringify({ type: "resync", lastEventId: this.lastEventId }));
      while (this.queue.length) {
        this.socket!.send(JSON.stringify(this.queue.shift()));
      }
    });
    this.socket.addEventListener("message", (event) => {
      const msg = JSON.parse(event.data) as ServerEvent;
      this.lastEventId = Math.max(this.lastEventId, msg.id);
      this.applyToStore(msg);
    });
    this.socket.addEventListener("close", () => {
      if (this.shouldReconnect) {
        setTimeout(() => this.connect(), 1000 * Math.min(2 ** this.retryCount(), 30));
      }
    });
  }

  sendRaw(message: unknown) {
    const raw = JSON.stringify(message);
    if (this.socket?.readyState === WebSocket.OPEN) {
      this.socket.send(raw);
    } else {
      this.queue.push(raw as unknown as ServerEvent); // held until resync
    }
  }
}

The lastEventId resync is the single most important piece of the client code. It guarantees at-least-once delivery because the server keeps an event log. If the client disconnects for 30 seconds, it just asks for events after the last one it saw. That makes the system resilient without any client-side retry logic beyond the resync message.

Event Sourcing: The Log That Never Forgets

Broadly, event sourcing means you treat the persistable truth as a sequence of immutable facts, not as the current state of a row. When a trade executes, you do not update a positions table. You append a trade_executed event to an events table. The positions table becomes a derived projection that can be rebuilt from the event log at any moment.

Why is this useful for a dashboard? Because the dashboard needs to show historical state, and users care about "what did my portfolio look like when I saw that balance?" If you only store current state, that question is impossible to answer. With an event log, you can replay the stream up to any point in time and reconstruct the exact portfolio as of that moment.

In Aura Finance, a transaction event looks like this:

CREATE TABLE events (
  id BIGSERIAL PRIMARY KEY,
  aggregate_type TEXT NOT NULL,      -- 'account' | 'position' | 'transaction'
  aggregate_id UUID NOT NULL,
  event_type TEXT NOT NULL,          -- 'trade_executed' | 'deposit_confirmed'
  payload JSONB NOT NULL,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX idx_events_aggregate ON events (aggregate_type, aggregate_id);
CREATE INDEX idx_events_sequence ON events (id);

We use Postgres as the event store. For Aura's scale this is plenty, and pushing everything into Kafka would have been operational overkill. The id BIGSERIAL gives us both the event ID for client resync and the ordering guarantee. Postgres sequences are monotonic under normal operation, so id > lastEventId is a clean query.

The write path is intentionally dumb. Here is the Python function we use in the API layer:

import json
from psycopg2.extras import Json
from db import get_connection

def append_event(conn, aggregate_type: str, aggregate_id: str, event_type: str, payload: dict) -> int:
    """Append an event and return its generated ID."""
    with conn.cursor() as cur:
        cur.execute(
            """
            INSERT INTO events (aggregate_type, aggregate_id, event_type, payload, created_at)
            VALUES (%s, %s, %s, %s, now())
            RETURNING id
            """,
            (aggregate_type, aggregate_id, event_type, Json(payload))
        )
        event_id = cur.fetchone()[0]
        cur.execute(
            "SELECT notify_websocket_broker(%s, %s)",
            (event_id, json.dumps({"type": event_type, "payload": payload}))
        )
    conn.commit()
    return event_id

The notify_websocket_broker function is our bridge between the transactional database and the WebSocket layer. It uses Postgres LISTEN/NOTIFY so the broker process learns of new events without a second write path. This is what transforms the event log from boring persistence into a real-time stream.

Projections: Turning Events Into Dashboards

The event log is the truth, but it is useless for answering a SELECT * FROM positions WHERE user_id = 42 query fast. You cannot scan the events table every time someone opens the dashboard. You need a projection: a read model that summarizes the event stream into a queryable structure.

In Aura Finance, a projection is just a Postgres table that we update transactionally after each event. The balance of an account is a projection of deposit_confirmed, withdrawal_initiated, and trade_settled events. The position of a stock is a projection of trade_executed events.

The trick is the update path. When the WebSocket broker receives a new event, it invokes a projector function. For a trade_executed event, the projector does three things in a single database transaction:

1. Upsert the positions row for that account and symbol.
2. Append a position_updated message to a outbox table for the WebSocket broker to publish.
3. Update the account's total equity figure in a portfolio_summary table.

If any step fails, the whole transaction rolls back and the event is not marked as projected. That gives us atomicity between the projection and the event acknowledgment. If the process crashes mid-way, we restart from the last projected event ID, and the upsert is idempotent because it stores absolute shares, not deltas.

This is where event sourcing really shines for a live dashboard: because every projection is derived from the same ordered log, there is exactly one transformation path. No more "why does the position screen disagree with the transaction list?" They both come from the same projector, and if they disagree, it is a bug, not a race.

The Snapshot Problem: Keeping Projections Warm

The first time we restarted the projection service, it tried to replay four months of events from the beginning. It took twenty minutes. The dashboard was dark, and I learned the difference between an event-sourced system and a snapshot-aware event-sourced system.

The solution is a snapshot every N events per aggregate. For each account, we store a snapshots table with aggregate_id, sequence_number, and a JSON blob representing the full state as of that sequence number. On restart, we load the latest snapshot, then replay only the events after it.

CREATE TABLE snapshots (
  aggregate_type TEXT NOT NULL,
  aggregate_id UUID NOT NULL,
  sequence_number BIGINT NOT NULL,
  state JSONB NOT NULL,
  created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  PRIMARY KEY (aggregate_type, aggregate_id, sequence_number)
);

We take a snapshot at every 1000 events per aggregate. With a few thousand aggregates, that means the worst-case replay is 999 events per aggregate, which takes milliseconds. The snapshots are immutable; we never mutate a snapshot in place. Instead we write a new row at a higher sequence number and let old snapshots become garbage.

We also learned to keep the projector's high-water mark in a dedicated projector_progress table. That table records the last event ID each projector has consumed, and on startup the projector resumes from there. This is how we avoid replaying events that were already projected to a different read model. Jobs that build weekly reports use their own progress rows, isolating them from the dashboard projectors.

Reconnects, Resyncs, and the Event ID Game

The hardest part of real-time dashboards is not the happy path. It is the client that disappears mid-event, the mobile phone that switches from Wi-Fi to LTE, the laptop that sleeps for three hours and wakes up. Your WebSocket server will see broken connections every single day.

The event ID resync approach handles this elegantly, but there is a subtlety we had to get right: the client must send its lastEventId on every reconnect, and the server must be able to say "I no longer have events before that ID" if a snapshot has truncated history. In that case, the server returns a full snapshot replay.

| Strategy | Pros | Cons | Where we use it | |---|---|---|---| | HTTP polling | Trivial to implement, works behind any proxy | High server load, polling interval latency, battery drain on mobile | Only for the initial page load (one snapshot) | | Server-Sent Events (SSE) | Built-in reconnect, simple HTTP semantics | One-way only, limited by proxy buffering | Not used, but recommended for pure read-only dashboards | | WebSocket with event ID resync | Bidirectional, low latency, precise resume point | More complex client code, must handle backpressure | Primary dashboard transport | | WebSocket sending full state | Simple on the server, client just replaces everything | Bandwidth waste, no delta granularity, full render loops | Never used for updates |

The golden rule we eventually encoded: never trust the client's timeline. The client might have missed events due to a proxy timeout, or it might have processed events out of order if the WebSocket delivers a cached frame after a retry. We always use the event ID from the server log as the source of truth. The WebSocket broker does not assign its own message ID; it forwards the event store's ID. That guarantees the ID space is unified and monotonic across all clients.

Backpressure: When the Client Can't Keep Up

Real-time dashboards hit a problem the tutorials never mention: what if the client is slow? When a user has the dashboard open in a background tab, the browser throttles JavaScript timers and sometimes delays WebSocket message processing. Meanwhile, the server is firing 200 events per second because markets are volatile. The client's message queue in the browser grows until memory spikes.

Our first solution was to send every event to every client that subscribed to that account. That collapsed under a burst of 1,500 trade events in one minute on a client with a throttled tab. The page froze for several seconds because the React reducer was applying a thousand individual updates.

We transitioned to a batching protocol. The server accumulates events per client for a maximum of 100ms or 100 events, whichever comes first, and then sends a single message: {"type": "batch", "events": [...]}. The client applies the batch, and the renderer runs at most once per frame. This reduced the React reconciliation overhead by an order of magnitude without adding observable latency.

For extremely slow clients, the server also maintains a per-client event queue with a fixed maximum of 2,000 events. If the client falls further behind, the server drops the queue and sends a {"type": "snapshot_required"} message. The client responds by requesting a fresh projection snapshot. The block of code below shows the batching loop in our Python broker process:

import asyncio
from collections import defaultdict
from websockets import WebSocketServerProtocol

BATCH_WINDOW_MS = 100
BATCH_MAX_EVENTS = 100
MAX_QUEUE_EVENTS = 2000

class DashboardBroker:
    def __init__(self):
        self.clients: dict[int, WebSocketServerProtocol] = {}
        self.pending: dict[int, list[dict]] = defaultdict(list)
        self._flush_task = asyncio.create_task(self._flush_loop())

    async def _flush_loop(self):
        while True:
            await asyncio.sleep(BATCH_WINDOW_MS / 1000)
            for user_id, events in list(self.pending.items()):
                if not events:
                    continue
                message = {"type": "batch", "events": events}
                try:
                    await self.clients[user_id].send(self._serialize(message))
                except Exception:
                    del self.clients[user_id]
                del self.pending[user_id]

    async def dispatch(self, user_id: int, event: dict):
        if len(self.pending[user_id]) >= MAX_QUEUE_EVENTS:
            self.clients[user_id].send('{"type": "snapshot_required"}')
            self.pending[user_id].clear()
            return
        self.pending[user_id].append(event)

This batching is a classic throughput-vs-latency tradeoff. We found that adding 100ms of buffering to update a dashboard chart that redraws at 30fps is imperceptible, but it reduces the number of WebSocket frames we send during a volatile market from 1,500 to 15. That is the difference between a dashboard that survives a burst and a dashboard that spins its CPU fan up to maximum.

What We Measured and What We Learned

After the migration, we ran metrics for two weeks. The numbers told the story we expected, but the magnitudes still surprised me.

• Dashboard render latency (event to pixel): p95 went from 4.2s to 230ms. The polling interval alone was 5s, so almost all of the improvement came from push delivery. The remaining 230ms includes 100ms of server-side batching, network RTT, React render, and the SVG chart animation.
• Database load: Peak read QPS dropped from 80 to 5. The only HTTP requests now are the initial snapshot load and the occasional explicit refresh. Postgres CPU usage on the read replica fell from 65% to 8%.
• Connection stability: We track WebSocket closure events and reconnects. The average client reconnects every 3.2 hours. Without the event-ID resync, every one of those reconnects would have required a full state reload. With resync, 92% of reconnects simply ask for the 5–30 events they missed and continue.

The most valuable architecture decision, though, was event sourcing not because it is trendy, but because it made debugging real-time bugs actually possible. Before, when a user reported a phantom balance, we had to poke at production data. Now, we ask for their account ID and replay that account's event stream in a local environment. We can see exactly which event caused the projection to drift, and we can fix the projector, then rebuild that account's snapshot. No guessing.

The other lesson is that real-time does not mean "instant." It means "eventually correct, fast enough to feel immediate, and explicitly acknowledging every gap." The snapshot_required fallback and the batch window are both about that honesty. When the dashboard cannot keep up, it tells the client and the client recovers rather than silently dropping data.

Conclusion

Building a real-time dashboard is less about choosing a fancy transport and more about deciding where the truth lives. In Aura Finance, the truth lives in an append-only event log. WebSockets are just the delivery vehicle, and projections are just convenient views. Every piece of the interface, from the balance chart to the transaction list, is derived from that log, which means every piece can be rebuilt at any time.

The journey from polling to WebSockets taught me that "live data" is an architectural commitment, not a feature flag. It forces you to handle disconnects, backpressure, and replay semantics with the same rigor as database transactions. I tried to skip those problems and paid for it in debugging hours.

If you are building a dashboard that needs to feel alive, start with the event log. Make it immutable, give it monotonic IDs, and then connect as many projections as you need. Do not treat real-time as an add-on to your existing REST API. Restructure the data path so push is the primary channel and polling is only the fallback.

The full source code for the dashboard, including the WebSocket broker, the event store, and the React client, is open source: View the project on GitHub.