Collecting data from thousands of buildings is one challenge. Making that data immediately available to countless applications is another. Doing both reliably and efficiently at scale requires some deliberate architecture choices.
Novant was designed from day one to solve both problems. Today, the platform handles millions of data points, collecting live values and historical trends while serving dashboards, analytics platforms, APIs, and AI assistants.
The challenge isn't simply moving large volumes of data. It's managing the complexities of building connectivity, preserving the context and relationships behind that data, and retaining years of historical trends without making storage prohibitively expensive. Meeting those requirements has shaped an architecture where collection, storage, and retrieval can scale independently.
The shape of building data
Building automation systems contain thousands of individual data points, but those points don't exist independently. They belong to controllers, meters, and other devices connected to a building's network. In Novant, we call these devices sources. A BACnet controller might expose hundreds of points, while a Modbus meter might expose dozens. Regardless of the protocol, the points belonging to a source are typically collected together, over the same network, by the same edge node.
This makes the source a natural unit of work. Rather than treating every point as an independent stream, Novant organizes communication, configuration, caching, and historical storage around sources. Configuration is delivered to the edge per source, live values are returned and cached per source, and historical data is synchronized and stored per source.
By working at the source level, Novant reduces the number of individual operations the platform needs to manage while preserving the natural grouping of data as it moves through the system. It's a relatively simple architectural decision, but one that influences almost everything else in the platform.
Connecting buildings to the cloud
Building networks are rarely designed for cloud applications. They often sit behind firewalls, have limited connectivity, and contain equipment that was never intended to communicate directly with the internet. Novant was designed to work within those constraints, without requiring inbound firewall rules, VPNs, or complicated network configuration.
Every connection between a building and Novant starts at the edge. An Edge Node establishes a secure, outbound WebSocket connection to the cloud over TLS. Because the connection originates inside the building, there's no need to expose inbound ports or establish separate remote access infrastructure. The Edge Node can communicate securely with Novant using the building's existing internet connection.

Once connected, the cloud orchestrates the session. Rather than having thousands of Edge Nodes independently decide when to transmit data, the cloud determines when to collect live values, synchronize trends, deliver configuration, or execute requested operations. The same secure connection also supports schedules and controlled writeback to building equipment, without requiring inbound access to the building network. This centralized approach allows Novant to adjust workloads and synchronization schedules in response to changing demand or system load, without requiring changes to individual Edge Nodes.
The cloud also manages Edge Node firmware, allowing updates to be delivered remotely and keeping deployed nodes on supported software versions without manual intervention at each building. This avoids the version drift and dependency issues that can arise when software must be maintained independently across hundreds or thousands of sites.
By keeping orchestration, configuration, and software management in the cloud, Edge Nodes can remain relatively simple and focused on their primary responsibility: communicating with local equipment and collecting data. The result is a more reliable edge architecture that's easier to deploy, maintain, and scale across large building portfolios.
Managing data in the cloud
Building data has different characteristics depending on how it's used. Live values need to be fresh and immediately accessible, trends need to be stored efficiently over long periods, and the semantic model needs to make relationships between points, equipment, and spaces readily available.

Live values
Every 30 seconds, the cloud requests a snapshot of current values from each Edge Node. The Edge Node returns the latest values for its sources, which are written to a shared Redis cache in the cloud. Unlike historical data, live values never need to be written to a database. Only the latest value matters, so there's no reason to continuously write and index data that would immediately become obsolete.
An important consequence of this design is that reading live data never requires communicating with the building. In traditional building automation systems, retrieving current values can involve establishing connections, managing subscriptions or leases, and waiting for equipment to respond. With Novant, the latest collected values are already available in the cloud, so dashboards, APIs, and AI assistants can retrieve them immediately, without additional communication with the underlying equipment.
Trends
Trends follow a different path, beginning at the edge. Each source has a configurable recording interval of 15 minutes by default, or 5 minutes for higher-resolution data. Analog values are recorded at that interval, while binary values are also recorded whenever they change state. Binary values are periodically recorded even when unchanged, making it possible to distinguish a steady state from missing data.
Rather than transmitting every sample individually, Edge Nodes collect and buffer trends locally until the cloud requests them. This decouples data collection from cloud connectivity and allows trends to be synchronized in batches, reducing network traffic and giving the cloud control over when data is transferred.
Synchronization uses a simple cursor-based approach. The cloud tracks the timestamp of the most recently stored sample for each source and requests anything newer. If a synchronization attempt fails, the cursor remains unchanged, allowing the cloud to retry without losing its place. The Edge Node doesn't need to maintain synchronization state or know what the cloud has already stored.
By separating trend collection from synchronization, Novant can continue recording data during temporary connectivity interruptions and catch up when communication resumes. The cloud can also adjust synchronization activity independently of live data collection, allowing the two workloads to scale without competing for the same resources.
Semantic model and project context
Values and trends tell us what's happening in a building, but their real value comes from understanding what they represent. A temperature reading, for example, becomes much more meaningful when it's associated with a particular piece of equipment, the space it serves, and its relationships to other systems.
Novant maintains this context in a semantic model, represented as a connected graph of points, sources, equipment, spaces, zones, and their relationships. Rather than treating these as isolated records, the model captures how they relate to one another: which points belong to an air handler, which zones it serves, or how equipment is connected across a building.
The semantic model is also ontology-agnostic. Rather than requiring buildings to conform to a single standard, Novant supports mappings to Brick, Project Haystack, Google Digital Buildings (DBO), RealEstateCore, and VBIS. This allows the same underlying building model to work across different ontologies and applications without changing how data is collected or stored.
Even a single building can contain thousands of equipment assets, spaces, and relationships. Novant caches the semantic model in Redis, with additional in-memory copies on application servers and in browsers. This allows applications to traverse complex relationships directly from memory, making contextual lookups nearly instantaneous without repeated database queries.
Because the model changes relatively infrequently, it can be cached independently of live values and trends. A dashboard can explore equipment relationships, retrieve current readings, and display years of history without repeatedly reconstructing the underlying building model. The same approach allows APIs and other applications to work with contextualized building data efficiently, even as projects and demand grow.
Storing years of trends
Novant does not use a traditional time-series database for historical data. Instead, we store trends in Amazon S3, organized into one object per source per UTC day. This approach allows us to retain up to 10 years of historical data while keeping storage costs low and avoiding the operational complexity of managing billions of individual samples in a traditional time-series database.
Trend samples are written once and read many times. Once complete, historical data is effectively immutable, and queries typically request known points over a specific time range. These characteristics allow us to use a simple, predictable storage layout without relying on a database to organize billions of individual samples.
A predictable storage layout
History is organized by project, source, and UTC day. Every part of that hierarchy is already known when a query arrives: the requested points, the sources they belong to, and the dates being requested. Novant can calculate exactly which objects it needs without searching an index, scanning a table, or querying a database to locate them.
For example, requesting a year of trends from one source translates directly into 365 daily objects. Each object can be retrieved independently, allowing queries to read many days or sources in parallel.

Using a day as the unit of storage also simplifies how data is managed. New samples are merged into the appropriate daily object, so writes remain small and contained. Once all samples for a day have been synchronized, that object becomes effectively immutable. Retention and deletion are similarly straightforward, since entire days can be removed without touching unrelated history.
Durability is another major reason we chose S3. Amazon S3 is designed for 99.999999999% annual object durability, with data redundantly stored across multiple Availability Zones. This gives Novant highly durable storage without having to build and maintain a separate backup infrastructure for trend data. Because history is partitioned into independent objects, any data integrity issue is isolated to a single source and day, rather than affecting an entire historical database.
A compact format for building data
Object storage solves the problem of where to keep the data, but storing years of trends from millions of points also requires an efficient representation.
Novant uses a custom binary time-series encoding inspired by the Gorilla compression algorithm. It's designed around two characteristics of building data: samples are usually recorded at regular intervals, and values often remain unchanged or change only slightly.
Consider a temperature setpoint that remains at 72°F for several days, a damper that stays fully open for hours, or a binary status that changes only a few times each day. Storing a full timestamp and floating-point value for every sample would repeat nearly identical information thousands of times.
Instead, Novant stores samples in compact runs for each point. The first sample contains a 32-bit timestamp and a 64-bit value, while subsequent samples encode only what changed. We use one-second timestamp resolution because the physical processes in building mechanical systems rarely require finer precision, reducing storage without sacrificing meaningful detail for typical building applications.
For timestamps, we encode the interval between samples. When that interval remains unchanged, as it usually does with scheduled trending, the next timestamp requires just one bit. For values, we use XOR-based compression to identify which bits changed between consecutive readings. Unchanged values also require just one bit, while small changes can be represented using only the relevant bits.
The result is that a point holding a constant value at a regular interval requires just two bits per additional sample. At the default 15-minute interval, a full day of 96 samples with a single temperature change can be represented in approximately 43 bytes, compared with 1,536 bytes for uncompressed 64-bit timestamps and values – a reduction of roughly 36 times.

The same encoding is used by Edge Nodes and the cloud, allowing trend samples to move directly into storage without conversion into another data format.
Together, the predictable object layout, built-in durability, and compact encoding allow Novant to retain years of trend data without the operational complexity of a large, continuously growing time-series database.
Getting data back out
Collecting and storing millions of building points is only half the challenge. The platform also needs to serve that data efficiently, across thousands of simultaneous requests from dashboards, analytics platforms, APIs, and AI assistants.
This is where the earlier architectural decisions come together. When Novant receives a historical query, it groups the requested points by source and determines which daily objects overlap the requested time range. Because points from the same source are often requested together, a single object can satisfy multiple parts of a query. And because the storage layout is predictable, Novant knows exactly where to find those objects without searching an index or querying a database.
Each source-and-day combination becomes an independent read operation. Novant retrieves and decodes these objects concurrently, assembling the samples into a common time-series result. Calculated points can be evaluated, and the data can be returned at its original resolution or resampled and aggregated to the requested interval.
But parallel retrieval within a single request is only part of the story. Novant's cloud application runs across multiple Amazon EC2 instances behind an Elastic Load Balancer (ELB), distributing incoming requests across available servers. Each server can independently retrieve historical objects from S3 or access live values and the semantic model through shared Redis caches.

This separation of compute and storage is central to how Novant scales. As demand increases, additional application servers can be added without redistributing historical data or changing how Edge Nodes communicate with the cloud. With the semantic model also cached and readily available, each server can efficiently resolve building relationships alongside live and historical data. The same architecture supports both individual queries spanning years of history and large numbers of concurrent requests from independent consumers.
Whether data is requested through the Novant application, REST APIs, or AI assistant integrations, it follows the same underlying paths. There's no separate analytics warehouse to maintain or secondary historical data store to keep synchronized. The platform is designed to scale not just the collection and storage of building data, but its availability to everything that needs it.
Putting it all together
Scaling building data isn't just about collecting more points or storing more history. It's about making that data, and the context behind it, consistently available to the applications and people that depend on it.
By separating collection, storage, and retrieval while keeping the semantic model readily accessible, Novant can grow across buildings, users, and applications without fundamentally changing how the platform operates. The result is a scalable and economically efficient foundation for building data, capable of retaining years of history while making both current and historical information readily available.