Foreword

This paper is published by Anker Innovations. It argues that the current smart home architecture has reached its limits and that the next generation needs a different foundation: intelligence that operates inside the home, under the control of the people who live there.

A letter from our founder and chief executive follows the main text. The paper reflects Anker's convictions and invites the rest of the industry to consider the same questions.

Executive Summary

The smart home was built on a bargain. At the time, early smart home devices could not practically support the processing power, memory, and storage required for more advanced capabilities. Doing so would have made mass-market products too expensive. Remote computing and storage offered a practical alternative. Voice commands could be processed in the cloud, and video could be stored remotely and accessed from anywhere. That trade produced a decade of useful products, and it was the right engineering answer to the constraints of its time.

But it did not deliver the intelligent home we were promised. Most products still operate separately, with limited understanding of what is happening elsewhere in the house or with other devices. And because much of their intelligence still sits outside the home, important decisions remain dependent on a connection to a remote system. That was workable for simple voice commands or streaming content. It is a poor foundation for a home expected to understand what is happening and act on its own. Meanwhile, the technical and economic constraints that drove intelligence and storage into remote data centers have largely disappeared. Models capable of interpreting speech, images, and events can now run locally on hardware practical for the home.

We propose a different foundation. Individual devices in the home should handle decisions that must happen immediately, while a shared household system handles decisions that require information from across the home or from earlier events. That household system must be able to connect to the devices around it regardless of who made them, remember the context that matters over time, and use both to understand what is happening, make decisions, and coordinate action across the home. The cloud will remain important for updates, remote access, and work that requires more computing power than the home can provide. But the home’s core intelligence should operate locally.

Achieving the intelligent home we were promised will require more memory, not less. To understand context and act on its own, the home will need to draw on what its devices see, hear, and measure over time: images and recordings, the people it recognizes, maps of the rooms, patterns of activity, and the routines of the people inside. Together, that becomes more than technical data. It becomes a record of life in the home.

That brings us to a question as important as the technology itself: where should that record be kept, and who should control it? We believe the home’s memory should stay in the home, under the control of the people who live there. The companies that build the technology should not become the custodians of that memory. It belongs to the household. This is the direction Anker is taking, and the standard we believe the industry should adopt.

The Promise

For years, we have been promised the futuristic smart home. It would be a home that understood us, anticipated our needs, adapted to our routines, and quietly handled the small tasks that fill our days. Technology would fade into the background, and the home itself would become an intelligent partner, making life simpler and safer.

That is not what most of us received. Instead, the smart home became a collection of connected devices: cameras, speakers, lights, locks, thermostats, appliances, and sensors, each with its own app, its own settings, and its own demands for attention. We gained more ways to control our homes, and more things to manage. The home became connected, and in some ways it became smarter, but it never became intelligent.

A connected home waits for instructions. An intelligent home understands context. It perceives what is happening, works out what it means, and coordinates across the home to decide how to help. It notices, remembers, and acts without being asked.

For most of the smart home's history, that kind of intelligence was beyond the reach of the devices themselves. The chips inside them were not powerful enough, storage was limited, and the systems required to interpret voices, images, and behavior could not run inside an ordinary household product. So the industry borrowed intelligence from somewhere else: the cloud.

But it also shaped what the smart home became. We did not build intelligence into the home. We built connections from the home to intelligence located somewhere else. The products sat inside our homes, but much of the processing that made them useful took place beyond their walls.

The same architecture also sent household information outside the home: a voice command, a camera recording, a map created by a robot vacuum, a face used to unlock a door, a record of when a room was occupied or when an ordinary routine changed. Each item may look limited on its own. Together, they create a detailed record of life in the home.

They become the home's memory: an accumulated understanding of the people who live there, the spaces they occupy, the routines they follow, and the ways their lives change over time.

That memory is essential to the intelligent home we were promised. A home cannot anticipate without learning, or coordinate without understanding context, and it cannot act on its own without recognizing the people, places, and patterns around it. But the same memory that lets a home help can also expose the people inside it. It can show who lives there, how they are recognized, how the home is arranged, and when it sits empty. So where that memory is stored matters.

This is the Faustian bargain at the center of the smart home era. The gifts came first: useful features, real convenience, and ordinary products that seemed to become intelligent almost overnight. The price became visible later, as more of the home's daily life came to rest on systems outside the household's control. The bargain was not irrational, and for a time there was no practical alternative. But a compromise made under the limits of one era should not become the foundation of the next.

The next era will need more of this memory, not less. The more a home does on its own, the more it must know about the home and the people inside it.

Where should the home's memory live, and who should control it?

The Faustian Bargain

In the German legend, Faust accepts a bargain that gives him knowledge and power in return for his soul. He understands the terms. He accepts because the gifts are immediate and extraordinary, and the bill is distant. The smart home made its own bargain, and the conditions that made it possible were established long before the first connected devices entered our homes.

The habit had already formed.

For roughly 15 years before smart home products became common, people were already storing personal material on remote servers. Webmail made the practice familiar in the 1990s. Online documents followed, then photographs, music, calendars, contacts, backups, and social activity. That exchange was easy to understand: people chose what to send and received something useful in return. A message stored online could be read from another computer, a photograph could be shared, a document could be opened from more than one place. These servers did not feel like someone else's computers; they felt like part of the service.

The constraint was real

When the first generation of smart home products arrived, the industry faced a real technical constraint. The computing needed for useful speech recognition or computer vision required too much processing power, memory, energy, and space to place inside an affordable doorbell, camera, speaker, or sensor, and adding meaningful local storage raised the cost of every unit. The intelligence existed, but it could not fit inside the device at a price most households would pay.

The science behind that intelligence was already decades old. Machine learning, computer vision, speech recognition, and statistical pattern analysis had developed through a long series of real advances before these capabilities began appearing widely in consumer products. Deep learning brought an important step forward around 2012, when large neural networks began producing major improvements in areas such as image recognition. ¹ But those models required computing and memory that small household devices could not provide, and for years afterward researchers were still developing compression and quantization methods to make them practical on mobile hardware. ²

We chose to borrow the intelligence

We had two choices. We could wait for local hardware to become powerful and affordable enough, delaying useful capabilities for years, or we could place the demanding computing and storage in large data centers and connect household devices to them. We chose the second path.

Instead of asking every device to do the difficult work itself, it could send information to a remote system and receive the result back. A voice assistant could send a spoken request to the cloud, where it could be recognized and answered. A camera could send video to the cloud for storage and remote access, avoiding the need for large amounts of storage inside the home.

For the technology available at the time, this was the practical solution. It allowed relatively simple and affordable devices to offer capabilities that would otherwise have required far more powerful hardware inside the home.

Cloud services also changed what could happen after a product was sold. A better recognition model could improve a camera without replacing its hardware. A speech service could understand more words without changing the speaker. A feature could be tested, revised, expanded, or removed while the physical product stayed where it was, inside the home.

More capable silicon, memory, and storage inside every product had to be paid for on every unit, and that cost had to be committed before anyone knew whether the feature would work well enough or whether consumers would value it. Shared computing changed the calculation. We could build the demanding intelligence once and make it available to millions of connected products, at a marginal hardware cost far below equipping every unit to do the same work alone. Computing resources could also be pooled and expanded as demand changed, without redesigning the product in the home, which was central to the cloud model itself. ³ Under those constraints, the cloud was the practical choice.

This architecture created an important asymmetry. Information flowed outward from the home: recordings, images, events, measurements, and patterns were sent to remote systems for storage or processing. What came back was usually much smaller: an answer, an instruction, a notification, or some other result. The computation happened elsewhere, and much of the information used to produce that result remained there.

Over time, millions of homes began sending what their devices saw, heard, and measured to the same remote infrastructure.

We knew the costs

We understood many of the practical costs when we made this choice. A request sent to a remote server and back would take longer than an action completed inside the device. A product dependent on a remote service would lose functions when the internet connection or the service failed. Hardware tied to a cloud platform could lose much of its value if the company supporting it changed direction. None of this was hidden from the people designing cloud-dependent products. The service was part of the product, and we knew it.

We also understood that accumulated data had value. By the beginning of the smart home era, personal information was already being described as a significant economic asset capable of supporting personalized services, innovation, and new business models. ⁴

A major European assessment published in 2009 described cloud scale as both an advantage and a risk. Concentrating computing resources could make security measures more efficient and easier to apply, but massive concentrations of resources and data also presented a more attractive target. The report also noted that cloud customers necessarily ceded control to providers over some matters affecting security. ⁵

These tradeoffs were well documented while cloud-dependent products were being built. We accepted them because the benefits were substantial and the alternative was not yet practical. The terms were understood by the people building the system, even when they were far less visible to the people bringing the devices into their homes.

What we did not see coming

We underestimated how much information the home would eventually create and how detailed it would become. Early connected devices captured simple commands, isolated events, and basic status. Later products could map rooms accurately, recognize faces and voices, measure sleep and heart rate, identify household members, and learn patterns of presence and absence. The information became more revealing, not just more abundant.

We also underestimated how many different products would contribute to the same picture. A camera sees one part of the home and a speaker hears another. A robot maps the rooms, a lock records arrivals and departures, and a health device adds information about the body. Each product serves its own purpose, but together they can create a continuous account of household life: who is present, where they are, what they normally do, and when something changes.

We also did not expect the original arrangement to last long enough to become the standard design for the entire category. Each new product inherited the same assumption: the device would sense what was happening in the home, then send that information elsewhere to be understood. Companies that entered the category later, including our own, inherited this architecture and built on it. Adopting it was still a choice.

What consumers saw

Households did not see the trade the way we did. Their understanding of remote storage came from the services they had already used, where they chose to send an email, upload a photograph, save a document, or store music. Each of those actions involved a visible object and a conscious decision, and the person knew what was being sent because the person had selected it.

The smart home changed that relationship. Much of the information it creates is ambient, produced by ordinary life without any act of sharing. A camera does not ask for a new decision each time it records movement. A speaker begins processing because it believes it heard a particular phrase. A sensor builds a history of activity simply by staying switched on. This was not the same trade consumers had made with email, documents, and photographs, even though it looked familiar.

We knew that the intelligence lived outside the house and that the products would stop working whenever the connection to it was lost. Consumers saw a physical product inside the home with smart features that just worked. Because the device was in the home, the experience felt local even though much of the system behind it was not.

The terms of the bargain

Devices became capable almost overnight, and much of what made them capable did not live inside the house. We gained useful features, lower hardware costs, faster improvement, and a way to make small products behave intelligently. In return, household devices sent remote systems the information they needed to see, hear, interpret, learn, and remember.

The trade was not only about where processing happened. The home also sent out the information from which its memory would eventually be built, and that memory was being stored somewhere else.

That is where the bill begins to come due.

The Bill Comes Due

Industry research now describes a progression from systems that generate answers to systems that plan and then to systems that perceive and act in the physical world. Generative AI produces content in response to a request. Agentic AI goes further, planning and completing a series of tasks. ⁶ Physical AI carries that ability into the world, where a system perceives its surroundings, decides, and acts through machines and devices. ⁷

The smart home's current architecture was not built for systems that perceive and act in the physical world.

Physical AI perceives what is happening in a space, decides what to do, and acts through something physical: a lock, a robot, an appliance, a light. The response is not a notification for someone else to act on. Research on embodied intelligence treats memory as a core part of this loop, alongside perception, action, and learning, because a system cannot adapt its behavior without retaining and using what came before. ⁸

The first generation of connected devices sent information to a person, or followed rules set in advance. A camera detected motion and sent an alert. A thermostat followed a schedule. The judgment came from the person or from a rule someone wrote beforehand. Physical AI asks the home to work out what is happening on its own, including in situations nobody anticipated, and to respond. The architecture we built for the first job cannot do the second, for four reasons.

The decision happens somewhere else

A home that must ask a distant data center what to do before it can act depends on that data center. A faster connection shortens the delay, but it does not move the decision. If the understanding and the judgment happen elsewhere, the home is not acting on its own, and no improvement in network performance changes that. Autonomy depends on where the decision happens, not on pushing response time below some number of milliseconds.

Cloud systems are fast and powerful, and many tasks can tolerate a round trip to a data center: a request to play music can wait far longer than a machine responding to an obstacle in its path, and a heating adjustment is less urgent than a response to smoke, a fall, or an unexpected entry. Every decision routed through a data center inherits distance, routing, congestion, and processing that the home does not control.

Research on edge computing has made this distinction for years. Centralized processing requires repeated communication between devices and geographically distant data centers, which constrains applications that need real-time response. ⁹ Distance matters when information has to be interpreted and acted on immediately, whatever computing power sits at the other end.

A single physical decision may depend on several streams of information arriving together: what a device sees now, what it saw seconds ago, what another device detected, and what the system knows about the ordinary state of the home. Each round trip adds another dependency between seeing an event and acting on it. A connection may be fast enough most of the time, but a household system also has to work when it is not.

The connection will fail

Internet connections drop. Cloud services have outages. Accounts lapse, software changes, and providers modify or end what they offer. These are ordinary conditions in any connected system, not rare failures that better engineering removes. 

In research across 1,503 smart home owners in the United States, the United Kingdom, and Germany (see Appendix A), 67 percent said devices in their homes had stopped working when the internet or Wi-Fi went down. Twenty-three percent said it was a real problem rather than an inconvenience.

For a device that reports information or plays music, an interruption is an inconvenience. The person waits, reconnects, tries again. The problem is more serious when the home is expected to understand events and act on them. If the connection fails while an event is developing, the path from perception to decision breaks. The sensors still work, the devices still have power, and the necessary action is still physically possible, but the system has lost the ability to interpret what it is seeing.

Research into continuous video analytics already treats disconnection as a normal design condition. Processing is placed near the source in part because links to remote computing can be expensive or unavailable, and local processing lets the system keep working when the connection does not. ¹⁰

The cloud service may return in seconds or in hours. The event inside the home does not wait.

It was available, on the condition that something outside the house kept working.

Cloud dependence also creates a longer-term risk. A cloud-backed capability lasts only while the service behind it stays active and compatible. A decision made outside the home can change what a device inside the home is able to do, even when the device itself is untouched. That was an accepted term of the first smart home era. It is much harder to accept when the service supplies the intelligence the home needs to perceive, decide, and act.

Being wrong now costs more

The first generation of consumer AI answered questions, classified images, sent alerts, and made recommendations. A person remained responsible for acting on any of it. When the software was wrong, the result was usually confusion or inconvenience.

When software controls physical devices, a wrong or missing decision can have a larger consequence. It directs robots, operates equipment, changes physical access, and responds to events as they happen. The system now changes the home instead of reporting on it.

Consider a security event with the internet down. A camera sees someone at a side door at three in the morning. Under the current architecture, the footage travels to a data center, a model classifies it, and the result comes back as an alert. With no connection, the camera is still recording and the lock still works, but nothing in the house can determine that this is a person rather than a passing animal, or that this door is not normally used at this hour, or that the home should respond. Every physical capability required is present inside the house. The judgment is not.

Cloud-supported actions are not inherently unsafe, and not every action should be automated. But as the system takes on more responsibility, its basic judgment needs to keep working during network interruptions.

The workload has changed

The original economics worked because the first generation of connected intelligence ran in short bursts. Someone spoke a command, a device uploaded an event, a server returned a result, and the connection went quiet.

Physical AI replaces occasional requests with continuous perception. Cameras, microphones, motion and environmental sensors, and machines all produce information throughout the day, and a system coordinating a home may need to interpret several of those sources together rather than handling each as an isolated event. The cloud can process continuous streams, but continuous observation across millions of homes requires far more network capacity, processing, and storage than occasional requests.

Edge computing research identified this pressure before the current generation of AI, noting that growing numbers of connected devices produce data volumes that increase network traffic and degrade response times when sent to centralized systems.⁹ Work on continuous video analytics reaches the same conclusion from a harder workload: streams are commonly processed on nearby servers because continuously sending them to distant systems demands expensive network capacity and leaves the system exposed to disconnection, and these deployments already involve multiple live feeds and multiple models running at once, well short of a full home.¹⁰

Continuous perception also creates a growing storage requirement. A home that acts on its own has to compare what is happening now with what it has learned over months and years, so that record has to exist somewhere and keep growing.

Faster connections do not solve this

Faster, more reliable networks will improve many connected services.

But a faster connection cannot remove the round trip, and it cannot make a remote decision originate inside the home. It cannot promise that a service will always be reachable, and it does not reduce the volume of information continuous perception produces. When a system acts in the physical world, the question is what happens when a service does not respond at all, not how quickly it usually does.

The home's basic ability to understand and act should not depend on an internet connection.

What people already feel

Consumers already see these limits in ordinary use. Commands lag. Devices lose functions when Wi-Fi drops. Products sold as intelligent still require constant management.

The same research asked how intelligent those products actually feel to the people who own them. Nearly half, 45 percent, do not consider the smart devices in their homes to be very intelligent.

The industry is moving toward more demanding systems: from systems that answer, to systems that plan, to systems that perceive and act in the physical world. That future needs continuity, coordination, memory, and the ability to respond when an event happens rather than when a remote service becomes available.

It succeeded. It cannot become the foundation of an autonomous home simply by carrying more data, adding more cloud computing, or using faster network connections.

The current design cannot support that future. It cannot give the home reliable judgment of its own without placing an increasingly complete record of household life outside the household's control.

The Death of Excuses

For most of the smart home's history, local intelligence was not practical. The processors were too limited, the models were too large, and enough memory and storage would have made ordinary household products too expensive to sell.

The cloud solved a problem the hardware of the time could not solve. Putting demanding intelligence outside the home was a rational choice. The hardware limits that made it necessary have since changed.

Models have become smaller and more efficient, while hardware inside the home has become more capable. Together, those changes have made household-scale local intelligence practical.

The cloud solved the original cost problem by centralizing demanding computing and sharing it across many devices. That same idea can now be applied inside one household. A shared local system can serve many devices without putting the same computing power, memory, and storage into every camera, speaker, sensor, lock, and appliance.

Models came down

Much of the public discussion of AI has focused on growth: more parameters, larger training runs, and powerful systems concentrated in data centers. Researchers have also learned how to get more useful performance from smaller models. Better training data, improved model structures, knowledge distillation, pruning, quantization, and lower-precision computation have all reduced the resources required for particular levels of capability. A growing body of work now focuses on models designed specifically for mobile, embedded, and edge environments.¹¹

The relationship between model size and performance is not linear. A widely cited 2023 technical report showed that a model with 13 billion parameters outperformed a model with 175 billion parameters on most of the benchmarks included in the comparison.¹² The result does not mean the smaller model is better at every task. It shows that architecture, training, and data can improve performance without relying only on model size.

Researchers have since developed language models with fewer than one billion parameters for on-device uses including chat and software command execution. These models remain more limited than the strongest cloud systems, but they show that useful capability can be designed for constrained hardware from the beginning instead of cut down from a larger model.¹³

A home has a narrower set of tasks than a general-purpose cloud service. It does not need one model that can answer every possible question, create every kind of content, or reproduce all the knowledge available across the internet. It needs systems that perform particular tasks reliably: interpreting speech, understanding a room, recognizing an event, comparing it with prior context, and determining what deserves attention.

The system also serves one household, not millions of unrelated users. Its models can be selected and refined for the work that happens inside that home. A smaller model does not need universal capability if it performs those household tasks reliably.

But frontier scale is no longer the only path to useful intelligence.

Local hardware became more capable

At the same time, hardware inside the home has become more capable within the power, space, and operating limits of household products.

Conventional processors generally store information in memory and perform calculations somewhere else. Running a neural network requires large quantities of model data to move repeatedly between memory and the processor. As processing performance has increased, the rate at which data can move through memory systems has not kept pace.

Researchers describe this as the memory wall. In many AI workloads, particularly the serving of decoder-based models, memory bandwidth can become the limiting factor even when the processor has unused computing capacity. Performance depends not only on how many calculations a chip can perform, but also on how quickly model data can move between memory and the processor.¹⁴

Data movement also consumes energy. In a data center, that energy contributes to operating and cooling costs. In a small device, it competes directly with the power required to sense, communicate, and perform useful work.

Compute-in-memory addresses this problem by performing selected operations where the relevant data is stored, within or close to the memory array. Instead of repeatedly carrying model weights between separate memory and processing units, some of the computation occurs where those weights already reside. The objective is to reduce data movement and improve the amount of AI computation possible within a limited energy budget.¹⁵

Compute-in-memory describes a family of approaches rather than one finished architecture, using different memory technologies, methods of calculation, and levels of integration. These approaches still face significant engineering challenges involving precision, reliability, programming, manufacturing, and suitability for different workloads.¹⁵

Engineers are now actively redesigning the boundary between memory and processing. What once looked like a fixed limitation is increasingly becoming a design choice.

Household compute is within practical reach

Memory prices are rising rather than falling. Gartner has forecast a sharp increase in DRAM and solid-state drive costs through 2026, along with a growing share of a device's total component cost going to memory.¹⁶ A household system capable of running meaningful models and retaining local context will carry a real hardware cost. Even so, the necessary compute, memory, and storage are now practical when they are provided once for the household instead of repeated at full scale inside every device.

Placing the most demanding silicon, memory, and storage into each individual product made little economic sense when the industry first faced that choice, and that reasoning still holds. A camera does not need the same resources as a system expected to interpret information from across the household. Neither does a lock, a motion sensor, or a light.

Requiring every product to carry a large model would recreate the cost problem that drove intelligence to the cloud. The original idea was to avoid paying for maximum capability in every unit. Nothing about local intelligence requires us to abandon that principle.

A local system concentrates that cost at the household level instead of repeating it in every device. The household pays for one hardware system at a particular generation, along with its electricity, maintenance, and eventual replacement. The same resource can support multiple devices.

A cloud-based system spreads costs differently. Processing, network use, supporting infrastructure, and service operation continue for as long as devices send information and request decisions. As the number of devices, the frequency of perception, and the complexity of the models increase, the continuing workload grows with them.

Local operation does not make the work free. It moves a defined portion of the cost into hardware serving one household and limits that portion of the workload to the needs of that household. Both approaches have costs; they put those costs in different places.

A household system does not need to match a frontier data center. It can use models selected for household functions, lower-precision representations, and storage sized for the context of one home instead of millions of users. The hardware still carries a real cost, but building it has become a product-design problem rather than a physical impossibility.

At four bits per parameter, the stored weights of a seven-billion-parameter model require roughly 3.5 gigabytes, less than the memory in a mid-range gaming laptop or a high-end tablet. A working system requires additional memory for runtime operations, context, software, and other overhead. Even with that qualification, this is no longer a scale that belongs only inside a data center.

The Architecture

Model size is not the same as system design

Useful local intelligence can now run on a wide range of hardware. Models with millions of parameters can run on constrained devices close to sensors, while models in the single-digit billions can run on a larger local system serving the household. Those models still need a clear division of work.

The intelligent home requires two local levels of intelligence. The first sits inside individual devices and handles immediate perception and response. The second operates at the household level and combines information across devices, maintains context, and supports broader judgment.

This division reflects two different kinds of work, not just the limits of today’s hardware. Immediate decisions need to happen where sensing and action occur, while household-level judgment requires context from across the home that no single device can provide.

The system also has to do three things: act on what it perceives, remember context over time, and reach the devices the household already owns.

The home works like a body

The intelligent home is easier to understand through the structure of a body than through a diagram of connected products. Sensors at the edges work like neurons, taking in what is happening in one specific place. Immediate local decisions work like reflexes, acting before anything has to travel. Home intelligence works like the brain, holding context and making the judgments that require a view of the whole, while memory gives that judgment continuity across time. The cloud sits outside the body, a window to the outside world instead of a part of it.

Some decisions cannot leave the device

Device-level intelligence sits close to the sensor and the action it controls. It interprets a limited set of signals and makes immediate decisions within a defined area of responsibility. It may identify a change in the environment, classify what is happening, and select an appropriate local response.

These decisions cannot wait for a round trip to another system. Even a local household hub may be too far away when a response depends on timing, continuous feedback, or direct control. Sending every sensor reading across the home would also increase communication requirements and make simple functions dependent on another part of the system.

The capability therefore has to sit where the sensing happens. A device should be able to perform its essential functions without asking home intelligence to approve every step. This gives the system speed, continuity, and a clear division of responsibility.

Device-level intelligence is narrow by design. It needs enough perception, computation, and control to make reliable decisions within the device's own role, without a complete understanding of the household.

Smaller models fit this device-level role. Models with millions of parameters can fit within constrained hardware and support focused tasks near the sensor. Their smaller size fits the limited scope and fast response required for these tasks.

The device layer also requires close coordination between software and hardware. Sensors, models, memory, computation, power use, and control systems cannot be designed separately and joined at the end. The architecture must treat them as one local operating system for perception and response.

Some decisions no device can make alone

Home intelligence operates across the household. It runs models inside the home, keeps accumulated household context, and handles work that no individual device can complete because it requires information from more than one location, system, or period of time. It does not replace the intelligence inside devices.

A single device has a limited view. A camera sees one part of the environment. A lock reports another. Other devices may detect sound, movement, temperature, energy use, or the position of an autonomous system, but none of them sees the household as a whole.

Home intelligence combines these separate views. It can receive interpreted information from devices, compare conditions across systems, and determine whether several events are related. It can also coordinate actions when the correct response involves more than one device. Broader decisions happen at this level because a device may know what it senses and what it can do without knowing what is happening elsewhere in the home.

Without this layer, each device remains separate. A device may perform its own task well, but the home still does not operate as one system. Information remains separated, decisions may conflict, and each device must build its own limited understanding of the environment.

Raw information would still have to travel away from the point of sensing, and immediate actions would still depend on a separate system. The architecture therefore needs both levels: device intelligence for fast local perception and execution, home intelligence for context, coordination, and judgment across the wider system.

The boundary between the two levels can change as hardware improves. A decision stays on the device when the device has enough information to make it correctly. It moves to the household level when it needs information from elsewhere in the home.

Devices that cannot think for themselves

Most homes already contain years of older products built to report a reading and receive a command. A contact sensor knows whether a door is open. A plug knows how much power is passing through it. Neither was designed to interpret anything, and households will not replace them just to fit a new architecture.

Home intelligence can supply what those products lack. A legacy device may only need to report its state accurately. Interpretation, comparison against everything else happening in the house, and the decision about what should follow can be performed by a system with a wider view. The household gains an intelligent home without replacing the products inside it.

Existing devices can contribute more than their original function. A robot vacuum builds a map of the floors and carries cameras and sensors to navigate them. A speaker has microphones. These were bought for one purpose, and the information they already produce can serve others once a local system can read them together. Many of the needed sensors are already in the home, spread across products that cannot combine their information on their own.

Device-level intelligence still matters. A device that has to respond in the moment should carry the capability to do it, and the reasoning above still holds. These changes happen on different timelines. Device-level capability will improve as products are replaced, but households should not have to replace every device before the home can work as one system.

The home has to be able to act

An autonomous home must do more than interpret information. The local system has to perceive what is happening, decide what should happen next, and carry out that decision through the devices it controls.

Current smart home devices can already detect motion, sound, temperature, occupancy, and the state of the things around them. What most systems cannot do is plan across those signals. Connected products usually follow rules written in advance and run the same action whenever a trigger appears. Planning requires the system to decide what to do when the exact situation was not programmed beforehand, and sometimes to coordinate that decision across several devices over time.

The final step is action. The system may open something, delay an appliance, move a machine, or change a condition in a room without a person confirming each step. That raises the cost of a mistake. A bad notification wastes someone's attention. A bad action changes the physical environment before anyone can review it.

An autonomous home needs all three stages. If perception happens locally but the decision to act still happens elsewhere, the home still depends on that outside system. The home still waits on a connection before anything happens inside it.

Judgment needs continuity

Home intelligence also needs memory: stored context that connects current conditions with earlier interactions and events. Without it, every event would be interpreted in isolation, and the system would have limited ability to recognize a recurring pattern or understand how the present situation relates to earlier conditions.

In a 2024 interview with Geek Park, Steven Yang, CEO and founder of Anker Innovations, said a long context holding everything a person has seen, heard, and done over ten or twenty years should never sit in the cloud.¹⁷

Memory does not mean that every piece of information must be stored permanently. The architecture will need to distinguish between temporary operating context and information that remains useful over a longer period. The next chapter addresses what should be retained, who controls it, and why it should remain local.

Connection has to reach every connected device

For the home to work as one system, home intelligence must be able to reach the connected devices in the house.

People buy devices over many years for different reasons, so most homes contain a mix of brands.

Matter is an application-layer standard rather than a radio protocol. It defines how a device describes itself, reports state, receives commands, and authenticates with a controller, running over Ethernet, Wi-Fi, and Thread, with Bluetooth Low Energy used for commissioning.¹⁸ Devices expose themselves through a common data model, so a household system can receive structured telemetry from any certified product without a brand-specific integration behind it. The standard operates on the local network, so that exchange happens without the request leaving the house. This means home intelligence can communicate with any certified device in the house, whatever brand made it, without going through that manufacturer's servers.

If home intelligence required a separate integration with every manufacturer, it would only see the devices those integrations supported. Matter removes that constraint. It gives home intelligence one way to read and act on the devices a home actually contains.

Matter does not interpret the information it carries. A thermostat reports a temperature without knowing which rooms are occupied or whether a window is open upstairs, and a washing machine reports that a cycle has started without knowing whether the house is producing its own power at that hour. Answering those questions requires context that no device holds on its own.

Where the work happens

Home intelligence needs to run inside the home because the home depends on it for coordination and decisions. It must keep working when the external connection is unavailable. If household decisions happen remotely, information must leave the home and return before devices can coordinate. Home intelligence does not have to sit in one physical box; it can be distributed across several local computing resources as long as its core functions remain available inside the home. The cloud can support the system, but the home's core decisions should not depend on it.

The cloud still has important roles. Training large models requires computing resources and data infrastructure that do not belong inside an individual home. Once trained, models or smaller versions of them can be prepared for local use and distributed to household systems.

The cloud can also deliver model and firmware updates and connect people to their homes when they are away. Those functions do not require moving the home's routine decisions or stored context outside the house.

Some work will still need the cloud, either because the task is unusually demanding or because the home is running older hardware. A local-first design should not assume that every home can handle every future workload. Local placement should be the default, with work moving outward when the home cannot complete it and the household able to see that it happened.

In that case, the home can send out the specific task while keeping its accumulated memory inside the house. A system can send the specific question it cannot answer without also sending the years of household context behind it. The home can use outside computing without sending out the record of household life.

Why this is possible now

Smaller, more efficient models and more capable local hardware now make a new system design practical: focused models can run at the device level, while more capable models serve the household without depending on remote computing for every decision.

In this design, devices handle immediate perception and response. Home intelligence coordinates across the household, stores context from earlier events, and reaches every certified device in the house. The cloud remains connected while sitting outside the core decision loop, supplying scale, updates, training, remote access, and the occasional task the home cannot complete on its own.

An autonomous home also needs to store much more information about the household, often over years. Where that record lives is now a design choice, not a technical necessity. The people who live in the home should control it.

Who Owns Your Home's Memory

Tomorrow's autonomous home will need to know a great deal about the space and the people in it, far more than today's connected devices collect. An intelligent home cannot act on its own without that information.

What the home's memory contains

An autonomous home can accumulate highly personal information. It can include interior video and audio, showing what happens inside private rooms and recording the voices of the people who live there. It can include a physical map of the home: rooms, entrances, furniture, obstacles, and the paths people take through the space.

It can also include biometric identifiers. Face templates, palm and fingerprint data, and voice prints may be used to distinguish one person from another. These identifiers describe the person, not an account the person controls. That difference matters after a failure. A compromised password can be changed. A face, fingerprint, palm, or voice cannot.

The system also observes presence. Occupancy, arrivals, departures, and repeated daily routines create a detailed account of household life. No single event may appear significant, but the accumulated pattern reveals when people are home, how they move through the day, and when something changes.

The home can also infer information about the body. A system that knows when a person usually sleeps, how they move through the house, and how that has changed over months may be holding health information even though no one wore a device and no one set out to create a medical record.

Our research asked how comfortable people are with each of these categories sitting on a company's servers instead of staying in the house. In all seven, most people were not comfortable. Recordings of what is said in the home drew the most objection, with 74 percent not comfortable, followed by video from interior cameras at 70 percent. Presence and daily routines and the names, faces, and voices of family members each reached 65 percent.

Even the categories that drew the least objection were still opposed by a majority. Fifty-six percent were not comfortable with health and body information on company servers, and 58 percent said the same about a map of the home's layout. Both describe material the household never deliberately provided.

Security is not the same as control

The debate about household data has largely been a debate about security. Companies describe encryption, access controls, audits, and certifications, and those measures matter. Those measures protect information from unauthorized access. They do not decide who should hold it, who controls how it is used, or what a household can do when those decisions change.

Asked what concerns them most, respondents split almost evenly among unauthorized access at 28 percent, sharing with third parties at 27 percent, and data breaches at 25 percent. Excessive collection and lack of transparency together drew 17 percent. Most respondents described their concern in terms of security rather than control.

A system can be secure and still leave the people in the home with no control over what happens to their information. Policies get revised. Companies are acquired, restructured, or wound down. Terms change, and legal obligations differ by jurisdiction and shift over time. None of these events is a breach, and none requires anyone to act in bad faith. They simply mean that decisions about the record of a household's life are made by people the household will never meet.

A service holding interior recordings, biometric identifiers, health patterns, occupancy history, and physical maps from millions of homes becomes a much more valuable target than the data from any one household. A 2009 European information security assessment found that large centralized systems can apply security resources efficiently while also creating a more attractive target because they concentrate data and computing.⁵ Research on biometric systems similarly identifies the stored template database as one of the most damaging points of attack in the system.¹⁹

Local storage does not remove risk, but it limits the scale of a breach. A failure inside one household is serious for that household, and it does not expose every other household using the same product. The architecture stops turning the combined memory of millions of homes into a single object worth attacking.

Household data can remain sensitive for decades. A face template captured this year can identify the same person twenty years from now, and a record of when a house sits empty can reveal habits long after those habits change. Security that is adequate today may not remain adequate for as long as the information is stored.

Biometric identifiers cannot simply be changed after a breach the way a password can. Standard encryption is also harder to apply to biometric matching because two readings of the same face or finger are never identical and matching often requires access to the underlying template. Researchers have long identified the difficulty of protecting biometric templates while preserving matching accuracy.¹⁹

Information gathered for one function can also be used for another, a problem the biometric literature calls function creep: an identifier collected to unlock a door can later be used to search an unrelated database or link a person to records they never connected.¹⁹ A household that agreed to be recognized by its own front door has not agreed to everything that recognition could eventually support.

Households should also control where this information is stored. Keeping household memory in the home does not make local systems invulnerable, but local processing is already used when sensitive material should not leave a site. Production video systems, for example, may process footage on premises for that reason.¹⁰ The home deserves at least the same treatment.

The memory belongs to the household

It is not aggregated into a central service, and it is not training data for someone else's model. The information exists to serve the household that created it.

Household material should stay in the home unless a person chooses to send it out. Continuous outward transfer should not be the normal price of intelligence.

Control also means being able to see what has been stored, delete what is no longer wanted, and decide what may be shared. If people cannot see, manage, or delete stored memory, they do not have meaningful control over it.

What people know and what they do not

Most people interact with connected devices through simple functions, applications, and account settings. They may know what a device does without knowing everything it records, where that information goes, or how long it is kept.

Respondents had very different ideas about where their information is stored. Thirty-seven percent said the information sits both on devices in the home and on company servers. Twenty-eight percent said it goes to company servers alone. Eighteen percent believed it stays on the devices inside the house, and another 18 percent said they did not know.

Most had not known what these systems collect. Shown the full range of what these devices can capture, 57 percent said they had not been very aware of it, and 27 percent said they had been only slightly aware or not aware at all.

Where people believe the memory should live

Twenty-eight percent said it could be stored on company servers as long as it was kept secure. The remaining ten percent had no preference or were unsure.

Presented with a device that does everything it does today while keeping sensitive information in the house, 79 percent said that would matter to them a great deal or quite a lot. Sixty-eight percent said they would pay more for it.

The survey did not test technical knowledge. It asked where people wanted household information to be stored, and the majority preferred to keep it under household control.

Knowledge can come in without the home's memory going out

A useful autonomous home still needs access to the outside world. A local system can use current information and connect people with news, entertainment, shopping, and services. The rule applies to stored household context, not to ordinary information people choose to receive from the internet.

When a person buys something online, they choose to send an order, a delivery address, and payment information. The service receives what it needs to complete that transaction.

The rest of the household context does not need to leave the home. The recording of the request, the identity of everyone present, the pattern of earlier conversations, and the household context used to understand what was meant can all remain inside the home. Only the information needed for the order needs to leave the home.

The system can do the same with other external services. It can send a selected query, request, command, or transaction without sending the stored household context used to understand it. The home can stay connected without sending its accumulated memory to every service it uses.

What the home's memory becomes

AI systems may also store memory differently in the future. Systems today keep the record outside the model and place retrieved material into the model's input when a question arrives. The approach works, and its limits are documented: longer inputs raise computation and latency, performance can degrade on tasks that require complex reasoning, and material supplied through the input is handled differently from the knowledge already held in the model's parameters.²⁰ Research has begun moving retrieved material directly into those parameters instead.²⁰

In a home, this could mean that household context becomes part of the model itself rather than a separate file the model retrieves. Years of household context could then influence the model directly, without a separate retrieval step selecting which records to provide.

A record kept in a database can be exported, audited, and deleted using established tools. Memory embedded in a model is harder to extract, return, or remove. If household history becomes part of a model itself, keeping that model inside the home avoids giving a remote provider control over information that may be difficult to separate from the system that learned it.

The Autonomous Home

In the future, smart homes will be defined by whether the home can operate as one intelligent system, and not by how many devices are connected. Today, a security camera, smart thermostat, robotic appliance, or home backup may all share the same network while knowing almost nothing about one another. They are connected, but they are not working from a shared understanding.

A shared network is not enough. The individual products must be able to perceive and act for themselves, but they must also contribute to an intelligence that understands the household around them.

An autonomous home needs a local system that combines those separate views and remembers context over time. A home that knows what is ordinary in this particular household can recognize what is not. The same event means something different depending on who is present, the hour, the place, and what happened before.

Many of these benefits do not depend on future robots. Home intelligence that can read those devices together can hold back a wash cycle until the house is running on its own stored power, stop heating a room that is venting to the outside, or recognize that the house is empty at an hour when it usually is not. None of those decisions needs new hardware. They need something in the house able to read products that were bought years apart, from different companies, for unrelated reasons.

The household stops writing the rules

The smart home asked people to become programmers of their own houses. Every useful behavior had to be specified in advance: a trigger, a condition, and an action, entered by hand in an application and revised whenever life changed. Households that never did this work owned connected products that waited for a button. Households that did it well ended up maintaining a small and brittle system of their own making.

A home that perceives and plans locally does not need that work. It observes what normally happens in the home and can decide what to do without someone having written a rule for that exact situation. The household describes outcomes and limits in ordinary language, and the system turns those instructions into device actions.

People could also spend less time configuring, naming, grouping, and maintaining products that cannot understand one another.

Robots that can see beyond themselves

Specialized robots will continue to perform repeated tasks such as cleaning floors or mowing lawns, while more capable machines take on broader forms of physical work. The home is unlikely to contain one perfect machine that replaces all of them.

A robot working alone depends on the sensors, cameras, maps, and models it carries. That may be enough for a defined task in a limited area. As robots become more autonomous and move through more of the household, their own view will no longer be enough.

They will need information from the rest of the home. Cameras may see into rooms beyond the robot's current position. Door and window contacts may show which routes are available. Motion sensors may indicate where people are moving, while the household map provides an understanding of spaces the robot has not yet entered. The home may also know that another machine is already working nearby, that a room should not be entered, or that a normal route is temporarily blocked. A robot drawing on that shared context does not have to rediscover the entire home through its own sensors every time it moves.

A floor cleaner, lawn machine, mobile security system, and future general-purpose robot will each have different capabilities and limits. Without shared home intelligence, they remain separate systems operating in the same place, and the household is left to coordinate them.

Who authorizes the home to act

Not every product needs the same level of intelligence. A microwave does not need the same model as a household agent, and a door contact does not need the same computing power as a robot. Each product only needs enough capability for its own role, while home intelligence coordinates across the system.

But an autonomous home also needs clear rules about permission. People in the home should set clear limits on what the system may do on its own, what requires approval, and which actions are never automatic.

The same people who control the home's memory should also control what the home is allowed to do. They should be able to see what has been stored, delete what they do not want, and decide which actions the system may take without asking.

If core decisions are controlled by a remote service, that service can also change the limits on what the system does. The household may not notice until the behavior changes. The household should retain the ability to set and change those limits.

What the industry should adopt

We believe the industry should adopt a common standard for household control.

Asked who should control the information a smart home collects, 55 percent said the people who live in the home, and a further 21 percent said it depends on the type of information. Six percent said the company that makes the device. The survey supports a standard built around household control. The architecture described in this paper makes that standard practical.

The home's accumulated memory should stay in the home by default. Interior recordings, biometric identifiers, spatial maps, presence patterns, and health information should not leave the household as a condition of the product working. Information should leave only when a person chooses to share it for a specific purpose.

People should be able to see what the system has stored, delete it, and decide whether anything may be shared.

These controls should be built into the product, not left only to a privacy policy. A local architecture can be examined, and independent testing can verify what leaves the home.

The home becomes one system

The first smart shome era connected products to networks, applications, and remote services, but it did not give the home a shared local understanding of itself. We ended up with a collection of useful devices rather than one coordinated home, and each new category inherited the same design assumption.

Homes can interpret events in context, coordinate machines that would otherwise work blind, and act within limits the household has set. That requires local intelligence and local memory that build an understanding of one home over time.

The terms the industry accepted were set by the hardware of 2010, and that hardware is gone. What remains is the arrangement it produced, still in place, still shaping what each new product assumes about where the home's understanding belongs.

This paper opened with two questions: Where should the home's memory live, and who should control it? The first has a technical answer: the memory can live in the house, on hardware a household can buy, running models small enough to work there. The second is a choice the industry makes product by product. We believe the people who live in the home should control the record of life inside it.

Letter from Founder and CEO, Steven Yang

Research Methodology

References

1. 1 Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 25. ↩
2. 2 Wu, J., Leng, C., Wang, Y., Hu, Q., & Cheng, J. (2016). Quantized convolutional neural networks for mobile devices. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4820–4828. ↩
3. 3 Mell, P., & Grance, T. (2011). The NIST definition of cloud computing (NIST Special Publication 800-145). National Institute of Standards and Technology. ↩
4. 4 World Economic Forum. (2011). Personal data: The emergence of a new asset class. World Economic Forum. ↩
5. 5 European Network and Information Security Agency. (2009). Cloud computing: Benefits, risks and recommendations for information security. ENISA. ↩
6. 6 World Economic Forum. (2025). AI agents: What are they and how will they impact business? World Economic Forum. ↩
7. 7 NVIDIA. (2024). What is physical AI? NVIDIA Blog. ↩
8. 8 Paolo, EAD, Buhrmann, T., & Barandiaran, XE (2017). Sensorimotor life: An enactive proposal . Oxford University Press. ↩
9. 9 Varghese, B., & Buyya, R. (2018). Next generation cloud computing: New trends and research directions. Future Generation Computer Systems , 79, 849–861. ↩
10. 10 Bhardwaj, K., Miranda, M., & Gavrilovska, A. (2022). Ekya: Continuous learning of video analytics models on edge compute servers. 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22) , 119–135. ↩
11. 11 Nguyen, T., et al. (2024). Small language models: Survey, measurements, and insights. arXiv preprint arXiv:2409.15790 . ↩
12. 12 Touvron, H., et al. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 . ↩
13. 13 Liu, Y., et al. (2024). MobileLLM: Optimizing sub-billion parameter language models for on-device use cases. arXiv preprint arXiv:2402.14905 . ↩
14. 14 Gholami, A., et al. (2024). AI and memory wall. IEEE Micro , 44(3), 33–39. ↩
15. 15 Sun, Z., et al. (2023). When in-memory computing meets spiking neural networks — A perspective on device-circuit-system-and application co-design. Advanced Intelligent Systems , 5(6), 2200339. ↩
16. 16 Gartner. (2024). Forecast: Semiconductor components, worldwide, 2022–2028, 3Q24 update . Gartner. ↩
17. 17 Yang, S. (2024, April 25). Interview with Geek Park [Video interview]. Geek Park. ↩
18. 18 Connectivity Standards Alliance. (2023). Matter specification version 1.2 . Connectivity Standards Alliance. ↩
19. 19 Jain, AK, Ross, A., & Prabhakar, S. (2004). An introduction to biometric recognition. IEEE Transactions on Circuits and Systems for Video Technology , 14(1), 4–20. ↩
20. 20 Su, Y., et al. (2024). Parametric retrieval augmented generation. arXiv preprint arXiv:2501.15915 . ↩