Instacart is building what it calls a “grocery world model” — a continuous learning AI system that connects physical store data from its Caper Cart smart shopping carts with its online commerce platform — in collaboration with NVIDIA.
The initiative represents how it intends to use its scale across 2,200-plus retail banners and 100,000-plus store locations to close the gap between in-store and digital grocery experiences.
The system combines NVIDIA Jetson edge computing on each Caper Cart with cloud-based AI infrastructure – including NVIDIA Dynamo-Triton for online item ranking and retail media – into a single, GPU-native platform designed to optimize decisions across both environments in real time.
What Caper Carts capture
Each Caper Cart carries basket-facing cameras, a weights-and-measures-certified scale, location-tracking systems and side-facing cameras. The carts run sensor fusion algorithms on NVIDIA Jetson hardware, processing weight, visual and location signals simultaneously to build an accurate picture of what is in the basket, where the cart is in the store, and what is on nearby shelves.
Instacart said thousands of Caper Carts are deployed across more than 100 cities, tripling year-over-year. The system updates its understanding of each store’s shelves as frequently as every hour, building what the company describes as a near real-time view of shelf conditions and item placement – independent of retailer planograms, which are often incomplete or out of date.
The sensor fusion problem is technically demanding. Weight signals take one to three seconds to stabilize after cart movement. Cameras are frequently blocked by hands, bags or stacked items. The company uses dual cameras to triangulate item location in 3D space, with weight signals acting as what it calls an “X-ray” of basket contents when visual signals fail.
Recommendations tied to context and location
Instacart’s recommendation engine combines signals from more than 1.6 billion lifetime grocery orders with real-time in-store data – basket contents, item removals, cart location and shelf visibility – to time recommendations to moments when shoppers are most likely to act. The company said prompts such as “Got everything you need?” are driving a nearly 1 percentage point lift in average basket size.
A recent ranking improvement using online data signals drove more than 1 percent basket lift on Caper Carts, the company said. Future development will use in-store Caper Cart data to personalize online shopping experiences – creating what Instacart describes as a data flywheel between physical and digital channels.
Online infrastructure gains
On the cloud side, migrating workloads from CPU to GPU serving using NVIDIA Dynamo-Triton decreased latency for whole-page ranking by 65 percent and item ranking by 40 percent. A transition to a transformer-based architecture increased click-through on sponsored products by more than 5 percent and corresponding retail media revenue.
Instacart also announced Cart Assistant, an omnichannel chatbot shopping companion available across the Instacart app, Storefront Pro and Caper Carts. The tool draws on purchase history, grocery lists, household preferences and in-store cart location to assist with meal planning, dietary guidance and list building.
NVIDIA’s perspective
Azita Martin, VP and GM of Retail and CPG at NVIDIA, said the grocery environment represents one of the most complex real-world AI deployment challenges in retail. “Instacart’s approach — combining edge computing, accelerated AI infrastructure, and deep marketplace data — unifies online and in-store intelligence by processing signals at the edge and scaling intelligence in the cloud to lay the foundation for the next era of omnichannel retail,” Martin said.
Long-term vision
Instacart’s stated goal is an AI system that understands not just individual products but the relationships between products, customer decision-making patterns and how stores operate commercially and physically. The company envisions deploying AI expert agents across catalog intelligence, inventory management, store operations, recommendations and logistics – capable of responding to natural language queries from store managers or autonomously coordinating with suppliers to optimize assortment.
For retailers, the company said Physical AI can drive larger baskets, higher retail media revenue, reduced shrink and fewer out-of-stocks. Instacart’s catalog engine has tagged more than 1.3 billion data points across its retail partner network, providing a data foundation the company said no competitor can replicate at equivalent scale.
[RELATED: Fareway Launches Online Pickup Through Instacart Partnership]