Case Study
Inventory management using 3D detection
on the Matterport platform
Company |
Industry |
 |
Facility management |
About
The goal of this project was to develop a computer vision model capable of automatically identifying and counting specific items, such as chairs, tables, fire extinguishers, lamps, and wall sockets - within digital twins (3D models) of physical stores on the Matterport platform. This solution was designed to improve facility and inventory management for a state-wide brand.
One major challenge was the lack of labeled data specific to the brand's stores, combined with a limited number of recorded stores. To resolve this, we used multiple open datasets with images of the target objects and developed a custom model trained on these images. This model could effectively detect and localize 3D objects within the Matterport platform, overcoming data limitations while meeting the project’s objectives.
Problem
We collaborated with an asset inventory management company that services brick-and-mortar stores. They were in the process of creating digital twins of all the stores and wanted to improve their processes of inventory management. Their goal was to answer key operational questions, such as:
- If I want to rebrand my stores, how many couches with the brand logo would I have to change?
- If I needed to service fire extinguishers, how much would that cost me?
- If I upgrade TV setups in the stores, how many TVs do I have to buy? Where are they located? Do only employees or everyone use them?
To meet these needs, we developed a system that enabled the inventory management company to count, locate and track the number and state of the inventory using AI models. This system replaced a lot of manual work (both visiting the stores and manual counting in 3D models) required in inventory management.
Challenges
Lack of labeled data
Our client had only a limited amount of recorded stores, and these stores were not labeled. Also, the process of labeling 3D images is tedious and can take a lot of time.
Integration with the Matterport platform
We had to figure out how the platform works, how to fetch the 3D model, understand the format of the 3D model, and create an interface where we could inspect and visualize the predictions. The 3D models consist of images, depth maps, and images that are mapped to 3D space.
Usage of image (2D) dataset to build a 3D model
We had to use images from multiple datasets to obtain enough examples of our use cases. Then we built a model that can detect and localize such objects on images. To use such a model on 3D spaces, we had to adjust a model for 3D spaces, create a custom pipeline, and merge 2D predictions into 3D detection with our custom algorithm.
Solution
Our solution was to build a custom model trained on plain 2D images, adjust it for 3D spaces, and create a custom pipeline that can detect and localize 3D objects in 3D spaces. We integrated our solution with Matterport Spaces, allowing for easier inventory management with significantly less manual work.
Tools and Technologies
Matterport
Python
PyTorch
OpenGL
Results
We developed a 3D object recognition system tailored for inventory management, enabling managers to track item quantities, locations, and conditions within spaces like stores and building floors. Larger items are logged with condition details to help predict maintenance costs. Our demo showcases accurate object identification and 3D mapping, allowing for report generation and historical inventory records to support more informed planning.
Discover More Case Studies
Discover the impact of our custom AI solutions on business success through customer stories
