Collaborate with MindInventory
Planning a dream home interior can be a hassle-bustle process. It is because selecting the right hues that harmonize nicely with natural lighting and home decor plays a crucial role in deciding the look and feel of the home.
This common dilemma sparked an innovative idea: “why not try out the paint virtually before committing?” The Passio.AI team (that raised $3M investment), with roots in the Stanford Research Institute and a proven track record in AI-powered mobile solutions, saw an opportunity to blend augmented reality with advanced computer vision.
Their vision was clear: to build a seamless, real-time, AI-powered color visualization solution that allows users to digitally experiment with colors and textures in their bedroom, office, wash area, and kitchen. The solution also identifies wall surface types, including concrete, wood, steel, and gypsum, while recognizing wall defects or damage types such as cracks, holes, and flaking.
As leaders in AI innovation, they developed the Paints AI model. However, their SDK faced difficulties in visualizing how a particular color would interact with natural lighting, existing furniture, and other design elements.
Even after multiple trial and error approaches, they frequently encountered challenges such as:
Traditional paint swatches and static images failed to capture the dynamic interplay of light and color within a room. This limitation made it difficult for clients to make Paints AI confidently choose a paint (with access to real paint colors from well-known brands) that would complement the overall ambiance of their spaces.
Accurate mapping of wall surfaces was a significant hurdle. Existing methods could not efficiently translate a room's architectural nuances into a digital model, complicating the process of applying colors and textures virtually.
Not all surfaces are smooth, crack-free, or perfect in every way. Some may have cracks, breakages, and stains. In the traditional Paints AI model, these anomalies went unnoticed, leading to imprecise paint applications and extra repair costs when opting for actual paint application.
The traditional Paints AI application struggled to differentiate between the surface and nearby objects. As a result, it often applied colors to both the wall and adjacent elements, which should not have happened.
We didn’t just work for Passio.AI; we partnered with them to enhance their Paints AI and integrate it into their Remodel-AI SDK. We recognized that achieving an advanced Paints AI solution required a multi-aspect approach:
Legacy Swatch: This approach allows users to modify painted wall objects by selecting specific wall squares and applying paint. It also enables area size measurement for more precise customization.
Floor Plan Approach: By leveraging computer vision techniques, this method detects corners and walls, allowing users to map the space accurately. GLSL is used for rendering AR walls, and users can measure the area size for better planning.
Room Plan (iOS Only): This feature lets users scan an entire room using the app camera to generate a detailed 3D geometry model. It can differentiate wall surfaces from objects placed on or beside them, ensuring precise paint application. Users can also measure area size within this approach.
Color Range: Using image graphics processing with shader languages like OpenGL ES, this approach scans walls, identifies colors, and enables users to change specific parts. However, it does not support area size measurement.
Once a wall is painted using any of these approaches, users can further refine their selections using the edit option, occluding specific areas or extending the painted region within the same wall.
We modified the AR models provided by the Passio team for different business applications. Upon that, our team also developed the AR/VR functionality for the Android platform, especially for the Paints AI and Remodel AI modules, which let users take a virtual tour of the property and get a comprehensive view of interior spaces. This enables users to jot down room corners, visualize a complete 3D model, and experiment with colors and textures on the surface for better selection.
Paints AI is designed to scale across multiple platforms and seamlessly integrates into a broader ecosystem of home remodeling tools. This includes its compatibility with both iOS and Android devices and its potential to integrate with other enterprise systems, reinforcing its value as a scalable solution.
Understanding the importance of precision and high-quality rendering, we trained the solution to produce photorealistic previews and ensure that lighting, texture, and spatial dynamics are rendered with exceptional accuracy, elevating the customer experience.
We also assisted in the development of the Segment Anything Model (SAM), an AI model that allows users to select any pre-captured image from their gallery and receive high-quality object masks. As a result, they could virtually “try on” new paints (from Multiple Color Palettes) on wooden objects, such as wooden furniture, along with walls with exceptional accuracy.
SAM is trained on a dataset of 11 million images and 1.1 billion masks, has strong zero-shot performance in segmentation tasks, and offers great accuracy.
Early tests revealed a challenge: paint colors were sometimes bleeding onto nearby objects. Our team of skilled 2D/3D artists stepped in to generate synthetic data, which helped train the computer vision model to distinguish the surface from objects. This crucial enhancement ensured that color and texture filters were applied only where they were needed.
Our solution incorporated computer vision techniques using Python and Google AI to analyze live video feeds. This enabled the detection of imperfections, like cracks, breakages, and holes, and even gauged the depth of the damage. With these insights, the system could recommend the precise amount of repair material, ensuring that every home surface, from interior to exterior, looks flawless before the application of the new paint.
The solution offers curated paint recommendations from well-known, reputable brands. This feature enables users to explore and compare various color options from industry leaders, ensuring that they receive high-quality, trusted suggestions for their home remodeling needs with cross-brand comparison for better decision-making. The solution is also integrated with purchase links of paints, which saves their time considerably.
Paints AI is not limited to indoor spaces. The solution also offers visualization for exterior surfaces, allowing users to experiment with paint options on both interior walls and building exteriors. This dual capability provides a comprehensive view, ensuring that users can see how different paint choices perform under various lighting conditions and environmental factors.
With their Paints AI SDK, now powered by our advanced Paints AI, the company has redefined home remodeling. Not only did this technology empower customers of Jazeera Paints (a leading paints company in the Middle East) to visualize and choose colors with unprecedented ease, but it also positioned Passio.AI as a true innovator in the field.
