Exploring the Frontiers of Retail: A Glimpse into AI's Potential with Our Vision Retail App

[Mykhaylo]

 Introduction

As the retail industry navigates through a period of digital transformation, artificial intelligence (AI) stands out as a key player in shaping the future of retail operations and customer experiences. Vision Retail is an emerging app that exemplifies the current capabilities of AI in the retail space. While it may not boast perfect precision, Vision Retail is a testament to the progress and potential of AI technology. It offers retailers a glimpse into the future with its product recognition features, suggestions for smarter product placement, and the ability to connect with online marketplaces. This app serves as a showcase for the practical applications of AI in retail, providing valuable insights despite its evolving accuracy. Through this article, we will explore how Vision Retail is paving the way for AI's role in retail, highlighting both its current functionalities and its promise for continued innovation

Overview of the process

1. The user chooses how to upload the image, either from the camera or the gallery
   
   
          
2. The image gets uploaded and analyzed   
   
                                                
    
3. The app shows the image report including: Detailed image description, the identified products including their amount, an improvement suggestion, and a list with the most similar items found on the Japanese marketplace from Yahoo

4. The user can save the image scan and later open it again from the home page

   
   Scan results

   Input Image:
   

    

   Description: The image shows a well-organized beverage shelf in a convenience store. The shelf contains various bottled drinks, including coffee, tea, and other beverages. The top shelf has 'FIRE ONEDAY Latte' and 'FIRE ONEDAY Black' bottles. The second shelf features 'Starbucks' branded drinks and 'BOSS' coffee. The third shelf includes 'GEORGIA THE LATTE' and 'GEORGIA THE BLACK' bottles, along with 'COSTA' coffee. The bottom shelf displays '綾鷹' green tea bottles. The shelves are labeled with price tags in Japanese.

   Improvement Suggestion: The product placement is generally good, but there are a few areas for improvement. The middle shelf has some empty spaces that could be filled to avoid gaps and make the display look fuller. Additionally, ensuring that all labels are facing forward can enhance the visual appeal and make it easier for customers to identify products. No security or health code violations are visible.

   Identified Products: 
   FIRE ONEDAY Latte: 6
   FIRE ONEDAY Black: 3
   Starbucks Coffee: 6
   BOSS Coffee: 6
   GEORGIA THE LATTE: 6
   GEORGIA THE BLACK: 3
   COSTA Coffee: 6
   綾鷹 Green Tea: 6  

   Possible found products:

    

    

    Out of 14 types of products the app was able to recognize correctly 8 which is 58%, it didn't recognized correctly the different types of tea on the bottom side which were poorly visible. Out of the 8 recognized products it found correctly 6 of them on the marketplace, leaving us with 42% of products being  correctly recognized and found on the marketplace.

   The app is particulary good at recognizing and searching products where the name is written in Latin alphabet, like in this case the coffee bottles. The model used for the recognition (GPT4-o) still has limited support for other languages rather than English for its Vision capabilities.

   Let's see more in detail how the app works.

   App Architecture

   
    

    

   Vision Retail is an iOS native app build with SwiftUI.
   When the user uploads the image, it gets uploaded with a POST request to the Image Processor through the REST API which analyses it with help of GPT-4o model in SAP GENAI Hub to retrieve infos like these:

   - Image title
   - Detailed image description
   - Improvement suggestion
   - Products list with properties like name, brand, color, package, material, size and amount.

   The products list gets send to the product search engine, it searches every product using the Japanese Yahoo Marketplace API.

   The problem is that the API gives a response with multiple articles, to choose the best match we again use The GPT-4o model from SAP GENAI Hub. The search engine analyzes the response and searches the best match considering all the properties mentioned above. After all the job is done the Image processor builds a package and send it back.

   Demonstration Video

   https://video.sap.com/media/t/1_g6uny5ge 

   Possible future use cases

   1. Inventory Management: Automatically update inventory levels based on the analysis of the products in the images. This can help in restocking and inventory tracking.
   2. Shelf Organization: Provide suggestions for optimal shelf organization based on the analysis of product placement and customer behavior.
   3. Sales Forecasting: Use historical data combined with the AI analysis to predict future sales trends for products and suggest better product placement.
   4. Customer Insights: Analyze the images to gain insights into customer preferences and shopping patterns.
   5. Loss Prevention: Identify potential theft or misplacement of items by comparing expected product placement with the actual shelf state.
   6. Compliance Monitoring: Monitor for general planogram compliance, such as ensuring that promotional endcaps are set up correctly or that the correct category of products is present in a designated area.
   7. Trend Analysis: Detect emerging trends in product popularity and stock levels.
   8. Automated Ordering: Based on the analysis, automate the reordering process for products that are running low.
   9. Dynamic Pricing Recommendations: Analyze the data to suggest dynamic pricing strategies, such as discounts on overstocked items or price adjustments based on demand.
   10. Health and Safety Compliance: Monitor for any health and safety violations, such as blocked fire exits or aisles, expired products, damaged packaging etc.
   11. Energy Consumption Optimization: Analyze store activity and equipment usage to provide recommendations for reducing energy consumption during low-traffic times.

   Conclusion

   While Vision Retail demonstrates the potential applications of AI in retail, it also highlights the current limitations of such technology. The app's performance metrics suggest that there is room for growth and optimization. As AI technology continues to evolve, it is likely that the accuracy and reliability of apps like Vision Retail will improve, offering more robust solutions for retailers looking to integrate AI into their operations.

   In summary, Vision Retail is an example of how AI can be applied to the retail sector, offering tools for product recognition and inventory management. Its current capabilities and limitations provide insight into the state of AI in retail, and its development trajectory points to a future where AI could play an increasingly significant role in the industry.

 

[Murali_Shanmugham]

Nice work