This is a project that we developed in one month, during the [2023 Silicon Valley Winter Boot Camp]. The title of the project and the service is BUYSELF. The service is AI-based self-checkout service.
What is BUYSELF?
We found the problem while choosing the theme. The number of self-service machines in large marts is increasing, but when there are many products to pay for, we end up lining up at counters with cashiers. We attributed the problem to slow computation speed due to immature barcode recognition technology.
Our solution is recognizing all the products at once by a single photo which is photographed by a camera attached to counter, instead of one-by-one by barcode. This makes it quicker and easier to finish the payment process.
DEMO
We implemented the above-mentioned solution. To see how it works, watch the demo video.
1. Search for products and add products to checklist
1) You can see images of products sold at that store.
2) If a product is not displayed, you can search for it
3) The products to be bought can be added to a checklist in advance. The checklist can be loaded at the time of payment. Following these steps, you will be able to more easily buy everything you want to buy, without leaving anything out, right?
2. Scan products
1) Putting goods on the counter, you can scan the products all together by pressing the camera button.
2) The products in the photo are recognized by the AI. When a product is recognized, it is removed from the checklist.
3) When you are ready to pay, you can press the payment button, and the total amount will be displayed. After confirming the price, you can make the payment by pressing the product payment button.
3. Payment with Kakao Pay
System Architecture
We used NginX as the web server and Gunicorn as the Web Server Gateway Interface (WSGI).
When the API request is received, it is stored in a RabbitMQ queue, and Celery workers connected to the queue handle the requests one by one.
The next section includes a detailed explanation of each tech stack.
Tech Stack
1. Front-end
React is a JavaScript library for building user interfaces.
Typescript is a statically typed language.
CSS-in-Js library for React
Redux is a predictable state container for JavaScript apps. The toolkit includes development tools officially provided by Redux to make Redux easier to use.
We have made it easy to manage the status of recognized product lists and checklists.
2. Back-end
Flask is a micro web framework written in Python.
S3 Bucket is a public cloud storage resource.
MySQL is an open-source relational database management system.
Cache
Non-relational type database for Cache Server
Java open-source distributed search engine based on Apache Lucene
Monitoring
Swagger: API Documentation
3. AI
Object detection Model for product recognition
We tried training the system with 7 products. As a result of the training, as shown in the above graph indicators, the loss converged to zero and the metric converged to 1, confirming that the AI learned the information properly.
