Safety1st is a company that manufactures electronic devices designed to simplify the implementation of AI/ML technologies and enable interactions with the environment based on data processing results. Our customer product SmartColumn can significantly improve the safety of employees in the construction and manufacturing industries. The device, by verifying the personal protective equipment, can prevent the commencement of work (e.g. blocking the entrance) or inform the occupational health and safety services about detected irregularities. The device can verify whether the employee who wants to get into the room has e.g. the appropriate uniform. In case of detection of irregularities it signals and informs appropriate services.
For the demo purpose they integrated in their SmartColumn device, an example application that can be used in most of the public spaces where during the Covid-19 time people are obligated to wear face-masks. Initially their facemask detection model was trained based on public dataset but very fast they identified they have lack of accuracy in the real environment.
As you can read in one of our blog post about dataset importance, data used to train effective object detection models are extremely important, of course in laboratory conditions very good results can be achieved and the same is true for mask detection models. We can often see in various scientific publications or company demonstrations, students or company employees who serve as extras to test the models in an ideal environment. Is this what we will find in the real implementation of the product? It is not difficult to answer this question.
The facemaks detection model trained on a public data set was implemented in the first version of the application and, together with their SmartColumn device, delivered to the first customer, which was a car dealership. As it quickly turned out, in these conditions the operation of the application and the device was not effective.
Car showrooms are designed using large areas of exterior walls made of glass, so that what is inside can be seen as easily as possible. However, for a camera used for facemask detection both too much and too little light entering the lens is quite a challenge. This ultimately resulted in very poor object detection and tracking results, with a large number of false detections, which made our customer’s device unusable.
Safety1st turned to us for help because time was pressing and their device was not profitable for them. Together we setup action plan where we identified the problem, determined the minimum requirements for proper operation of the device in terms of object detection based on the collected samples and then determined the size of the data set to be determined. They gathered around fifteen thousand photos from their smart devices and used us in defining annotation strategy and execution. For this project we decided to use bounding boxes, rectangles that are used as a point of reference for object detection in computer vision applications. The main target for drawing bounding boxes is to enclose the object of interest and assign a class to it (eg. person, car, cup). Bounding boxes help algorithms to find what they’re looking for in an image and associate the detected object with what they were initially trained for. In the end, when Safety1st trained its model on the dataset we prepared, the model’s performance under production conditions reached about 90% precision. The smart column device was able to perform its task and generate revenue for our client!
This case shows very well that you can be the best in designing and building electronic devices, and we believe that our client is exactly like that, but the lack of knowledge in carrying out projects in the field of AI/ML, especially in the preparation of data can be destructive to the whole concept of business. Therefore, it is not worth learning from your own mistakes, but use the professional knowledge in this area, which, as it turns out, can be crucial even in the case of demonstration projects!
