Skip to content
Case Studycase studyfacemask detectionobject detection

Safety 1st – Facemask Detection Case Study

Chris from BoBoxUpdated 5 min read
Facemask detection case study safety system

TL;DR — Safety1st's facemask detection model looked great in the lab but produced so many false positives at a real car dealership that the device was unusable. BoBox rebuilt the training dataset from about 15,000 photos taken by the devices on site, and the retrained model reached roughly 90% precision in production.

At a glance

  • Client: Safety1st — electronic devices that make AI/ML deployment simpler
  • Product: SmartColumn — workplace safety device (PPE verification, access control, alerts to occupational health services)
  • Task: facemask detection in a public space during COVID-19
  • Deployment site: car dealership showroom
  • Dataset: ~15,000 on-site photos, bounding-box annotation
  • Result: ~90% precision in production

Who is Safety1st and what is SmartColumn?

Safety1st manufactures electronic devices that simplify AI/ML implementation. Their SmartColumn product improves workplace safety in construction and manufacturing environments: it verifies personal protective equipment, blocks unauthorised access and alerts occupational health services to irregularities.

What went wrong in production?

During COVID-19, Safety1st added a facemask detection application to SmartColumn for public spaces that required masks. The model was trained on publicly available datasets and performed well in laboratory conditions.

The real installation — a car dealership — told a different story:

  • Glass-walled showrooms created both too much and too little light
  • Ceiling spotlights cast harsh shadows on faces from unusual angles
  • Real-world variety — different demographics, mask types and ways of wearing them

The result was a high number of false positives, which made the device ineffective and undermined the client's confidence in the product.

Why did a model that worked in the lab fail on site?

Because the training data didn't represent the deployment environment. Public datasets are:

  • Collected in controlled conditions
  • Biased toward certain demographics and lighting scenarios
  • Missing the specific edge cases of your site

This lab-to-production gap is one of the most common failure modes of AI products — the topic of our article AI Model Is Only as Good as the Training Dataset.

How did BoBox fix it?

Safety1st engaged BoBox to resolve the performance problem. Together we followed five steps:

  1. Problem identification — documenting the specific failure modes at the deployment site
  2. Minimum requirement specification — defining the precision/recall threshold the business needed
  3. Dataset size determination — based on collected samples and class distribution
  4. Annotation strategy definition — a bounding-box approach suited to the detection task
  5. Annotation execution — about 15,000 photos collected from SmartColumn devices, annotated with bounding boxes

The dataset came from actual SmartColumn footage at the dealership, so it captured the real lighting, camera angles and variety of people.

What were the results?

The retrained model achieved approximately 90% precision in production conditions. That made the deployment profitable and validated SmartColumn for a broader commercial rollout.

What happened next?

Safety1st continued working with BoBox. We later prepared datasets for computer vision on construction-site mobile surveillance towers — read Mobile Surveillance System Powered by Computer Vision.

What can other teams learn from this?

  • Hardware expertise alone doesn't guarantee AI success — data preparation is a discipline of its own, even for projects that look simple.
  • Set the business threshold first — knowing the precision you need tells you when the dataset is good enough.
  • Train on the environment you deploy to — a model is only as good as the conditions its training data represents.

The mask-wearing use case belongs to the pandemic years, but the lesson applies to every safety model we work on today, from PPE compliance to intrusion detection. See safety, security & surveillance annotation, or start with a paid pilot on your own data.

Frequently asked questions

Why did the facemask detection model fail after deployment?

It was trained on public datasets collected in controlled conditions. The car dealership had glass walls, harsh ceiling spotlights and a wide variety of people and mask types — conditions the training data didn't represent — so the model produced many false positives.

How large was the new dataset?

About 15,000 photos collected from SmartColumn devices at the deployment site, annotated with bounding boxes.

What precision did the retrained model achieve?

Approximately 90% precision in production conditions, which made the device commercially viable.

Is this lesson specific to facemask detection?

No. The same lab-to-production gap affects PPE detection, retail analytics, vehicle detection and industrial inspection. Any model deployed in an environment its training data doesn't represent is at risk.

How can I avoid the same problem in my project?

Define the minimum precision and recall your business needs, collect data from the real deployment environment, agree annotation rules for edge cases, and validate the model on production data — not only on a test split of the training set.

Work With Us

Need training-ready visual data?

BoBox helps AI teams prepare, annotate, and validate Computer Vision datasets. From ground truth to production-ready — we handle the full annotation pipeline.

Get in touch