Data labeling Specification

Recycling is vital for a sustainable and clean environment. We use just 481.6 billion plastic bottles every year and only about 9% of them are recycled. In this project, we’ll look at how computer vision can be used to identify different types of plastics, glass, and metal from Household waste. Initially we should look at the dataset and label them.

1. The dataset spans three main classes: glass, plastic, and metal
2. Subclasses should be specified according to the type and color of each main class

Note: plastics have 7 types: PET, HDPE, PVC, LDPE, PP, PS, and Others.

---

How to label data ?

Before diving into the labeling system, we should take a look at the dataset and its requirements:

1. We need to have deformed and broken objects in our dataset in addition to dirty ones.
2. Images should be taken from different angles in case they have been deformed or broken.
3. We should regulate brightness and contrast.
4. To enhance the dataset, using the Augmentation method is a must.
5. It should be considered that all the data have the same size and resolution.
6. The pictures should be taken by placing the object on a dark background.
7. To avoid confusing objects with bright background, we can use one or some of the following methods:

• 7.1. Create a tight bounding box for every object
• 7.2. Omit background
• 7.3. Mask object
• 7.4. Edge Detection

8. The initial form of data is an image, but it should be converted to other required formats concerning the detection method.
9. We should collect a balanced and diverse dataset for each category.

---

There are two common methods for labeling:

• Manual
• Automatic

Manual labeling system

Here we utilize Label Studio (https://labelstud.io/) which is one of the most flexible data annotation tools for labeling and exploring multiple types of data. we enjoy performing different types of labeling with many data formats.

Let's see how we can label our dataset:

1. Install label studio (https://labelstud.io/guide/install.html)
2. Start Label Studio with the label-studio command.
3. Sign up with an email address and password that you create.
4. Click to create a project and start labeling data.

5. Click Data Import and upload the data files that you want to use.

6. Click Labeling Setup and choose a template and customize the label names for your use case.

There are different options for annotation recommended on this site including Semantic Segmentation with Polygon, masking, bounding boxes, and so on.

8. Click Save to save your project.

According to the above picture, some of the icons are not available. It means that we should consider our project and requirements and then choose the appropriate labeling method.

Here we annotate a bottle with Polygon annotation:

To know how to use this labeling system, take a look at this guide: https://www.youtube.com/watch?v=UUP_omOSKuc

Finally, we take a quick look at some formats in the Label Studio:

• COCO (image segmentation, object detection) [x_min, y_min, width, height]

Popular machine learning format used by the COCO dataset for object detection and image segmentation tasks with polygons and rectangles.

• Pascal VOC XML (image segmentation, object detection) [x_min, y_min, x_max, y_max]

Popular XML format used for object detection and polygon image segmentation tasks.

• YOLO format (image segmentation, object detection) [x_center, y_center, width, height]

Popular TXT format is created for each image file. Each txt file contains annotations for the corresponding image file, that is object class, object coordinates, height & width.

• Brush labels to NumPy (image segmentation)

Export your brush labels as NumPy 2d arrays. Each label outputs as one image.

---

Apart from this method, we prioritize automatic labeling, because as it is blindingly clear for large datasets the above method is time-consuming and expensive, so it is advisable to establish and develop methods for the automatic sorting of data.

Automatic labeling

---

The most difficult challenge in the waste management is complex, and not unified guidelines regarding segregation rules. However, in recent years machine learning (ML) based systems that can support or fully cover sorting processes were implemented, accelerating this procedure as a result. In this project, one of the state-of-the-art deep learning (DL) models are implemented to assign proper class based on the photo, and the garbage is moved to the appropriate bottom container.

There are different detection models, but our aim is to find an extremely precise model with 98% accuracy for Pet and 95% for other groups and in this regard, without a doubt, Yolov5 families stand out from other detection models.

Here is a plot to make a clear comparison between Yolov5 and EfficientDet in terms of speed and accuracy

---

Dataset

1. Images per class. ≥1.5k images per class

2. Instances per class. ≥10k instances (labeled objects) per class total

3. Image variety. Must be representative of deployed environment. For real-world use cases we recommend images from different times of day, different seasons, different weather, different lighting, different angles, different sources (scraped online, collected locally, different cameras) etc.

4. Label consistency. All instances of all classes in all images must be labelled. Partial labeling will not work.

5. Label accuracy. Labels must closely enclose each object. No space should exist between an object and it's bounding box. No objects should be missing a label.

6. Background images. Background images are images with no objects that are added to a dataset to reduce False Positives (FP). We recommend about 0-10% background images to help reduce FPs (COCO has 1000 background images for reference, 1% of the total).

---

How Automatic Labeling works?

The automatic labeling system would work properly if seeing too much different data. To do that, first, we collect 4817 images from the Drinking Waste Classification of the Kaggle site with four classes (Alu Can, Glass, HDPE, and PET). In the second step, yolov5 augmentations are applied to data to improve detection. After 52 epochs on 3854 training images and 1% background images(80% of data), we achieve the following results on 963 validation data:

The above result obtains from pre-train weights of yolov5 models on the coco dataset with 28 classes.

---

Another experiment has also been conducted on this dataset under the same condition except for weights of deep plastic (https://github.com/gautamtata/DeepPlastic )with one class used for transfer learning.

Following that, the weights from the above training are used to detect unseen data collected from the telegram bot.

Although the above results are eye-catching enough, the model still needs to be improved to distinguish better all types of our data in various positions. Therefore, the following work specifically focuses on higher accuracy on a larger dataset.