Deteksi Masker Wajah Menggunakan Tensorflow, Keras, Open CV dan Python

Haloo temen-temen, berbagi kali ini agak berbeda dengan sebelumnya karena kita akan belajar tentang deep lerning menggunakan beberapa library seperti tensorflow, keras, dan open cv. Kita akan membuat face mask detection atau mendeteksi penggunaan masker wajah seracara realteime menggunakan kamera atau webcam temen temen.

Gif

Face detector dalam hal ini memiliki dua fase diantaranya adalah:

  • Train Face Mask Detector : Di sini kita akan fokus pada pengolahan dataset face detection, mebuat train data model (menggunakan Keras / TensorFlow) pada dataset, dan kemudian membuat face detector. Yang tujuannya adalah untuk melatih model mendeteksi apakah seseorang memakai masker atau tidak memakai masker.
  • Implementasi Face Mask Detector : Setelah pendeteksi face mask detector ditrain, lalu melakukan deteksi wajah, dan kemudian mengklasifikasikan setiap wajah with_mask atau without_mask.

Dataset

Dataset ini terdiri dari 1.376 gambar yang termasuk dalam dua kelas:

  • with_mask : 690 gambar
  • without_mask : 686 gambar

Dataset yang akan saya gunakan dibuat oleh PyImageSearch, Prajna Bhandary dengan proses sebagai berikut.

  • Mengambil gambar wajah normal
  • Kemudian membuat script Python untuk menambahkan masker wajah ke mereka, sehingga menciptakan dataset buatan (akan tetapi masih kelihatan seperti nyata hehe)

Struktur Project

Setelah temen-temen mendapatkan dataset tadi, sekarang kita lihat struktur project saya dengan menggunakan tree --dirsfirst --filelimit 10 diterminal dan hasilnya seperti berikut ini.

├── dataset
│   ├── with_mask [690 entries exceeds filelimit, not opening dir]
│   └── without_mask [686 entries exceeds filelimit, not opening dir]
├── examples
├── face_detector
│   ├── deploy.prototxt
│   └── res10_300x300_ssd_iter_140000.caffemodel
├── detect_mask_video.py
├── mask_detector.model
├── plot.png
├── requirements.txt
└── train_mask_detector.py

Vritualenv

Sebelum kita lanjut alangkah lebih baiknya kita menggunakan virtualenv supaya library atau modul-modul dobuat terlihat rapih dan terstruktur. Fungsi Virtualenv ini adalah menyimpan modul-modul atau lribary khusus untuk project kita sendiri sesuai dengan kebutuahan.

Saya akan jelaskan kurang lebih begini, jika kita tidak menggunakan virtualenv maka modul-modul atau library akan terinstall secara global dan bisa digunakan semua project yang menurut saya jadi kruang rapih struktur dan memakan resource karena seharusnya ada yang tidak kita pakai jadi ikut serta ada didalamnya karena terinstall secara global dari project-project yang lain, akan tetapi jika kita menggunakan virtualenv libary akan tersintall pada folder tersendiri sesaui dengan kebutuhan project temen-temen.

Cara membuat virtualenv

  • python3 -m venv /path/to/new/virtual/env untuk membuat virtualenv nya
  • sourece /path/to/new/virtual/env/bin/activate untuk mengkatifkan virtualenv nya

Karena disini saya menggunakan python 3.7 jadi perintahnya python3. Dan untuk /path/to/new/virtual/env adalah lokasi temen-temen akan menyimpan virtualenv. Jika temen-temen masih kurang jelas, bisa cari di google ada tutorial yang lebih bagus dari ini tentang virtualenv hehe.

Requirement

Tidak lupa kita akan menginstall modul-modul atau library pada virtualenv yang sudah dibuat dan diaktifkan tadi, berikut librarynya yang sudah saya buat menjadi requirement.txt yang isi nya :

absl-py==0.9.0
astunparse==1.6.3
cachetools==4.1.1
certifi==2020.6.20
chardet==3.0.4
cycler==0.10.0
gast==0.3.3
google-auth==1.18.0
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio==1.30.0
h5py==2.10.0
idna==2.10
importlib-metadata==1.7.0
imutils==0.5.3
joblib==0.16.0
Keras-Preprocessing==1.1.2
kiwisolver==1.2.0
Markdown==3.2.2
matplotlib==3.2.2
numpy==1.19.0
oauthlib==3.1.0
opencv-python==4.3.0.36
opt-einsum==3.2.1
Pillow==7.2.0
protobuf==3.12.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
pyparsing==2.4.7
python-dateutil==2.8.1
requests==2.24.0
requests-oauthlib==1.3.0
rsa==4.6
scikit-learn==0.23.1
scipy==1.5.1
six==1.15.0
sklearn==0.0
tensorboard==2.2.2
tensorboard-plugin-wit==1.7.0
tensorflow==2.2.0
tensorflow-estimator==2.2.0
termcolor==1.1.0
threadpoolctl==2.1.0
urllib3==1.25.9
Werkzeug==1.0.1
wrapt==1.12.1
zipp==3.1.0

Jika sudah bisa langsung jalankan perintah pada terminal pip install requirement.txt atau bisa install satu-satu pip install tensorflow==2.2.0 kalo aku si males hehe.

Script Training Face Mask Detector

Setelah kebutuhan semuanya terpenuhi, langkah selanjutnya kita kan membuat training untuk mendapatkan model untuk mendeteksi face mask detetor, berikut script pada train_mask_detector.py

# import the necessary packages
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.layers import AveragePooling2D
from tensorflow.keras.layers import Dropout
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Input
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.applications.mobilenet_v2 import preprocess_input
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.utils import to_categorical
from sklearn.preprocessing import LabelBinarizer
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
from imutils import paths
import matplotlib.pyplot as plt
import numpy as np
import argparse
import os


# construct the argument parser and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-d", "--dataset", required=True,
	help="path to input dataset")
ap.add_argument("-p", "--plot", type=str, default="plot.png",
	help="path to output loss/accuracy plot")
ap.add_argument("-m", "--model", type=str,
	default="mask_detector.model",
	help="path to output face mask detector model")
args = vars(ap.parse_args())

# initialize the initial learning rate, number of epochs to train for,
# and batch size
INIT_LR = 1e-4
EPOCHS = 20
BS = 32

# grab the list of images in our dataset directory, then initialize
# the list of data (i.e., images) and class images
print("[INFO] loading images...")
imagePaths = list(paths.list_images(args["dataset"]))
data = []
labels = []
# loop over the image paths
for imagePath in imagePaths:
	# extract the class label from the filename
	label = imagePath.split(os.path.sep)[-2]
	# load the input image (224x224) and preprocess it
	image = load_img(imagePath, target_size=(224, 224))
	image = img_to_array(image)
	image = preprocess_input(image)
	# update the data and labels lists, respectively
	data.append(image)
	labels.append(label)
# convert the data and labels to NumPy arrays
data = np.array(data, dtype="float32")
labels = np.array(labels)

# perform one-hot encoding on the labels
lb = LabelBinarizer()
labels = lb.fit_transform(labels)
labels = to_categorical(labels)
# partition the data into training and testing splits using 80% of
# the data for training and the remaining 20% for testing
(trainX, testX, trainY, testY) = train_test_split(data, labels,
	test_size=0.20, stratify=labels, random_state=42)
# construct the training image generator for data augmentation
aug = ImageDataGenerator(
	rotation_range=20,
	zoom_range=0.15,
	width_shift_range=0.2,
	height_shift_range=0.2,
	shear_range=0.15,
	horizontal_flip=True,
	fill_mode="nearest")

# load the MobileNetV2 network, ensuring the head FC layer sets are
# left off
baseModel = MobileNetV2(weights="imagenet", include_top=False,
	input_tensor=Input(shape=(224, 224, 3)))
# construct the head of the model that will be placed on top of the
# the base model
headModel = baseModel.output
headModel = AveragePooling2D(pool_size=(7, 7))(headModel)
headModel = Flatten(name="flatten")(headModel)
headModel = Dense(128, activation="relu")(headModel)
headModel = Dropout(0.5)(headModel)
headModel = Dense(2, activation="softmax")(headModel)
# place the head FC model on top of the base model (this will become
# the actual model we will train)
model = Model(inputs=baseModel.input, outputs=headModel)
# loop over all layers in the base model and freeze them so they will
# *not* be updated during the first training process
for layer in baseModel.layers:
	layer.trainable = False

# compile our model
print("[INFO] compiling model...")
opt = Adam(lr=INIT_LR, decay=INIT_LR / EPOCHS)
model.compile(loss="binary_crossentropy", optimizer=opt,
	metrics=["accuracy"])
# train the head of the network
print("[INFO] training head...")
H = model.fit(
	aug.flow(trainX, trainY, batch_size=BS),
	steps_per_epoch=len(trainX) // BS,
	validation_data=(testX, testY),
	validation_steps=len(testX) // BS,
	epochs=EPOCHS)

# make predictions on the testing set
print("[INFO] evaluating network...")
predIdxs = model.predict(testX, batch_size=BS)
# for each image in the testing set we need to find the index of the
# label with corresponding largest predicted probability
predIdxs = np.argmax(predIdxs, axis=1)
# show a nicely formatted classification report
print(classification_report(testY.argmax(axis=1), predIdxs,
	target_names=lb.classes_))
# serialize the model to disk
print("[INFO] saving mask detector model...")
model.save(args["model"], save_format="h5")

# plot the training loss and accuracy
N = EPOCHS
plt.style.use("ggplot")
plt.figure()
plt.plot(np.arange(0, N), H.history["loss"], label="train_loss")
plt.plot(np.arange(0, N), H.history["val_loss"], label="val_loss")
plt.plot(np.arange(0, N), H.history["accuracy"], label="train_acc")
plt.plot(np.arange(0, N), H.history["val_accuracy"], label="val_acc")
plt.title("Training Loss and Accuracy")
plt.xlabel("Epoch #")
plt.ylabel("Loss/Accuracy")
plt.legend(loc="lower left")
plt.savefig(args["plot"])

Sekarang jalankan script tersebut python train_mask_detector.py - set data dataset

[INFO] loading images...
WARNING:tensorflow:`input_shape` is undefined or non-square, or `rows` is not in [96, 128, 160, 192, 224]. Weights for input shape (224, 224) will be loaded as the default.
2020-07-11 12:31:44.568277: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f92db7e0f20 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-11 12:31:44.568313: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/mobilenet_v2/mobilenet_v2_weights_tf_dim_ordering_tf_kernels_1.0_224_no_top.h5
9412608/9406464 [==============================] - 2s 0us/step
[INFO] compiling model...
[INFO] training head...
Epoch 1/20
34/34 [==============================] - 62s 2s/step - loss: 0.4685 - accuracy: 0.7753 - val_loss: 0.0943 - val_accuracy: 0.9819
Epoch 2/20
34/34 [==============================] - 57s 2s/step - loss: 0.1238 - accuracy: 0.9569 - val_loss: 0.0417 - val_accuracy: 0.9928
Epoch 3/20
34/34 [==============================] - 65s 2s/step - loss: 0.0792 - accuracy: 0.9766 - val_loss: 0.0279 - val_accuracy: 0.9964
Epoch 4/20
34/34 [==============================] - 64s 2s/step - loss: 0.0608 - accuracy: 0.9822 - val_loss: 0.0192 - val_accuracy: 1.0000
Epoch 5/20
34/34 [==============================] - 67s 2s/step - loss: 0.0416 - accuracy: 0.9888 - val_loss: 0.0160 - val_accuracy: 1.0000
Epoch 6/20
34/34 [==============================] - 61s 2s/step - loss: 0.0402 - accuracy: 0.9831 - val_loss: 0.0121 - val_accuracy: 1.0000
Epoch 7/20
34/34 [==============================] - 64s 2s/step - loss: 0.0328 - accuracy: 0.9897 - val_loss: 0.0114 - val_accuracy: 0.9964
Epoch 8/20
34/34 [==============================] - 58s 2s/step - loss: 0.0276 - accuracy: 0.9925 - val_loss: 0.0140 - val_accuracy: 0.9964
Epoch 9/20
34/34 [==============================] - 57s 2s/step - loss: 0.0367 - accuracy: 0.9878 - val_loss: 0.0104 - val_accuracy: 0.9964
Epoch 10/20
34/34 [==============================] - 59s 2s/step - loss: 0.0209 - accuracy: 0.9944 - val_loss: 0.0124 - val_accuracy: 0.9964
Epoch 11/20
34/34 [==============================] - 66s 2s/step - loss: 0.0192 - accuracy: 0.9906 - val_loss: 0.0100 - val_accuracy: 0.9964
Epoch 12/20
34/34 [==============================] - 59s 2s/step - loss: 0.0191 - accuracy: 0.9944 - val_loss: 0.0103 - val_accuracy: 0.9964
Epoch 13/20
34/34 [==============================] - 59s 2s/step - loss: 0.0154 - accuracy: 0.9953 - val_loss: 0.0087 - val_accuracy: 0.9964
Epoch 14/20
34/34 [==============================] - 61s 2s/step - loss: 0.0215 - accuracy: 0.9897 - val_loss: 0.0116 - val_accuracy: 0.9964
Epoch 15/20
34/34 [==============================] - 62s 2s/step - loss: 0.0146 - accuracy: 0.9944 - val_loss: 0.0080 - val_accuracy: 0.9964
Epoch 16/20
34/34 [==============================] - 65s 2s/step - loss: 0.0183 - accuracy: 0.9944 - val_loss: 0.0082 - val_accuracy: 0.9964
Epoch 17/20
34/34 [==============================] - 60s 2s/step - loss: 0.0142 - accuracy: 0.9963 - val_loss: 0.0067 - val_accuracy: 1.0000
Epoch 18/20
34/34 [==============================] - 59s 2s/step - loss: 0.0091 - accuracy: 0.9981 - val_loss: 0.0081 - val_accuracy: 0.9964
Epoch 19/20
34/34 [==============================] - 62s 2s/step - loss: 0.0084 - accuracy: 0.9981 - val_loss: 0.0067 - val_accuracy: 0.9964
Epoch 20/20
34/34 [==============================] - 63s 2s/step - loss: 0.0123 - accuracy: 0.9963 - val_loss: 0.0073 - val_accuracy: 0.9964
[INFO] evaluating network...
              precision    recall  f1-score   support

   with_mask       0.99      1.00      1.00       138
without_mask       1.00      0.99      1.00       138

    accuracy                           1.00       276
   macro avg       1.00      1.00      1.00       276
weighted avg       1.00      1.00      1.00       276

dari hasil tersebut akan memdapatkan model dan hasilnya adalah mask_detector.model

Implementasi Face Mask Detection Secara Real Time

Sekarang setelah data kita train, mari kita pelajari bagaimana kita dapat:

  • Muat gambar
  • Deteksi wajah pada gambar
  • Gunakan pendeteksi Face Mask untuk mengklasifikasikan wajah sebagai salah satu dari keduanya with_mask atau without_mask
# import the necessary packages
from tensorflow.keras.applications.mobilenet_v2 import preprocess_input
from tensorflow.keras.preprocessing.image import img_to_array
from tensorflow.keras.models import load_model
from imutils.video import VideoStream
import numpy as np
import argparse
import imutils
import time
import cv2
import os


def detect_and_predict_mask(frame, faceNet, maskNet):
    (h, w) = frame.shape[:2]
    blob = cv2.dnn.blobFromImage(frame, 1.0, (300, 300),(104.0, 177.0, 123.0))
    faceNet.setInput(blob)
    detections = faceNet.forward()
    faces = []
    locs = []
    preds = []

    for i in range(0, detections.shape[2]):
        confidence = detections[0, 0, i, 2]
        if confidence > args["confidence"]:
            box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
            (startX, startY, endX, endY) = box.astype("int")

            (startX, startY) = (max(0, startX), max(0, startY))
            (endX, endY) = (min(w - 1, endX), min(h - 1, endY))

            face = frame[startY:endY, startX:endX]
            face = cv2.cvtColor(face, cv2.COLOR_BGR2RGB)
            face = cv2.resize(face, (224, 224))
            face = img_to_array(face)
            face = preprocess_input(face)

            faces.append(face)
            locs.append((startX, startY, endX, endY))

    if len(faces) > 0:
        faces = np.array(faces, dtype="float32")
        preds = maskNet.predict(faces, batch_size=32)

    return (locs, preds)


ap = argparse.ArgumentParser()
ap.add_argument("-f", "--face", type=str, default="face_detector", help="path to face detector model directory")
ap.add_argument("-m", "--model", type=str, default="mask_detector.model", help="path to trained face mask detector model")
ap.add_argument("-c", "--confidence", type=float, default=0.5, help="minimum probability to filter weak detections")
args = vars(ap.parse_args())

# load our serialized face detector model from disk
print("[INFO] loading face detector model...")
prototxtPath = os.path.sep.join([args["face"], "deploy.prototxt"])
weightsPath = os.path.sep.join([args["face"], "res10_300x300_ssd_iter_140000.caffemodel"])
faceNet = cv2.dnn.readNet(prototxtPath, weightsPath)
# load the face mask detector model from disk
print("[INFO] loading face mask detector model...")
maskNet = load_model(args["model"])
# initialize the video stream and allow the camera sensor to warm up
print("[INFO] starting video stream...")
vs = VideoStream(src=0).start()
time.sleep(2.0)

while True:
    frame = vs.read()
    frame = imutils.resize(frame, width=400)

    (locs, preds) = detect_and_predict_mask(frame, faceNet, maskNet)

    for (box, pred) in zip(locs, preds):
        (startX, startY, endX, endY) = box
        (mask, withoutMask) = pred
        label = "Mask" if mask > withoutMask else "No Mask"
        color = (0, 255, 0) if label == "Mask" else (0, 0, 255)
        label = "{}: {:.2f}%".format(label, max(mask, withoutMask) * 100)
        cv2.putText(frame, label, (startX, startY - 10),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.45, color, 2)
        cv2.rectangle(frame, (startX, startY), (endX, endY), color, 2)

    cv2.imshow("Frame", frame)
    key = cv2.waitKey(1) & 0xFF

    if key == ord("q"):
        break

cv2.destroyAllWindows()
vs.stop()

Up next