In this project, we will be making a prototype of a smart eyeglass that can help a visually challenged person to recognize the person in front of him and learn about obstacles ahead. This will be enabled by face recognition and distance detection features.
Bill of Material
You will also need an earphone but since most of us have it, I haven’t mentioned it in the
First, open RPi SSH or Terminal and install the required libraries and modules.
Then install the espeak module, followed by the espeak library for Python.
Follow the snippet below for installation (Refer Fig 1,2).
Now install the ‘opencv’, ‘numpy’, ‘dlib’ library. You might need to swap the
memory for installation. Follow the instruction for installation of opencv and dlib
available on the internet.
After that install the face recognition library for python 3 using the following
command in the terminal.
‘pip3 install face recognition’
As we have installed our required library, let’s start coding.
For this project we will make two different codes in Python3, one to tell the distance of
obstacles ahead and other for face recognition. The first code measures the distance of
obstacles and alert the user using voice output. We can also add a vibration motor for haptic
Code 1: Open the Python3 IDE to start coding. Here we need two modules – time and
espeak. We will import both the modules in code. After that we created a distance function
where we import the GPIO and set the pins for distance sensor. Next we created a while
loop that continuously checks the distance. In this while loop we also added an if condition
to check the distance between the user and the obstacle. If it detects an object close to the
user, it will automatically inform the person by giving audio and haptic feedback. (Refer fig
Code 2: This is to recognise the person in front of the user (known or unknown). Here in this code we will import 3 modules: face recognition, cv2 and numpy. We have to create different arrays for known faces and their names. We must write the image file name for the face recognition of that member (Refer Fig 5,6).
In the next part of code, we will try to match the face in camera video with the array of known faces. If the face matched, then the code will run the espeak synthesizer to call the name of that person using the syntax ‘espeak.synth ()’ as in the pic below.(Refer fig 7).
Note :- You can Change the output language to hindi and voice to different modes for example wisher mode, Male voice , and female voice by uncommenting lines in code.(Refer fig 8).
After all is done, save the code in the face recognition folder.
Since we have created the code, now it’s time to connect all the components.
Insert the camera ribbon in camera module and solder the wires of ultrasonic sensor as illustrated below.
|Raspberry Pi||Ultrasonic Sensor|
|BCM Pin 27||TRIG|
|BCM PIN 22||ECHO|
After the connection, set all sensors and components as in pic below.
Now power the Raspberry Pi using battery or power bank. Then open the Pi desktop window in remote desktop or VNC (If you do not know how to enable VNC and remote desktop on Pi follow the instructions online.) Next run the distance.py and facerecogination.py code and you are ready to go. When any known or unknown person come in front of the user (visually challenged person), it will detect the face and tell about them. If the user comes closer to any obstacles like wall or any other object, the device will alert about it to avoid accident.
Note: – This is the first version of the project, we will update it soon with more features to
enable detection of colour and ambient light. The update may come with optical character
and sign board recognition feature as well.
Note: –During the installation of cv2 , dlib, and face recognition library you might need to swap the memory of Raspberry Pi do it carefully.If you face any error or problem during the installation of library refer to online guide of that python library.