The smart mirror is a project we believe would be helpful for everyone on a daily basis. The purpose of the mirror is to provide you with information about the day ahead, such as the date, the time, the weather, breaking news or your plans for the day (calendar). All of this information can be accessed in many different ways nowadays, but with our gadget it would be easier to access it during the morning routine without losing time, since you would be talking to your mirror while doing your make-up, your hair or getting dressed.

Documentation

Objective
Convenience, is the focus of our project, our goal is to connect a voice recognition device and small screens to the Arduino, and build a mirror around the circuit. Our objective is to create an affordable smart object. Easy to build and that eases anyone’s daily routine, by providing simple and relevant information for the user.

Target Audience
Our target audience is open to all ages ranging from young adults (20) to a much more  mature generation (60) and anyone open  to new technologies and smart objects. Like a product they could use on a daily basis. They are most likely to enjoy the product since they’ve grown up surrounded by smartphones, tablets, smart watches, etc. Their financial status might also fit our affordable product

They could make great use of this product since it’s intended to make the life of the user much easier, providing essential information on demand. Which could be useful and

Market research
After searching the market for products similar to the smart mirror we want to create, we’ve branched off the famous personal assistants such as Alexa and Google Home. We’ve also found similar projects where the that only Raspberry Pi offers an option to connect a monitor to a see-thru mirror, and that the options to create such a device turns out to be very expensive.

Products ordered

Two-way Mirror panel
Wi-Fi module
Voice recognition module
LCD panels

Tasks

Brainstorming session – Everyone
Research – Everyone
Final project decision – Everyone
Sketches and storyboard – Matthew
Low-fi prototype – Everyone
Timetable – everyone
3D design – Abhigyan
Order Components – Abhigyan
Pseudocode – Abhigyan, Carolina
Code – Roberto + Everyone
Assemble components – Everyone
Testing and Debugging – Roberto + Everyone
Maker Manual – Matthew, Carolina
Pictures – Matthew
Meeting Notes – Carolina
Design Blog – Carolina
Presentation – Everyone

Timetable/Gantt Chart

Project development

Meeting updates
15/11/2019
First meeting
Project decision option 1 and option 2

19/11/2019
Brainstorming session
Timetable and Gantt Chart creation
Tasks division
Decision of products that need to be ordered
Low-fi prototype

22/11/2019
Email to obtain access to our blog
Update of products list to order
Storyboard and sketches
First coding approach

26/11/2019
Blog creation
Decision of using LCD panels
Collection of Wi-Fi module and female connectors

29/11/2019
Update of the blog
Coding of the Wi-Fi module
Start of making the mirror frame

03/12/2019
Collection of mirror and LCD panels
Investigation of how to connect the panels without taking over too many pins
Consideration of using more than one Arduino board
Final decision of the panels position behind the mirror
Coding of the Wi-Fi module and voice recognition module

05/12/2019
Voice recognition module not working
3D CAD design of the mirror finished
LCD panels code ready

06/12/2019
Meeting with demonstrators to fix voice recognition module
Start of pseudocode
Circuits on tinkercad
Working on mirror box
Starting of presentation

09/12/2019
Arrival of new voice recognition module
Final touches to the mirror box
Presentation finished

10/12/2019
Presentation day
Last coding tests
Assembly of Arduino boards and mirror casing

Process

We first started reading the list of project ideas offered. After reading all of them, we decided to choose the Teddy Bear as a back-up project, but made the original idea of a smart mirror our first option.

After a brainstorming session (mainly spoken), we got our first main ideas: using small screens behind a see-thru mirror that would show basic information about the day, all of this using voice recognition. We started working on it and we were surprised by an issue with the voice recognition module because it wasn’t working. After testing the microphone and realising it was working, we were sure the problem was the microprocessor, so we decided to order a new one. Luckily, the new module worked perfectly and we were able to continue with our first idea.

Meanwhile, we were also working on the overall design of our final project prototype, building a box around the mirror and adding a black board to its back where we would have a space for the LCD panels and another space for our microphone. We made previous drawings and a story board to make sure we know exactly where and how to start building.

Overall, the result was close to our expectations, but we would’ve liked to implement more elements such as other forms of feedback, better Wi-Fi modules or clearer messages on the LCD screens.

Design principles

Utility – Quick access to basic everyday information without the need to check another device

Usability – Easy and intuitive interaction based on voice recognition and a screen response

Experience – Fun and easy to use for anyone, and easy to assemble

Conceptual Model – a mirror with a small screen on the side for the user to see the information they’re asking for, plus a microphone on top of the mirror for the user to give voice instructions

Visibility – Microphone – indicates need of voice instructions

Mapping – Message on screen asking for voice instructions

Feedback – Messages on the screens (visual)

Low-fi prototype

With this Low-Fi model/sketch which was intended to help us map out the components and visualise the physical properties of the mirror.

Story board

3D DESIGN

Building process

Circuit

Wi-Fi module

Voice Recognition Module

LCD screen

The final result-Two Arduino boards connected to an LCD screens, Wi-Fi modules and a Voice recognition module.

Code

Server

import socket
import requests
import json
from datetime import datetime

def processData(data):
response = data.decode((“utf-8”))
response_switch = {
“1”: GetWeather(),
“2”: GetTime(),
“3”: GetNews(),
“4”: “Meeting today at 4:00 p.m.”,
“5”: “Hello”
}

return response_switch.get(response, “error”)

def encodeJson(data, type):
message = {
“correct” : 1,
“type” : type,
“data” : data
}
json_message_ = json.dumps(message)
return json_message_

def GetWeather():
r = requests.get(‘https://api.openweathermap.org/data/2.5/find?q=London&units=metric&APPID=3f93552d0ff77a4926fb4334ab248909’)
info = r.json()[‘list’][0]
return “Weather is ” + info[“weather”][0][“description”] + “, temperature: ” + str(info[“main”][“temp”])

def GetTime():
now = datetime.now()
current_time = now.strftime(“%H:%M:%S”)
#print(“Current Time =”, current_time)
return current_time

def GetNews():
r = requests.get(‘https://newsapi.org/v2/top-headlines?country=gb&apiKey=35f33f8c935444a385878a4689b351f2’)
info = r.json()[‘articles’][1][‘title’]
#print(info)
return info

# print(GetWeather())
# print(GetTime())
# print(GetNews())

# Create a socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Ensure that you can restart your server quickly when it terminates
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

# Set the client socket’s TCP “well-known port” number
well_known_port = 8080
sock.bind((‘0.0.0.0’, well_known_port))
socket_2_ = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket_2_.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
socket_2_.bind((‘0.0.0.0’, 8081))

# socket_3_ = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# socket_3_.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# socket_3_.bind((‘0.0.0.0’, well_known_port + 2))

# Set the number of clients waiting for connection that can be queued
# socket_3_.listen(5)

# loop waiting for connections (terminate with Ctrl-C)
try:
while 1:
sock.listen(5)
s_1_, address = sock.accept( )
print(“First connected”)

#socket_2_.listen(5)
#s_2_, address2 = socket_2_.accept( )
print(“Second connected”)
# s_3_, address3 = socket_3_.accept( )

print (“Connected from”, address)
# loop serving the new client
while 1:
receivedData = s_1_.recv(1)
if not receivedData: break
index = receivedData.decode(“utf-8”)
print(“Got data: ” + receivedData.decode(“utf-8”))
response = processData(receivedData)
if response != “error” :
print(“Response: ” + response)
final_message_ = response
print(“Final message: ” + final_message_)
if index == “1” or index == “3”:
s_1_.send(final_message_.encode(“utf-8”))
else:
s_2_.send(final_message_.encode(“utf-8”))

# s_1_.send(“Test”.encode(“utf-8”))
# s_1_.send(final_message_.encode(“utf-8”))
# Echo back the same data you just received
# s_1_.send(receivedData)
s_1_.close( )
print (“Disconnected from”, address)
finally:
sock.close( )

Arduino – 1st part – 1 sketch

#include <ArduinoJson.h>
#include <SoftwareSerial.h>

#include <LCDWIKI_GUI.h> //Core graphics library
#include <LCDWIKI_KBV.h> //Hardware-specific library
LCDWIKI_KBV mylcd(ILI9486,A3,A2,A1,A0,A4); //model,cs,cd,wr,rd,reset

//define some colour values
#define BLACK 0x0000
#define BLUE 0x201F
#define RED 0xF800
#define GREEN 0x07E0
#define CYAN 0x07FF
#define MAGENTA 0xF81F
#define YELLOW 0xF5E0
#define WHITE 0xFFFF

#define RX 10
#define TX 11

#define RX_voice 12
#define TX_voice 13

#define MAX_TRIES 50
const int capacity = JSON_OBJECT_SIZE(3);

byte com = 0;

String wifi_name = “PixelPhone”; // CHANGE ME
String wifi_pass = “test1234”;

String address = “35.234.136.128”;
String port = “8080”;
String response_;

int number_of_tires_ = 50;

SoftwareSerial wifi_module(RX,TX);
SoftwareSerial voice_(RX_voice,TX_voice);

void setup(){
Serial.begin(9600);
voice_.begin(9600);
wifi_module.begin(115200);

mylcd.Init_LCD();
Serial.println(mylcd.Read_ID(), HEX);
mylcd.Fill_Screen(RED);

Serial.println(“Setting up voice recognition”);
voice_.write(0xAA);
voice_.write(0x37);
delay(2000);

Serial.println(“Importing voice recongition set”);
voice_.write(0xAA);
voice_.write(0x21);

Serial.println(“Delay”);
delay(2000);

Serial.println(“Checking wifi module”);
SendMessageToWifiModule(“AT”, “OK”);

Serial.println(“Setting the wifi mode”);
SendMessageToWifiModule(“AT+CWMODE=1”, “OK”);

Serial.println(“Setting wifi”);
String instruction = “AT+CWJAP=\”” + wifi_name + “\”,\”” + wifi_pass + “\””;

SendMessageToWifiModule(instruction, “OK”);

Serial.println(“Setting max connections”);
instruction = “AT+CIPMUX=0”;
SendMessageToWifiModule(instruction, “OK”);
Serial.println(“Conecting to TCP connection”);
instruction = “AT+CIPSTART=\”TCP\”,\”” + address + “\”,” + port;
SendMessageToWifiModule(instruction, “OK”);

Serial.println(“Setup complete”);
}

void loop(){
voice_.listen();
while(voice_.available())
{
com = voice_.read();
String response_;
switch(com)
{
case 0x11:
Serial.println(“Weather”);
SendRequest(1);
break;
case 0x12: SendRequest(2); break;
case 0x13: SendRequest(3); break;
case 0x14: SendRequest(4); break;
case 0x15: SendRequest(5); break;
}
Serial.println(“Final response: ” + response_);
}
}
void SendRequest(int index_){
Serial.println(“Sending request”);
SendMessageToWifiModule(“AT+CIPSEND=1”, “OK”);
wifi_module.println(String(index_));

String buffer_s_ = “”;
while(wifi_module.available() > 0){
buffer_s_ += wifi_module.readString();
}
response_ = process_response(buffer_s_);
showinfo(index_, response_);
Serial.println(response_);
}

/*
void SendRequest(int index_){
DynamicJsonDocument json(512);

int tries = 0;
do{
Serial.println(“Sending request”);
SendMessageToWifiModule(“AT+CIPSEND=1”, “OK”);
wifi_module.println(String(index_));

String buffer_s_ = “”;
while(wifi_module.available() > 0) {
buffer_s_ += wifi_module.readString();
}

String response_ = process_response(buffer_s_);
Serial.println(buffer_s_);
Serial.print(“Response: “);
Serial.println(response_);
DeserializationError err = deserializeJson(json, response_);
if(err != DeserializationError::Ok){
Serial.println(“Deserialization error”);
Serial.println(err.c_str());
}
else{
if(json.containsKey(“data”)){
return String(json[“data”].as<char*>());
}
}
tries++;
delay(2000);
}while(tries < 10);
Serial.println(“Request timed out”);
return “Sorry, server could not be reached”;
}
*/

void SendMessageToWifiModule(String instruction, String response_){
int tries = 0;
wifi_module.listen();
while(tries < MAX_TRIES){
wifi_module.println(instruction);
delay(1000);
if(wifi_module.find(“OK”)){
return;
}
tries++;
}

Serial.println(“Unable to send request”);
}
String process_response(String data_){
auto iterator_ = data_.indexOf(“+IPD,”);
auto second_it_ = data_.indexOf(“:”, iterator_);

auto data_length_ = data_.substring(iterator_ + 5, second_it_).toInt();

auto response_ = data_.substring(second_it_ + 1, second_it_ + data_length_ + 2);
// Serial.println(“Data: ” + data_);
// Serial.println(“Response: ” + response_);

return response_;
}

void showinfo(int code, String information){
mylcd.Print_String(information, 0, 0);
if (code == 1 ){
mylcd.Fill_Screen(YELLOW);
mylcd.Set_Text_Mode(0);
mylcd.Set_Text_Back_colour(YELLOW);
mylcd.Set_Text_colour(WHITE);
mylcd.Set_Text_Size(3);
mylcd.Print_String(information, 0, 0);
delay(5000);
}
else if (code == 3 ){
mylcd.Fill_Screen(BLUE);
mylcd.Set_Text_Mode(0);
mylcd.Set_Text_Back_colour(BLUE);
mylcd.Set_Text_colour(WHITE);
mylcd.Set_Text_Size(4);
//mylcd.Print_String(“RECENT NEWS”,0,0);
mylcd.Print_String(information, 0, 0);
delay(5000);
}
else{
mylcd.Fill_Screen(BLUE);
mylcd.Set_Text_Mode(0);
mylcd.Set_Text_Back_colour(BLUE);
mylcd.Set_Text_colour(WHITE);
mylcd.Set_Text_Size(4);
mylcd.Print_String(information, 0, 0);
mylcd.Print_String(information, 0, 0);
}
}

String addnewLines(String string_){
int string_length_ = string_.length();
String newString = “”;
const int n = 8;
for (int i=0 ; i<string_.length() ; ++i)
{
if (i%n == 0 && i != 0)
newString += ‘\n’;
newString += string_[i];
}
return newString;
}

Arduino – 2nd part – 1 sketch

#include <ArduinoJson.h>
#include <SoftwareSerial.h>

#include <LCDWIKI_GUI.h> //Core graphics library
#include <LCDWIKI_KBV.h> //Hardware-specific library
LCDWIKI_KBV mylcd(ILI9486,A3,A2,A1,A0,A4); //model,cs,cd,wr,rd,reset

//define some colour values
#define BLACK 0x0000
#define BLUE 0x201F
#define RED 0xF800
#define GREEN 0x07E0
#define CYAN 0x07FF
#define MAGENTA 0xF81F
#define YELLOW 0xF5E0
#define WHITE 0xFFFF

#define RX 10
#define TX 11

#define MAX_TRIES 50

byte com = 0;

String wifi_name = “PixelPhone”; // CHANGE ME
String wifi_pass = “test1234”;

String address = “35.197.238.235”;
String port = “8081”;
String response_;

int number_of_tires_ = 50;

SoftwareSerial wifi_module(RX,TX);

void setup(){
Serial.begin(9600);
wifi_module.begin(115200);

mylcd.Init_LCD();
Serial.println(mylcd.Read_ID(), HEX);
mylcd.Fill_Screen(RED);

Serial.println(“Delay”);
delay(2000);

Serial.println(“Checking wifi module”);
SendMessageToWifiModule(“AT”, “OK”);

Serial.println(“Setting the wifi mode”);
SendMessageToWifiModule(“AT+CWMODE=1”, “OK”);

Serial.println(“Setting wifi”);
String instruction = “AT+CWJAP=\”” + wifi_name + “\”,\”” + wifi_pass + “\””;

SendMessageToWifiModule(instruction, “OK”);

Serial.println(“Setting max connections”);
instruction = “AT+CIPMUX=0”;
SendMessageToWifiModule(instruction, “OK”);
Serial.println(“Conecting to TCP connection”);
instruction = “AT+CIPSTART=\”TCP\”,\”” + address + “\”,” + port;
SendMessageToWifiModule(instruction, “OK”);

Serial.println(“Setup complete”);
}

void loop(){
wifi_module.listen();
String buffer_s_ = “”;
while(wifi_module.available() > 0){
buffer_s_ += wifi_module.readString();
}
if(buffer_s_.length() != 0){
response_ = process_response(buffer_s_);
int index_ = response_ == “Meeting today at 4:00 p.m.” ? 2 : 4;
showinfo(index_, response_);
}
delay(1000);
}
void SendRequest(int index_){
Serial.println(“Sending request”);
SendMessageToWifiModule(“AT+CIPSEND=1”, “OK”);
wifi_module.println(String(index_));

String buffer_s_ = “”;
while(wifi_module.available() > 0){
buffer_s_ += wifi_module.readString();
}
response_ = process_response(buffer_s_);
showinfo(index_, response_);
Serial.println(response_);
}

/*
void SendRequest(int index_){
DynamicJsonDocument json(512);

int tries = 0;
do{
Serial.println(“Sending request”);
SendMessageToWifiModule(“AT+CIPSEND=1”, “OK”);
wifi_module.println(String(index_));

String buffer_s_ = “”;
while(wifi_module.available() > 0) {
buffer_s_ += wifi_module.readString();
}

String response_ = process_response(buffer_s_);
Serial.println(buffer_s_);
Serial.print(“Response: “);
Serial.println(response_);
DeserializationError err = deserializeJson(json, response_);
if(err != DeserializationError::Ok){
Serial.println(“Deserialization error”);
Serial.println(err.c_str());
}
else{
if(json.containsKey(“data”)){
return String(json[“data”].as<char*>());
}
}
tries++;
delay(2000);
}while(tries < 10);
Serial.println(“Request timed out”);
return “Sorry, server could not be reached”;
}
*/

void SendMessageToWifiModule(String instruction, String response_){
int tries = 0;
wifi_module.listen();
while(tries < MAX_TRIES){
wifi_module.println(instruction);
delay(1000);
if(wifi_module.find(“OK”)){
return;
}
tries++;
}

Serial.println(“Unable to send request”);
}
String process_response(String data_){
auto iterator_ = data_.indexOf(“+IPD,”);
auto second_it_ = data_.indexOf(“:”, iterator_);

auto data_length_ = data_.substring(iterator_ + 5, second_it_).toInt();

auto response_ = data_.substring(second_it_ + 1, second_it_ + data_length_ + 2);
// Serial.println(“Data: ” + data_);
// Serial.println(“Response: ” + response_);

return response_;
}

void showinfo(int code, String information){
if (code == 2 ){
mylcd.Fill_Screen(YELLOW);
mylcd.Set_Text_Mode(0);
mylcd.Set_Text_Back_colour(YELLOW);
mylcd.Set_Text_colour(WHITE);
mylcd.Set_Text_Size(3);
mylcd.Print_String(information, 0, 0);
delay(10000);
}
else if (code == 4 ){
mylcd.Fill_Screen(BLUE);
mylcd.Set_Text_Mode(0);
mylcd.Set_Text_Back_colour(BLUE);
mylcd.Set_Text_colour(WHITE);
mylcd.Set_Text_Size(4);
mylcd.Print_String(“RECENT NEWS”,0,0);
mylcd.Print_String(information, 0, 0);
delay(10000);
}
else{
mylcd.Fill_Screen(BLACK);
}
}

Pseudocode

State LCD Pins

State Wi-Fi Pins

State Voice Recognition Pins

Void Setup () {

Initialize LCD, WIFI module, Voice Recognition Module

Set LCD background to Black

Connect Wi-Fi module to Wi-Fi and server

Send Introduction message to Screens

}

 

Void Loop () {

Check for user voice input

If input () {

Check which command was spoken

Based on command request information from server

Display information to LCD for 10 seconds

Reset display to blank and input.

}

}

Presentation

Click he[ninja_form id=null]re to access our presentation.

Maker Manual

Click here to access our maker manual.

Final project prototype