Documentation of Arduino group project

Physical based programming interface

 By Charlie Mrachal Hanns and Eirini (Renata) Makropoulou

Project 11: Physical based programming interface Lots of different companies have made a variety of different toys aimed at different age groups to teach programming. Osmo, Lego boost robotics, code & go robot mouse activity set, the code-a-pillar and Cubetto are just some of a vast range of such toys. Come up with a toy prototype of your own. You will need to think of some very basic use cases to test with your prototype. 

Overview of “Spell It”:

What is its purpose?

“Spell It” is a children’s educational toy whose main aim is to teach children how to spell and practice their already existing spelling knowledge. At the same time, “Spell It” also introduces the concept of how programming works. Inputting the different letters to form the word required and immediately getting feedback from the box, indicates two of the main elements in programming, input and output. 

How does it work?

Once the ON/OFF button is pressed, the speaker starts reading random words and asks the child playing to spell it. The child has to go on and spell the requested word using the wooden letter pieces that represent each letter of the alphabet. Once, the user is happy with the word he/she put together, they can send their response to the game, which will compare the child’s spelling to the original word. If the word is spelled correctly, a sound with connotations of success is played indicating that all the letters used are correct and in the right order. The user can now go on and press the reset button to reset the game and play again with a new word. If the word has a misspelling, for example, the word “dog” is written “dof”, then a sound with connotations of failure will be played indicating that the child should try again.

Who is our target audience?

Our target audience consists of children from the age of 5 to the age of 7 years old. This tight age window is relevant as these are the common years in which children start learning how to read, spell and start associating letters from written words to the sounds from spoken words. Having such a young target audience played a major role in our design decisions and what our product would end up to look like.

Market Research:

After having read the project brief, we started researching any existing games that follow the same principle like ours. We researched games like Osmo, Lego boost robotics, Project Bloks by Google and Cubetto but we mostly focused and analyzed Cubetto and Project Bloks. 

Project Bloks:

Cubetto:

After having carried out deeper research for these two products, we decided that “Spell it” should comprise a box-like structure with removable components for the letters introducing tactile actions for the toy. Having tactile sense as the main sense of the game through the touching of the different letter blocks, helps stimulate the child’s brain and adds to the entertainment aspect of the game as there is a direct interaction. Also, we noticed that most toys were either made from wood or plastic and we concluded that wood would be the right fit for “Spell It” as it is a sustainable material which will not add to our carbon footprint. Furthermore, wood is a child friendly material and many toys in the market use wood as their main material.

Meeting log:

Meeting 1 – 11th November

During this meeting we started brainstorming ideas and luckily enough “Spell It” was one of our first ideas during the session. After an hour or so of brainstorming, we spent some time analyzing which idea would be more interesting and unique as our main goal was to create something that has not been done already for this module. We then went on and researched the different games that already exist in the market and that aim to teach spelling to children but also familiarize them with the main ideas of programming. The outcome of this meeting included a finalized concept for Project 11 and an overall idea of what is available in the market. We decided to set some tasks for the upcoming week which we would discuss during our second meeting. Getting inspiration from our research, we had to come up with some ideas that would later on turn into sketches and a storyboard concentrating on how our product would work. 

Meeting 2 – 14th November

During this meeting we initially discussed our ideas and thoughts we had during the weekend and combined them to come up with the best sketch and storyboard. We decided that Charlie would go ahead and complete the sketch and Eirini would focus on the storyboard. Both the sketch and the storyboard can be seen below:

Sketch of initial concept:

Storyboard indicating the functionality of “Spell It”:

The sketch and storyboard helped us and the demonstrators get an initial grasp of what our product will look like and how it will work. The storyboard faceted like a visual organizer and helped us finalize the functionalities that our toy should have. It also gave us an overall idea of the components that were required in order to make “Spell It” operate like seen in the storyboard.

For our next meeting we decided to do some further research regarding the individual components needed and how these should work in order to have the desired outcome.

Meeting 3 – 25th November

After the research we carried out, we concluded on the components that needed to be ordered and shared with each other, how each component that we researched would work. For example, we decided to use color sensors to allow different letters to be recognized by detecting the rgb values for each different color that would be allocated to the bottom of each letter. We decided to use a micro SD card and upload our voice recordings of each word and later on play them using a conventional JBL speaker. We also thought that adding buttons to turn on/off the game, send your answer once you spelled the word and restart the game would be ideal and would also add to the tactile aspect of our game. In the meantime, we also searched for example code and breadboard layout that would help us individually test each component once they arrived. Some of the code and circuit layouts found are presented below:

Code to test if the color sensors works and deduct the rgb values of each colored sheet being used:

#define s0 8       //Module pins wiring

#define s1 9

#define s2 10

#define s3 11

#define out 12

int data=0;        //This is where we’re going to stock our values

void setup() {

 pinMode(s0,OUTPUT);    //pin modes

 pinMode(s1,OUTPUT);

 pinMode(s2,OUTPUT);

 pinMode(s3,OUTPUT);

 pinMode(out,INPUT);

 Serial.begin(9600);   //intialize the serial monitor baud rate

digitalWrite(s0,HIGH); //Putting S0/S1 on HIGH/HIGH levels means the output frequency scalling is at 100% (recommended)

digitalWrite(s1,HIGH); //LOW/LOW is off HIGH/LOW is 20% and LOW/HIGH is  2%

}

void loop(){    //Every 2s we select a photodiodes set and read its data

digitalWrite(s2,LOW);       

digitalWrite(s3,LOW);

Serial.print(“Red value= “); 

GetData();               //Executing GetData function to get the value

digitalWrite(s2,LOW);

digitalWrite(s3,HIGH);

Serial.print(“Blue value= “);

GetData();

digitalWrite(s2,HIGH);

digitalWrite(s3,HIGH);

Serial.print(“Green value= “);

GetData();

Serial.println();

delay(2000);

}

void GetData(){

data=pulseIn(out,LOW); 

Serial.print(data);        

Serial.print(“\t”);     //The higher the frequency the lower the duration

delay(20);

}

The circuit layout used to setup the color sensor and test the above code:

We also researched how to connect an everyday JBL speaker to our board and combined all our findings to come up with a working solution once the components arrived. 

Video that shows how to connect a speaker to the arduino Board: 

https://www.youtube.com/watch?v=gi9mqIha8n0&list=PLVEwT-Kx4yau2rnao-8_alJY1pV6aW059

Code in this video:

#include <SD.h>                       //include SD module library

#include <TMRpcm.h>                   //include speaker control library

#define SD_ChipSelectPin 4            //define CS pin

TMRpcm tmrpcm;                        //crete an object for speaker library

void setup(){

  tmrpcm.speakerPin = 9;              //define speaker pin.

                                       //you must use pin 9 of the Arduino Uno and Nano

                                     //the library is using this pin

  if (!SD.begin(SD_ChipSelectPin)) {  //see if the card is present and can be initialized 

return;                           //don’t do anything more if not

  }

  tmrpcm.setVolume(6);                //0 to 7. Set volume level

  tmrpcm.play(“1.wav”);     //the sound file “1” will play each time the arduino powers up, or is reset

}

void loop(){}

In this tutorial we learnt how to play audio files with Arduino with the use of a speaker and a SD card module to keep the audio files. 

Meeting 4 – 2nd December

We went on campus to collect all of our components and then spent the entire day setting up and testing our code and layout. We firstly made a list of 3-letter words that we would use and started recording them in order to input them in the micro SD card and play them through the speaker. Later, Charlie focused on sorting out the code to make the speaker play the .wav files and recognize the word that is being said so it could later on be linked to the individual letters for each word.

Eirini, focused on working on the color sensor code and adjusting it to our needs. Also, she started testing all the different color sheets that we have in order to find the colors with the most distinct colors that would easily be recognized by the sensor. The color sheets tested can be seen below:

From the 2nd of November onwards, we went on campus everyday and slowly progressed with our project. Testing different components individually and figuring out ways to include them all together in one Arduino board. We tested the different types of buttons that we would use, and the different sounds that would play during the game to indicate a right or wrong answer. Furthermore, we started and finished our box which would be the base for our game and added the holes needed to add the different components colour sensors, buttons, etc. as well as produced the blocks of letters that the children will use to spell the words.

  

Meeting 5 – 8th November

We started working on the user manual and documentation and focused on the problems that we encountered because of the limited amount of pins available on the arduino UNO. We put our finishing touches on the box such as spray painting it and making a whole through which the cable connecting the arduino and our power source would go through. 

Meeting 6 – 9th November

The circuit we had assembled was also implemented within the box and the buttons, speaker and color sensor were put into their place. We put the final code together, tested and tried to debug its shortcomings. We found that the button to read the user rating (colour entered by the colour sensor) was not working when combining codes.

Meeting 7 – 10th November

During this meeting we tried to fix the shortcomings found in the meeting 6. We added the reset button so that the code will produce a new word when it is pressed.

 

Project Task List:

Gantt chart:

Tasks and Roles Allocated:

• Ideation of concept – All
• Brainstorming & Research – All
• Low-fi Prototyping – All
  • Sketch – Charlie 
  • Storyboard – Eirini
• Deciding on the components needed – All
• Ordering and Collecting components – All
• Finding and editing codes for testing components – All
• Making the physical hi-fi prototype – Eirini
  • Laser cutting the box and the holes for the speaker, button and blocks
  • Laser cutting the letter blocks
  • Adding colored paper under each letter block
• Testing & debugging components – All
• Putting together the final code – Charlie
• Documentation – Eirini
  • Design Blog
  • Time chart & Task list 
  • Maker manual
• Taking final pictures and organizing existing ones – All
• Breadboard layout – Charlie
• Finalizing the documentation – All
• Creating the presentation slides – All

Carrying out the code and any problems encountered:
While writing the code, we realized that the Arduino uno board did not have enough pins to accommodate all the components that we wanted to include. Hence, some sacrifices needed to be made. After a lot of consideration, we decided that the best way to still have a working hi-fi prototype in time while showing its main functionalities, would be to only use one color sensor instead of all three that we initially had  planned on using. Even Though this was not ideal, it was the only thing that we could do in such a short time as none of the solutions that we found through research on increasing pin availability had worked. For example, we tried using the method from this video https://www.youtube.com/watch?v=ZsSJdGdLF-A  but unfortunately did not find any luck. Hence, we went ahead with only one color sensor working which meant that our game could only tell whether or not the first letter of the word was correct.

While designing this project, we also realized that we couldn’t get the button to work to send user input (the color value read by the color sensor) to the checkYourself () function. This is most likely due to a code writing error. However, we failed to make it work for the end product.

Splitting code into separate components and testing for each one:

Color Sensor:

Color Sensor Testing Video

Random audio when button pushed:

Random Audio Testing Video

Random Audio & User Input:

Random Audio + User Input Testing Video

How to make “Spell It” – Maker Manual:

“Spell It” is an educational toy that aims to teach children how to spell while introducing them to the basic principles of programming. 

To create the box and physical components of “Spell It” you will need:

• A laser cutting machine
• A CAD software for the sketches for the laser cutting (we used MakerCase for the box and Solidworks for the letter blocks and holes)
• Black spray paint to paint the entire box
• A Vernier Caliper to measure all components with accuracy

For the CAD files similar to the ones shown below should  be created for the box, the holes and the letter blocks:

To create the electronic aspect of “Spell It” you will need:

• 3 color sensors
• 40 male/female jumping wires 
• Colored paper (At least 26 different colors)
• 3 arcade buttons
• A JBL clip 2 speaker 
• 1 Arduino SD Card Module
• 1 2GB Micro SD card module
• A recording device (you can use your phone)

Building process:

We started our box by thinking what the ideal dimensions would be. As this is a game directed to children, the box needed to be of a large size, hence we decided to make it 300 mm x 250 mm x 100 mm. In this way, all our components would easily fit on the box with ease and would add a good aesthetic to the box. Once we decided on our dimensions, we carefully measured each component using a vernier caliper and created the CAD drawings for the box and the holes that had to be made. Once these were ready through laser cutting, we glued the base and the sides together, the top compartment was not glued with the rest in order to make the process of inserting and removing the Arduino board as easy and as possible. We later on spray painted the entire box to give it a minimalistic approach and make the colors of the buttons stand out. We let the box dry overnight and the next day we made the CAD sketches for the letter blocks and laser cut those as well. We decided to leave the letters in a natural wood color in order to contrast the rest of the box and make sure that the letters on the blocks would show clearly. Once this was done, we glued a smaller wooden square on each letter block to make sure that once placed on the holes of the box, the blocks would lock and would not be able to move. We then covered that smaller square with colored paper that represented each different letter.

The outcome for bottom of the letter blocks can be seen below:

Circuit layout using TinkerCad:

Commented and well organised code: 

Demonstration Video:

Demonstration Video

Known shortcomings:

We are fully aware that this does not meet the standards that we initially planned on meeting. The fact that we could not find a solution for using less pins for the color sensors, really affected our final hi-fi prototype. In fact, the product that we demonstrated cannot check for the spelling of the entire word. It only manages to check whether or not the first letter of the word is correct or not. Even Though we have the individual components work to a high standard, we were unable to complete our initial idea due to the unexpected errors which arose. We definitely have some future improvements to implement into “Spell It”. To start with we need to find a way to fit all three sensors into our circuit to achieve the hi-fi prototype’s full potential. We would also like to record and experiment with more 3-letter words and once this is finalized we would like to further extend “Spell It” into using 5-letter words and hence advancing the difficulty level of the toy.