JSS MAHAVIDYAPEETHA

JSS ACADEMY OF TECHNICAL EDUCATION NOIDA

DEPARTMENT OF ELECTRONICS ENGINEERING
PROJECT SYNOPSIS

GROUP NO: 31

Smart homes using Augmented Reality and IoT

Name of the group leader: Divyanshu Dubey

Email ID: divyanshudubey80@gmail.com

Group Members:

ROLL NO Name Mobile no. Email ID

1609131025 Ankit Singh 7651943226 singh.ankit12339@gmail.com

1609131168 Tanya Saxena 8376037652 tanyaavii1325@gmail.com

1609131055 Divyanshu Dubey 9521606482 Divyanshudubey80@gmail.com

Recommended/ Not- Recommended

(DEC)
Objective-
The project aims at developing a highly cost-effective wireless smart home demo integrated with
Artificial Intelligence and Augmented Reality.

This project is being created as a proof of concept that household devices like Lights and Fans
can be controlled wirelessly over a Web Server with an AR Avatar.

The initial prototype will generate a temporary IP Address when connected to an Internet
Hotspot. The Smart Home can then be controlled from a Smartphone or a Computer that will be
connected to the same Hotspot through the temporarily generated IP address.

SCOPE OF THE PROJECT:

1. Despite of Home Automation being one of the most common projects I was driven towards it
as everything have a scope of improvement. Amazon Homes and Google Homes are doing
some great work in the field of smart homes collaborating two technologies being IoT and
Machine Learning. I was motivated to integrate one more forefront technology to it i.e.
Augmented Reality.
2. According to studies companies are spending big bucks over the augmented reality.
Augmented reality is still under the clouds and there are many prospects on which the
developers are working. Reportedly, Microsoft has paid as much as $150 million to buy IP
assets related to augmented reality and even Sony PlayStation5 are coming up with an idea of
Virtual Reality which they are planning to release in 2020 itself *.
3. Undoubtedly Augmented Reality and Machine Learning is the future so why not integrate
both the technologies and make something more efficient?

PROJECT BACKGROUND AND LITERATURE REVIEW

4. IoT (Internet of Things)
The Internet of things (IoT) is the network of physical devices, vehicles, home appliances, and
other items embedded with electronics, software, sensors, and connectivity which
enables these things to connect, collect and exchange data.

In the project, the appliances will be connected to the NodeMCU which is an open
source IoT platform. It includes firmware which runs on the ESP8266 Wi-
Fi SoC from Espressif Systems, and hardware which is based on the ESP-12 module.
The platform will give the scope of Wi-Fi and acting as a gateway will connect all the
devices to a web server.

⮚ Augmented Reality (AR)

Not to be confused with Virtual Reality!

Augmented Reality (AR) is an interactive experience of a real-world environment whereby the

objects that reside in the real-world are "augmented" by computer-generated perceptual
information, sometimes across multiple sensory modalities,
including visual, auditory, haptic, somatosensory, and olfactory.
It’s many times misunderstood with Virtual Reality augmented reality alters one’s ongoing
perception of a real world environment, whereas virtual reality completely replaces the
user's real world environment with a simulated one.

In the smart home demo an AR avatar will be created by a programming language C# and some
assets with whom the user can talk to and make it operate the home appliances. This AR avatar
will be connected to the web server created.

⮚ Artificial Intelligence (Machine Learning)

Machine learning is a field of computer science that uses statistical techniques to give computer
systems the ability to "learn" with data, without being explicitly programmed.

In this project demo the AR avatar will be given the ability to “learn” data. The avatar will be
given intelligence so as to understand the speech of the user, convert it in text, match it with its
data and again converting the answering text to the speech and giving it back to the user in audio
form while operating the appliance simultaneously

CHALLENGES
-SPEECH TO TEXT CONVERSION
-CHARGES FOR API’S

FLOW CHART/ WORK FLOW:

RESEARCH
STUDY ON AR
AR AVATAR
SPEECH TO TEXT MODULE
CONNECTIONS ON NODE MCU
MAKE HTML PAGE
CONNECT NODE MCU AND webpage through Iot.
 Testing
HARDWARE/SOFTWARE TOOLS REQUIRED:
NodeMCU (ESP8266)
LED’s
DC Motor
PCB board and wires
5 Volt relay module
Unity Software

5. REPONSIBILITY OF EACH MEMBER

Name of Member 1:
Name of Member 2:
Name of Member 3:
GRANT CHART:

Tasks Aug Sept Oct Nov Dec Jan Feb Mar Apr

Research and
study

AR Avatar

Speech to text
conversion

Connections on
NODEMCU

Creating
webpage

Assembling and
testing

Error correction
and
modification
Report writing

REFERENCES / LIST OF ACCESSED RESOURCES:

● Andrew Ng. Machine Learning. Retrieved 2018,May 24 from

https://www.coursera.org/learn/machine-learning
● Aseem Sethi. ESP8266 for absolute beginner – Arduino alternative 2018. Retrieved
2018,September 20 from https://www.udemy.com/getting-started-with-nodemcu-step-by-
step/
● Augmented Reality. Retrieved 2018,October 3 from
https://en.m.wikipedia.org/wiki/Augmented_reality
● Internet of things. Retrieved 2018,October 3 from
https://en.m.wikipedia.org/wiki/Internet_of_things
● Machine Learning. Retrieved 2018,October 3 from
https://en.m.wikipedia.org/wiki/Machine_learning