A Natural User Interface and Blockchain-Based

In-Home Smart Health Monitoring System1

1
Md. Abdur Rahman, 2Khalid Abualsaud, 3Stuart Barnes, 3Mamunur Rashid, and 4Syed Maruf Abdullah
  1MCRLab, University of Ottawa, Canada
  2Computer Science & Engineering, Qatar University, Qatar
  3
Consumer and Organizational Digital Analytics (CODA) Research Centre, King’s College London, UK
4
  Next Generation Media Laboratory
  e-mail: mrahm080@uottawa.ca, {stuart.barnes, mamun.rashid}@kcl.ac.uk, kabualsaud@gmail.com, s.maruf@nextgenmedialab.com
 
  Abstract— In this paper, we propose a gesture-based secure monitoring a reality [7]. These include smart contract for
  interaction system with smart home IoT health devices to Blockchain-based smart home health monitoring applications
  support elderly people or people with special needs. The [8], proof of location based digital certificate for health
  framework uses a decentralized blockchain consensus for
monitoring application [9] [10] [11] [12] [13], IoT health data
  storing the smart home IoT health data and user identities. The
security with spatio-temporal Blockchain smart contract [14],
  framework leverages off-chain solution for storing raw
  multimedia IoT sensory payload and gesture data. Using our
and storing massive amount of health-related IoT data in
  proposed health monitoring framework, a smart homeowner or decentralized off-chain repositories [15]. These technological
  service provider can create a cyber-physical space with a secure innovations have allowed each patient’s or user’s in-home
  digital wallet for each human resident and authorized IoT health monitoring medical IoT data to be stored within the
  health devices. Multiple authorized home residents can interact private Blockchain of the owner of the health data i.e. the
  with the IoT-based smart home monitoring sensors, carry out patient while the recently introduced multi-dimensional
  user and IoT health sensory media registration, and transfer Blockchain can store the horizontal records of each patient in
  transactional values via secure tokens, as well as raw IoT health
different service provider’s vertical private blockchain [16].
  data payload through gesture. In this paper, we will present the
  This type of secure collaboration and handshake between
design and implementation details of the smart home health
  monitoring framework in terms of gesture-based interaction
private and public Blockchain have ushered in the new
  with smart home IoT health devices, such as smart locks and horizon of hybrid Blockchain applications for the in-home
  light bulbs, which uses decentralized blockchain for security, health monitoring medical IoT applications. Although work
  trust, digital identity and privacy preservation. has been done on securing medical IoT devices’ data, few
  research has addressed the suitability of using Blockchain and
  Keywords— Secure Multimedia IoT; blockchain; off-chain; off-chain solutions for monitoring gesture tracking medical
  IoT.
IoT devices transactional data and raw multimedia payload
 
  I. INTRODUCTION respectively [17].
  The Industry 4.0 revolution has enabled smart homes to In this paper, we address the challenge of securely
  benefit from IoT, and the edge computing paradigm, not only controlling smart home IoT multimedia data payload through
  for processing large amounts of data from smart home
  intelligent gesture tracking IoT devices at the edge [18] [19]
multimedia IoT sensory, but also for combining decentralized [20] . Since smart home IoT devices produce a large amount
 
  security techniques. For example, both blockchain and off- of IoT data at the edge, we have enhanced analytics to process
  chain can provide data security and privacy of the IoT data and the IoT data at the edge [21] [22], we have enhanced analytics
  the identity of the smart home residents [1] [2] [3] [3] [4] [5]. to process the IoT data at the edge, extract business semantics
  Once the IoT health data and the interaction with the smart from the IoT data related to health [23] [24] [25] [26], secure
  home IoT devices are secured through a decentralized and analyze the transactions, and save the transaction to the
  blockchain consensus mechanism, the data source location, blockchain. Digital wallets for different actors such as IoT
  the captured event and type of data, and the identity of the
  vendors and IoT data consumers have been developed. The
transaction-parties can be made ubiquitously [6]. multimedia payload available from the IoT devices is stored
 
  Blockchain solutions have gained popularity for making in decentralized repositories. Using our proposed framework,
  health monitoring systems immutable and tamper proof for smart contract-based business intelligence can be applied at
  legal and real-life use cases [6]. Some key technological the edge to easily coordinate smoothly with the IoT health
  sensor data processing framework [27] [28] [29] [30]. This
advancements have made the in-home usage of smart health
 
 
  1This work was supported by the Qatar National Research Fund (a member of the Qatar Foundation) under Grant NPRP10-1205-160012.
  The statements made herein are solely the responsibility of the authors.
 

978-1-7281-4821-2/20/$31.00 ©2020 IEEE 262

Authorized licensed use limited to: Fondren Library Rice University. Downloaded on May 17,2020 at 04:59:06 UTC from IEEE Xplore. Restrictions apply.
 allows the required types of security to be maintained,
particularly confidentiality, integrity, availability, and
authentication of the data, whether at the edge network, during
the data communication among the edge and the decentralized
cloud, or at the remote cloud.
The remainder of the paper is structured as follows. In
Section II we present the proposed system, followed by details
implementation, in Section III. The paper concludes in Section
IV with the future vision of the research.
II. DESCRIPTION OF THE SYSTEM
Figure 1 shows the high-level component architecture and
various key actors of the propose environment. Our proposed
system consists of two types of human actors: the IoT vendor
and the homeowner that intends to securely combine a number
of IoT devices as a gesture-based interaction service. We have
two types of IoT actors in the system – IoT devices for gesture
recognition and IoT devices as smart health monitoring
sensors [24-26]. Before using the system, the blockchain
profile and secure digital wallet have to be created for each
actor. During registration, the multimedia IoT data such as
images, audio, and video, which cannot be saved into the
blockchain due to limitations on block size, which is stored to
a decentralized repository, while a hash or a set of hashes [21-
23], are connected inside the blockchain, and a digital wallet
containing the secure keys and virtual currencies are created.
After each gesture-based IoT data transaction, the account
balance (amount of virtual currency in the wallet) is updated,
generate notifications, and the status of the IoT devices are
updated in the blockchain [27-30]. The homeowner can
always examine the live status of the IoT devices in utilize or
in ownership through our live blockchain and off-chain visual
data analytics.

Fig. 1. High-Level System Architecture.

Fig. 2. Implementation environment with gesture sensors, smart home IoT devices, and custom-built mobile edge device with
communication circuits.

263

Authorized licensed use limited to: Fondren Library Rice University. Downloaded on May 17,2020 at 04:59:06 UTC from IEEE Xplore. Restrictions apply.

III. FRAMEWORK IMPLEMENTATION
Fig. 3. (Left) Implementation of smart home monitoring
system with Blockchain and off-chain explorer, and (right)
Fig. 4. The test instance data flow diagram
Ethereum was used for business transactions and
Hyperledger Fabric for blockchain nodes, while for the
communication with the IoT gateway a smart contract
management and API, and Node.js V8.12.0 was used. The
decentralized storage was implemented using the
InterPlanetary File System (IPFS). Figure 1(a) illustrates the
incorporation of the IoT devices that could be registered into
the system and combined to various business scenarios. In the
implemented scenario, we have used two IoT devices – a
smart lock and a smart bulb that could be controlled using two
gesture-based IoT devices, Leap Motion and the Myo
armband. Figure 1(b) depicts the user interfaces of the
distributed smartphone applications that will be leveraged.
264
Authorized licensed use limited to: Fondren Library Rice University. Downloaded on May 17,2020 at 04:59:06 UTC from IEEE Xplore. Restrictions apply.
the distributed Apps for maintaining the lifecycle of the smart
home IoT data-monitoring framework.
Figure 3 shows the proposed hardware for our work
consists of client-side devices, such as Arduino and ESP8266
mcu boards with portable power supplies, servomotors
running as smart door locks, ZigBee and Wi-Fi-based smart
lamps that are interfaced with the Arduino board, a Myo
Armband and Leap Motion for Gesture detection, and a
smartphone and Laptop for running dApps with gesture
detection APIs. Details about the gesture tracking methods
and the recognition rates can be reached at our earlier
publications [16] [17] [18] [19]. The MQTT broker, API
server and Hyperledger Fabric Blockchain nodes are running
in an AWS virtual machine, running Ubuntu 64-bit with 8GB
of RAM., we have used Firebase by Google was used for the
real-time database. Used Infura, a scalable blockchain
infrastructure for secure, reliable, and scalable access to
Ethereum transactional APIs and IPFS gateways.
The dApps shown in Figure 1(b) act as a middle-tier edge
node. The gesture detection sensors send data on a user’s
interaction with an IoT device to dApps, running on either a
smartphone or a PC secured with an encrypted digital wallet.
The dApp finally sends the encrypted IoT payload data to off-
chain for real-time processing and to blockchain for
immutable storage.
Figures 1(c) and Figure 1(d) show example for use case
interfaces. In each of the business cases, the location of the
authorized smart home inhabitants and the IoT nodes are
stored in the blockchain. These items could be shared with
anyone securely and through gesture-based interactions. We
have used a no-SQL key-value pair database for storing the
 live location and any other data available from any of the peer
nodes due to that the blockchain cannot be used as a real-time
database. The live data available is supported by a queue
service that can store a very large number of key-value pairs
in order to share with the API server. The API server react
with the smart contract, blockchain and off-chain to store the
live location and other data in the permanent decentralized and
fixed repositories. Figure 1(e) shows the smart health
monitoring sensors integration with the framework. Figure 2
shows the implemented environment hardware setup
consisting of IoT devices and gesture tracking sensors
integrated with the dApps within a smart home. Figure 3
shows the setup running onboard sensors and communication
modules to connect with IoT devices and the smartphone
DApps running within the edge network.
Figure 4 shows the data flow diagram while we setup the
testing framework. The IoT Node Server/Broker uses MQTT
server as broker between IoT Node and Blockchain while
Node-Red middleware interface establishes data flow between
IoT Node and Blockchain and Firebase Real-time database.
Firebase Real-time database is used for live data update and
view the IoT geo-spatial data archived or live from
Blockchain.
Blockchain machine farm with 4 physical cores, 8 threads,
and 16 GB of RAM has been utilized in this experiment. The
IoT data send rate was at 330 transaction per second (tps). In
the system benchmark test, which is shown in Figure 5, the
maximum latency was is 3.55 seconds, the minimum latency
was 1.27 seconds, and the average latency was 2.345 seconds.
The latency is due to the fact that the high-throughput gesture
tracking IoT data needs to be captured, parsed and then sent to
the private, permissioned blockchain nodes, which approve
the transactions and add the IPFS hash to the blockchain.
We have then done a scalability test of adding more nodes
with variable rate of transactions per second from each node.
Figure 6 shows the result of doubling the number of nodes
while each node introduce a variable number of randomly
selected transactions per second within a simulated workload.
As shown in Figure 6, while the number of nodes increase at
any given moment, the latency increases steadily. In our future
initiative, we will work on even lowering the latency by
adding an overlay of caching mechanism.
Fig. 5. Latency of committing transaction on blockchain
265
Authorized licensed use limited to: Fondren Library Rice University. Downloaded on May 17,2020 at 04:59:06 UTC from IEEE Xplore. Restrictions apply.
Fig. 6. Scalability test result of committing transaction on
blockchain
IV. CONCLUSION AND FUTURE WORK
In this paper, we have presented the design of a working
prototype of gesture-based secure interaction with smart home
medical IoT devices. The communication, the ID of users and
IoT devices, and the raw IoT and interaction data are securely
stored in off-chain while the salient transactions are stored in
the blockchain.
In the future, we are planning to use the framework within
real-life applications. In order to achieve different types of
health scenarios, we will be investigating the performance of
federated and private blockchain systems. In additions, the
scalability of the system will be improved to support millions
of IoT data transactions at any given time. We will also work
on adding more health sensors to be able to monitor heart rate,
pulse oximeter, blood pressure, to name a few. We will also
work on adding more types of gesture tracking options such
as using hyper-spectral camera and radar.
ACKNOWLEDGMENT
The authors would like to thank Dr. George Loukas of
University of Greenwich and Areej Abdu.
REFERENCES
[1] M. Abdur Rahman, M. Shamim Hossain, E. Hassanain, and G.
Muhammad, “Semantic Multimedia Fog Computing and IoT
Environment: Sustainability Perspective,” IEEE Commun. Mag., vol.
56, no. 5, pp. 80–87, 2018.
[2] X. Li, P. Jiang, T. Chen, X. Luo, and Q. Wen, “A survey on the
security of blockchain systems,” Futur. Gener. Comput. Syst., pp. 1–
13, 2017.
[3] A. Rahman, E. Hassanain, and M. S. Hossain, “Towards a Secure
Mobile Edge Computing Framework for Hajj,” IEEE Access, vol. 5,
pp. 11768–11781, 2017.
[4] H. Yin, D. Guo, K. Wang, Z. Jiang, Y. Lyu, and J. Xing,
“Hyperconnected Network: A Decentralized Trusted Computing and
Networking Paradigm,” IEEE Netw., vol. 32, no. 1, pp. 112–117,
2018.
[5] Q.-V. Pham, T. LeAnh, N. H. Tran, and C. S. Hong, “Decentralized
Computation Offloading and Resource Allocation in Heterogeneous
Networks with Mobile Edge Computing,” IEEE Access, vol. 6, 2018.
[6] H. Dai, S. Member, Z. Zheng, S. Member, Y. Zhang, and S. Member,
“Blockchain for Internet of Things : A Survey,” IEEE Internet Things
J., vol. PP, no. c, pp. 1–19, 2019.
 [7] S. Aich, S. Chakraborty, M. Sain, H. Lee, and H. Kim, “A Review on
Benefits of IoT Integrated Blockchain based Supply Chain
Management Implementations across Different Sectors with Case
Study,” in 2019 21st International Conference on Advanced
Communication Technology (ICACT), 2019, pp. 138–141.
[8] Y. Zhang, S. Kasahara, Y. Shen, X. Jiang, and J. Wan, “Smart
Contract-Based Access Control for the Internet of Things,” IEEE
Internet Things J., vol. PP, no. c, p. 1, 2018.
[9] E. Leka, L. Lamani, B. Selimi, and E. Deçolli, “Design and
Implementation of Smart Contract : A use case for geo-spatial data
sharing,” in MIPRO, 2019, pp. 1565–1570.
[10] M. Amoretti, G. Brambilla, F. Medioli, and F. Zanichelli,
“Blockchain-based Proof of Location,” in 2018 IEEE International
Conference on Software Quality, Reliability and Security Companion
(QRS-C), 2018, pp. 146–153.
[11] F. Boeira, M. Asplund, and M. Barcellos, “Decentralized proof of
location in vehicular Ad Hoc networks,” Comput. Commun., vol. 147,
no. June, pp. 98–110, 2019.
[12] F. Boeira and M. Asplund, “Vouch : A Secure Proof-of-Location
Scheme for VANETs,” in MSWiM, 2018, pp. 1–4.
[13] A. Litke, D. Anagnostopoulos, and T. Varvarigou, “Blockchains for
Supply Chain Management: Architectural Elements and Challenges
Towards,” MDPI Logist., vol. 3, no. 5, pp. 1–17, 2019.
[14] M. A. Rahman, M. M. Rashid, M. Shamim Hossain, E. Hassanain, M.
F. Alhamid, and M. Guizani, “Blockchain and IoT-Based Cognitive
Edge Framework for Sharing Economy Services in a Smart City,”
IEEE Access, vol. 7, pp. 18611–18621, 2019.
[15] A. Savelyev, “Copyright in the blockchain era: Promises and
challenges,” Comput. Law Secur. Rev., vol. 34, no. 3, pp. 550–561,
2018.
[16] M. A. Rahman, M. S. Hossain, M. M. Rashid, S. J. Barnes, M. F.
Alhamid, and M. Guizani, “A Blockchain-Based Non-Invasive
Cyber-Physical Occupational Therapy Framework: BCI Perspective,”
IEEE Access, vol. 7, pp. 34874–34884, 2019.
[17] M. S. Hossain, M. A. Rahman, and G. Muhammad, “Towards energy-
aware cloud-oriented cyber-physical therapy system,” Futur. Gener.
Comput. Syst., pp. 1–14, 2017.
[18] M. A. Rahman et al., “Blockchain and IoT-based Secure Multimedia
System for a Massive Crowd: Sharing Economy Perspective,” in ACM
ICMR, 2019.
[19] M. A. Rahman and M. S. Hossain, “m-Therapy: A Multi-sensor
Framework for in-home Therapy Management: A Social Therapy of
Things Perspective,” IEEE Internet Things J., vol. 4662, no. c, 2017.
[20] D. Lin, S. Hu, Y. Gao, and Y. Tang, “Optimizing MEC Networks for
Healthcare Applications in 5G Communications With the
Authenticity of Users’ Priorities,” IEEE Access, vol. 7, pp. 88592–
88600, 2019.
266
Authorized licensed use limited to: Fondren Library Rice University. Downloaded on May 17,2020 at 04:59:06 UTC from IEEE Xplore. Restrictions apply.
[21] K. Abualsaud, T. Elfouly, T. Khattab, E. Yaacoub, L. Ismail, M. H.
Ahmed, and M. Guizani, “A Survey on Mobile Crowd-Sensing and
Its Applications in the IoT Era”, IEEE Access, Vol. 7, No. 1, pp. 3855-
3881, December 2018, doi: 10.1109/ACCESS.2018.2885918.
[22] K. Abualsaud, M. Mahmuddin, M. Saleh, and A. Mohamed,
"Ensemble classifier for epileptic seizure detection for imperfect EEG
data," Sci. World J., vol. 2015, Dec. 2015, Art. no. 945689, doi:
10.1155/2015/945689.
[23] M. S. Hossain, G. Muhammad, and A. Alamri, “Smart healthcare
monitoring: a voice pathology detection paradigm for smart cities,”
Multimedia Systems, vol. 25, no. 5, pp. 565-575, October 2019
[24] M. S. Hossain, M. A. Rahman, and G. Muhammad, "Cyber-physical
cloud-oriented multi-sensory smart home framework for elderly
people: An energy efficiency perspective," Journal of Parallel and
Distributed Computing, vol. 103, no. 2017, pp. 11-21, May 2017
[25] M. S. Hossain, and G. Muhammad, “Emotion-Aware Connected
Healthcare Big Data Towards 5G,” IEEE Internet of Things Journal,
vol. 5, no. 4, pp. 2399-2406, Aug. 2018.
[26] M. Masud, M. S. Hossain and A. Alamri, "Data Interoperability and
Multimedia Content Management in e-Health Systems," IEEE
Transactions on Information Technology in Biomedicine, vol. 16, no.
6, pp. 1015-1023, Nov. 2012
[27] S Gowid, R Dixon, S Ghani, “Performance Comparison Between Fast
Fourier Transform-Based Segmentation, Feature Selection, and Fault
Identification Algorithm and Neural Network for the Condition
Monitoring”, Journal of Dynamic Systems, Measurement, and
Control 139 (6), 061013, 2017.
[28] Khalid Abualsaud, Massudi Mahmuddin, Ramy Hussein, Amr
Mohamed, “Performance Evaluation of Compression-Accuracy trade-
off Using Compressive Sensing for EEG-based Epileptic Seizure
Detection in Wireless Tele-monitoring”, in the proc. of The 9th IEEE
Inter. Wireless Communications and Mobile Computing Conference
(IWCMC 2013), Sardinia – Italy, July 1-5, 2013.
[29] Khalid Abualsaud, Massudi Mahmuddin, Amr Mohamed, "WBASN
signal processing and communication framework: Survey on Sensing,
Communication Technologies, Delivery and feedback", in J. of
Computer Sciences (JCS), New York, 2012.
[30] Khalid Abu Al-Saud, Amr Mohamed and Massudi Mahmuddin,
"Survey on Wireless Body Area Sensor Networks for healthcare
applications: Signal Processing, data analysis and feedback", in the
Proc. of the 3rd Inter. Conf. on Computing and Informatics,
ICOCI2011, Indonesia, 8-9 June, 2011.