Sie sind auf Seite 1von 108

Five Custom Android ROMs

You Can Have Fun With


Cool FFmpeg ` 120
Tricks

Volume: 04 | Issue: 07 | Pages: 108 | April 2016

SUBSCRIBER COPY
NOT FOR RE-SALE
Sale and purchase
of this copy is illegal.

An Interview With An Interview With


Piyush Somani, Founder, MD And Sanjay Kaul, MD, Service Provider
CEO, ESDS Software Solutions Pvt Ltd Business, Cisco India And SAARC
Contents
Admin
22 vnStat: A Lightweight
Network Traffic Monitor
29 Sharing Resources Across
Diverse OSs with Samba
90 An Introduction to Riak S2

Developers
49 JavaScript: The New Parts

Use Sync for


24
52 GNU Emacs: How to
Work with HTML Mode,
Indentation and Magit Stress-free Data Backup
56 Get Started with Developing

60
MS Office Add-ins
Build a Personal Safety
Ceph: A Storage
Application in App and Backup
Inventor 2

33
64 A Guide to Material Design,
Solution for
a Modern Software Design Every Need
Language

OpenGurus
67 The Z File System: It’s
Honest and Different
69 SimpleCV: Making Vision
Computing Easy and
Effective

For U & Me
73 A Beginner’s Guide to Linux
75

80
GNU Linux Mint 17.3: The
Friendly Distro
Cool FFmpeg Tricks
37 Manage File
Storage to
Give the Best
42 Bare Metal System
Backup and
Recovery Using
83 Getting Familiar with GRUB Customer Service Open Source Tools
87 How to Run Android Without
Google
92 An Overview of Open Source
REGULAR FEATURES
Databases 07 You Said It... 10 FOSSBytes 104 Tips & Tricks
95 Five Cool Custom Android 08 New Products 55 Editorial Calendar
ROMs You Can Have Fun With

4 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Contents Editor
Rahul chopRa

Editorial, SubScriptionS & advErtiSing


Delhi (hQ)
d-87/1, okhla industrial area, phase i, new delhi 110020
ph: (011) 26810602, 26810603; Fax: 26817563
E-mail: info@efy.in

MiSSing iSSuES
E-mail: support@efy.in

back iSSuES
Kits ‘n’ Spares
new delhi 110020
ph: (011) 26371661, 26371662
E-mail: info@kitsnspares.com

nEwSStand diStribution
ph: 011-40596600
E-mail: efycirc@efy.in

advErtiSEMEntS
mumbai
ph: (022) 24950047, 24928520
E-mail: efymum@efy.in

Creating a Storage System beNGaluRu

using Rockstor 46 ph: (080) 25260394, 25260023


E-mail: efyblr@efy.in

PuNe
ph: 08800295610/ 09870682995
E-mail: efypune@efy.in

GuJaRaT
ph: (079) 61344948
E-mail: efyahd@efy.in

chiNa
power pioneer group inc.
ph: (86 755) 83729797, (86) 13923802595
E-mail: powerpioneer@efy.in

JaPaN
tandem inc., ph: 81-3-3541-4166
E-mail: tandem@efy.in

SiNGaPORe
publicitas Singapore pte ltd
ph: +65-6836 2272
“Our goal is to give “A CIO should E-mail: publicitas@efy.in

the world niche, cost- have digitisation TaiwaN

effective technology
J.k. Media, ph: 886-2-87726780 ext. 10
on his agenda” E-mail: jkmedia@efy.in

solutions” uNiTeD STaTeS


E & tech Media
—Piyush Somani, —Sanjay Kaul, ph: +1 860 536 6677
E-mail: veroniquelamarque@gmail.com
founder, MD and CEO of MD, service provider
ESDS Software Solutions business, Cisco India
98 102
printed, published and owned by ramesh chopra. printed at tara
Pvt Ltd and SAARC, art printers pvt ltd, a-46,47, Sec-5, noida, on 28th of the previous
month, and published from d-87/1, okhla industrial area, phase i, new
delhi 110020. copyright © 2016. all articles in this issue, except for
interviews, verbatim quotes, or unless otherwise explicitly mentioned,
will be released under creative commons attribution-noncommercial

News 3.0 unported license a month after the date of publication. refer to
http://creativecommons.org/licenses/by-nc/3.0/ for a copy of the

15 PGDay India 2016:


DVD of The MonTh licence. although every effort is made to ensure accuracy, no responsi-
bility whatsoever is taken for any loss due to publishing errors. articles
that cannot be used are returned to the authors if accompanied by a
A Summary of the Show Back up and manage all self-addressed and sufficiently stamped envelope. but no responsibility
is taken for any loss or delay in returning the material. disputes, if any,
your valuable data. will be settled in a new delhi court only.

Columns • deepin 15.1 • FreeNAS 9.3 SUBSCRIPTION RATES


• Linux Lite 2.8 • Clonezilla 2.4.5 Year Newstand Price You Pay Overseas
(`) (`)
17 CodeSport • Rockstor 3.8.12 Five 7200 4320 —
three 4320 3030 —
19 Exploring Software: one 1440 1150 uS$ 120

106
kindly add ` 50/- for outside delhi cheques.
Visualising Data Using please send payments only in favour of EFY Enterprises Pvt Ltd.
non-receipt of copies may be reported to support@efy.in—do mention
googleVis your subscription number.

6 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


YOU SAID IT
The OSFY effect ED: We regret the inconvenience caused. I would request you
I am currently a B. Tech student at Christ University, to please send an email to support@efy.in with details like the
Bengaluru. I have been reading your magazines ever since I subscription number and missing issue. They will revert to you
joined my professional course and find them amazing. They and arrange to send a copy.
teach me the topics that a computer engineer must know about
in the current tech world and which are out-of-syllabus. I am
also a regular reader of EFY and I had attended the EFY Expo Articles for beginners
2016 which was held recently in Bengaluru. The Expo was a Please guide me on the online courses for Linux. Also, some topics
great place for the tech-savvy. I met a few interesting people covered in OSFY go over my head, as I am a newbie. Could I
and also learned a lot about electronics and IoT. request you to please include some more articles for beginners?
I actually like security related topics. Also, thanks for
bundling the bootable Kali Linux and Backbox distros (if —Goswami Prashanta, pacificg@gmail.com
not, I would have had to spend time in downloading ISO files
of 3-4GB). Your magazine, OSFY, has transformed me so ED: Thanks for your feedback. We always try to carry articles
much that I’ve started my own blog on ‘Open Source Security that cater to different types of readers. The current issue of
Tools’ hosted on GitHub (https://hackwith.github.io) and also OSFY, for instance, has a number of articles for beginners.
started a regularly maintained page/account on Facebook, Also, as you suggested, we will try and include more articles for
Twitter and LinkedIn. newbies. We urge you to continue sharing your opinion with us,
as this motivates us and helps us to deliver even better.
—B.N.Chandrapal, bnchandrapal@hotmail.com

ED: First of all, a big thank you for your words of Writing for opensourceforu.com
appreciation. Through the years, we have constantly worked I have over 11 years of experience as a Web developer. My
at improving the content, layout and design of the magazine hobby is to write articles on Web development using WordPress,
in order to make your reading experience more enjoyable. Joomla, Drupal and other such themes. I would like to contribute
It really makes us feel great knowing how our readers have such articles to opensourceforu.com. I hope the content in these
benefited by reading OSFY. And as our readers have always would be beneficial to your specific audience. Let me know if
done, we urge you to continue sharing your views with us, as this is an option.
this motivates us and helps us to deliver even better.
—Reegan, reegan786@gmail.com

Missing subscription copy ED: It’s great to hear of your interest in writing for us! Before
I did not receive the February issue of Open Source For You doing so, you can send us the detailed Table of Contents (ToC)
magazine. Kindly let me know how to get it. at osfyedit@efy.in for the topic you want to write on. Our team
will review it, and once they give you the thumbs-up, you can go
—Sanjeevkumar Badrinath, sanjeevkumarrao@gmail.com ahead with the article.

Share Your
Please send your comments
or suggestions to:
The Editor,
Open Source For You,
D-87/1, Okhla Industrial Area, Phase I,
New Delhi 110020, Phone: 011-26810601/02/03,
Fax: 011-26817563, Email: osfyedit@efy.in

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 7


NEW PRODUCTS
An affordable Affordable Bluetooth speakers from Zebronics
smartphone from Lava Zebronics, the IT peripherals and
accessories manufacturer, has
Indian smartphone manufacturer launched its Bluetooth 2.1 ‘Rock n
Lava has launched an affordable Roll’ speakers, which are equipped
smartphone, namely, the Lava with surround sound, as well as sub-
Fuel F2. It features a 12.7cm (5 woofers with 7.6cm low range drivers. Price:
inch) FWVGA display with an The satellite speakers are powered by ` 1,111
854x480 pixel resolution. The 6.3cm mid/high range drivers for high
phone is powered by a 1.3GHz performance sound quality. Rock n Roll speakers are available via
quad-core processor with 512MB of The Rock n Roll speakers enable online and retail stores.
RAM, and runs on the Android 5.1 users to play music via a USB device
Lollipop operating system. or SD card through the USB reader, Address: Zebronics India, 15/14,
The dual SIM phone comes with which supports MP3/WMA dual 12th A Cross, Sindhi Hospital Road,
onboard storage of 8GB, further format decoding. The speakers also Sampangiram Nagar, Bengaluru
expandable up to 32GB via microSD support FM tuning. The Zebronics 560027, Karnataka; Ph: 080-41713242
card. Packed with a 3000mAh
battery, the Fuel F2 comes with
a 5 megapixel rear camera with Innovative Bluetooth headphones from Creative
LED flash and a VGA front facing
camera. The connectivity options Creative, the maker of digital
of the device include 3G, Wi-Fi, entertainment products, has unveiled
Bluetooth and GPS. its latest Bluetooth over-the-head
The Lava Fuel F2 is available in headphones, named ‘Outlier’, which
black and white options, via eBay. are crafted from durable material with
a flexible headband. The ear cushions
Address: Lava International are made from soft-protein leatherette
Ltd, A-56, Sector – 64, Noida, Uttar that allows them to be worn all day,
Pradesh 201301; Ph: 01204637100; with comfort.
Website: http://www.lavamobiles.com Targeted at urban commuters and
Price:
fitness enthusiasts, the headphones
come with six pairs of interchangeable
` 4,881
colour rings, allowing for 30
customised colour combinations. The headphones include a
Weighing just 93 grams, they are 3.55mm audio-in socket, which lets
equipped with a built-in MP3 player, users connect them with a typical
enabling users to insert a microSD stereo cable or any source equipment.
card and use them without a paired Bluetooth v4.1 technology supports
smartphone. the MP3, WMA and WAV formats.
The headphones’ audio is powered The company claims that the
by 32mm neodymium dynamic drivers, headphones offer 10 hours of
and a ‘Buddy’ app can be installed to uninterrupted music on a full charge.
read out the messages received from The Creative Outlier is available
paired Android devices. They also via amazon.in.
feature NFC connectivity for quick
Bluetooth pairing with supported Address: Amazon.com, 26/1, 8th
devices, along with lossless USB audio Floor, Brigade Gateway, Dr Rajkumar
Price:
playback in order to charge the device Road, Malleshwaram West, Bengaluru,
` 4,444 while playing music. Karnataka 560005

8 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


NEW PRODUCTS

4G tablet from Micromax High capacity compact


Indian consumer electronics company
SSD from Samsung
Micromax has launched a new tablet Samsung has launched its latest solid-
in India called Micromax Canvas Tab state drive (SSD) — the T3. Similar to
P702, with the highlighted feature the previously launched T1, the new T3
being 4G LTE support at an affordable looks like a slightly thicker credit card
price. The dual SIM supporting and weighs around 51 grams.
device features a 17.7cm (7 inch) The drive sports a dark silver
HD (1280x720 pixels) resolution coloured metal body with a black
IPS display and runs on Android 5.1 design and anti-scratch urethane
Lollipop, out-of-the-box. Price: coating, which gives it a premium look.
Powered by a 1.3GHz quad-core ` 7,999 It is powered by Samsung’s proprietary
MediaTek processor, the device has a Vertical NAND (V-NANO) and SSD
2GB RAM along with 16GB inbuilt Wi-Fi, Bluetooth and micro-USB TurboWrite technology.
storage, expandable up to 32GB via connectivity, including dual speakers The SSD offers sequential read/
microSD card. The tablet is backed powered by DTS audio technology. write speeds of up to 450 megabytes
with a 3000mAh battery rated to It is bundled with an OTGB cable, per second (MB/s) with a USB 3.0
deliver 250 hours of standby and three headphones and is available in bold super speed interface, along with AES
hours of playtime. black and white, via Snapdeal. 256-bit hardware encryption. It has a
As for the camera, the Canvas Tab shock-resistant internal frame, which
P702 features a 5 megapixel auto- Address: Micromax Informatics can withstand an accidental drop from
focus rear camera with LED flash Limited, 90B, Sector-18, Gurgaon up to 2 metres, along with a thermal
along with a 2 megapixel front facing 122015; Ph: +91-124-4811000; guard to manage workload and prevent
camera. The tablet supports 4G LTE, Email: info@micromaxinfo.com the device from overheating.

Sony’s pocket-sized projector for smartphones


Sony recently unveiled its portable
projector for smartphones,
namely, the MP CL1.
Price:
The pocket-sized projector comes ` 10,999 for 250GB,
encased in an aluminium body with a ` 18,999 for 500GB,
matte finish and weighs around 210 ` 37,999 for 1TB and
grams. It uses a laser light source ` 74,999 for 2TB
and offers full HD resolution images Price:
with a contrast ratio of 80,000:1, at ` 45,899 The T3 SSD is compatible with
an aspect ratio of 16:9 and maximum a wide range of USB supported
brightness of 32 ANSI lumens. The projector uses Sony’s laser devices, including the latest Android
The MP CL1 is a short-throw beam scanning technology, which smartphones, tablets, etc. It uses the
mobile projector capable of displaying enables focus-free projection and exFAT file system so that users need
a screen size of up to 304.8cm (120 does not require even surfaces. not reformat while using the drive with
inches) at a distance of 3.45 metres. According to company sources, the different computers.
It offers several connectivity options, Sony MP CL1 lasts for up to 120 The Samsung T3 SSD is available in
including HDMI or MHL. minutes on HD resolution playback 250GB, 500GB, 1TB and 2TB variants
The projector features built-in with a built-in 3,400mAh battery. across all leading online and retail stores.
screen mirroring that allows users It is available via amazon.in.
to mirror content via Wi-Fi. It’s Address: Samsung India Electronics Pvt
suited for business presentations Address: Sony India, No. A-31, Ltd, 2nd, 3rd and 4th Floors, Tower-C,
or home entertainment, as it comes Mohan Co-Operative Industrial Vipul Tech Square, Sector-43, Golf
with a built-in speaker for displaying Estate, Mathura Road, New Delhi Course Road, Gurgaon 122002, Haryana
content from smartphones and tablets. 110044; Ph: 011 66006600 (India); Ph: 0124-4881234; Website:
www.samsung.com/intp-link.in/

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 9


FOSSBYTES Powered by www.efytimes.com

Linus Torvalds releases the Microsoft joins Open Source


Linux kernel 4.5 Eclipse Foundation
The Linux kernel 4.5 has been released and one Microsoft has made an announcement
can read more about it on the ‘Kernel Newbies’ regarding its association with the
page. Linus Torvalds has been working on the Eclipse Foundation, which is an open
Linux kernel 4.5 and now it has finally come to source group known for its Eclipse
the market. The new kernel patch offers driver IDE offering. The foundation also
updates for USBs, Watchdog, GPU, RDMA, ATA, offers many other developer tools. The
and sound; improvements for the ARM, x86, MIPS collaboration implies Microsoft will
and SPARC architectures, and file system (Btrfs, now shake hands with other Eclipse
CIFS, Ceph and JFFS2) enhancements. sponsors like Google, Novell, IBM,

Linux
The announcement was made by Linus Debeka and Oracle.
Torvalds on the Linux kernel mailing list. The announcement is a bit
Overall, everything is quite small and the

kernel 4.5
surprising as Microsoft offers its own
diffstat appears slightly bigger for an xfs IDE called Visual Studio. Besides,
fix, as that fix comprises three clean-up Microsoft plays an active role in the
refactoring patches preceding it. Eclipse ecosystem through its Azure
There is also an access type toolkit for Eclipse, along with Java
pattern affixed to the sound layer SDK for Azure which Eclipse users
which generates a lot of noise; can utilise for building their cloud
however, it’s pretty simple in the applications.
end. There are other random small
fixes all over. The kernel comprises a
shortlog, which is appended for people who would like to skim the details.
One can also get a great summary regarding all that is in the new version on
the ‘Kernel Newbies’ page.

Google joins Facebook’s Open Compute Project


Facebook’s Open Compute Project was aimed at accelerating the development of
Internet hardware as well as to increase its uptake. The increase in demand would
eventually drive down hardware costs.
The move became very According to Microsoft’s general
popular and saw big manager, Shanku Niyogi, “We have
names in the game joining worked with the Eclipse Foundation
the race. These included for many years to improve the Java
companies like Microsoft, experience across our portfolio of
HP and Quanta. There were application platform and development
only two companies who services, including Visual Studio Team
were left out—Google and Services and Microsoft Azure.”
Amazon. And in a recent The collaboration highlights
announcement made by Microsoft strengthening its role in
Google at the annual Open the open source ecosystem. Also,
Compute Summit held in the company has announced that
San Jose, California, the search giant has also joined the project and is working it will be open sourcing its Team
with Facebook on new open source hardware. Explorer Everywhere plugin for
As per Facebook’s vice president of infrastructure, Jason Taylor, “The Eclipse. Microsoft will be adding
community and Google have inched closer and closer together.” Azure IoT support to the Eclipse
Google’s announcement is also indicative of other changes within the big foundation’s Kura IoT framework,
Internet players. It marks the entry of artificial intelligence technology, which has and launching enhanced support for
been making rapid headway in many parts of the world. Java developers in Azure.

10 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


FOSSBYTES

Raspberry Pi 3 comes with 64-bit processor


and built-in Wi-Fi NGA releases seven tools under
The Raspberry Pi Foundation has launched the new Pi 3 with Wi-Fi open source licence
and Bluetooth, but at the price of Pi 2. The new version is an iterative The National Geospatial-Intelligence Agency
update on the Pi 2. (NGA) of the USA provides combat and
According to Raspberry Pi Foundation’s CEO, Eben Upton, the Pi 3 has intelligence agencies with critical and often
been in development for over a year. The new version comes with a quad-core classified intelligence. NGA is trying to share
64-bit 1.2GHz ARM Cortex A53 chip. This is about 50 per cent faster than and collaborate on geodata in the open, and
its predecessor. Upton further pointed out that, “Our primary goal in moving has expanded its transparency efforts with the
to A53 from A7 was to get a better 32-bit core. A53 running in 32-bit mode release of seven open source tools on GitHub.
outperforms A7 at the same clock speed by 20-30 per cent.” These are:
1. DigitalGlobe Reader: This is a C++ class
loading TIFF imagery and associated XML
metadata files, along with images obtained
from Digital Globe’s World View 3 satellite.
2. SWIR signal detection: This tool runs a
spectral signature comparison between
the provided spectral library file and a
signature provided by the user.
3. Social media picture explorer: This, along
with its user interface, utilises machine
learning techniques to assist users in
exploring social media by combining
similar images and offering instant object
recognition.
4. Nounalyzer: This tool categorises the nouns
and entities in a RSS feed and displays the
results through visualisations for analysts to
The new Pi 3 comes with the same 1GB RAM and the clock speed has distil information faster.
been upgraded to 900MHz from 450MHz. There is also an improvement in the 5. WordPress revision slider: This
VideoCore IV graphics from 250MHz to 400MHz. Both of these are based in tool modifies a WordPress theme to
a board that retains the same dimensions as the Pi 2. The addition of integrated interactively show a post’s revisions and
Bluetooth 4.1 and 802.11n Wi-Fi will help in reducing the need to look for get readers comfortable with consuming
component sites for cheap USB dongles. stories written with version control.
Those looking to exchange their old Pi with the new one will need to 6. Rational polynomial coefficients mapper:
upgrade their power source, as the Pi 3 requires a 2.5A input. The Pi 3 can be This tool maps an object’s latitude, longitude
bought from Element14, RS Components and the other usual stockists for US$ and altitude to a two-dimensional point.
35. It is also available at http://kitsnspares.com for ` 3300. 7. Spectral library reader: This reads files in
the splib06a spectral reflectance database
Open source adoption by the Indian government so that users can graph out the spectral
The government’s decision to switch over to open source options which are free reflectances of the materials in a multi-
could result in substantial cost savings but is also a matter of concern in terms of band image.
security and operational efficiency, both of which need to be addressed. The GEOINT Pathfinder project was
The government’s total IT spend was going to be US$ 66.98 billion in 2015 launched in August 2015, and assists the
as per the Ministry of Communications and IT. This was derived from several NGA in navigating the competitive world
global IT research and advisory forecasts. According to NASSCOM’s Data of commercial geospatial intelligence in an
Security Council of India, the average expense on cyber security is nearly 2 to 3 unclassified setting. According to sources in
per cent of the total IT spending, i.e., US$ 1.5 to US$ 2 billion. the NGA, the project endeavours to answer
As per Rahul De, Hewlett-Packard’s chair professor on ICT for sustainable research questions via unclassified tools, data,
economic development at IIM Bangalore, a major proportion—almost 15-20 information technology and services available
per cent—of the government’s IT spend is on closed software. Transitioning from public or open sources.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 11


FOSSBYTES

to open software would assist


TP Link blocks open source the government in cutting costs.
router firmware in US However, open source adoption
TP Link, a networking hardware is going to take a lot of time as
vendor, has said that it is going to stop it needs a certain ecosystem to
the loading of open source firmware work smoothly. He said that the
on routers that are sold in the United government’s policy towards it is
States. This is in compliance with a positive move.
the new Federal Communications De further stressed that since
Commission (FCC) requirements. open source software is available
The FCC is trying to restrict to the public, there are many who
interference with other devices by limiting feel that this makes it easy for
user modifications, which can result in malicious users to hack into it.
radios operating out of their licensed radio But he went on to clarify that this
frequency (RF) parameters. The FCC was not the case; rather, the opposite was true. Since the source code is available
stated that it is not aiming at banning the freely, more people work on such software and take care of its vulnerabilities.
use of third-party firmware like DD-WRT De also said that it’s tougher to hack into open source software than proprietary
and OpenWRT. Theoretically, router software. Among the first government organisations that adopted open source
manufacturers can still load open source were the US military forces. This was done since it was felt that open source
firmware, so long as they deploy controls software was a more secure option. Although users of open source can access the
which prevent devices from operating algorithm that makes it function, no one can access its encryption key or the set
outside their permitted frequency range, of numbers which act as a password. In the absence of the password, no one can
types of modulation, power levels, etc. hack into them.
However, users of open source One of the main reasons for the government’s push towards open source
firmware are worried that hardware is the need to incorporate vernacular languages in governance. Also, the
manufacturers are going to ban third- government is relying on BOSS (Bharat Operating System Solutions), which is
party firmware entirely, as it would be a Linux based operating system to meet this particular need and also to impart
the simplest way to comply with the improved knowhow on open source across India. However, experts are also of
FCC requirements. the opinion that the government should focus more on applications and then
move on to operating systems.
Further, the IT ministry has given its approval for a policy on ‘Collaborative
Application Development by Opening the Source Code of Government
Applications’. This policy is aimed at increasing the speed of application
development and their quick roll-out through the adoption of open source
principles. The government is aiming to encourage the reuse of already
developed applications. It also intends to boost innovation, both within the
government as well as outside, by boosting collaborative development to bring
As per the FAQ stated in TP-Link’s out better products, faster, sources claimed.
website, the above theory is confirmed.
The FAQ accepts that TP-Link is limiting White House to share custom code with open source
the functionality of its routers. The community
company stated that, “The FCC requires The White House has released the draft of its Source Code Policy that sets out
all manufacturers to prevent user[s] from rules regarding sharing of customised software among federal agencies. The
having any direct ability to change RF policy aims at improving government access to applications as well as lowering
parameters (frequency limits, output development costs. The policy also states that the Obama administration will
power, country codes, etc).” be launching a pilot programme, which will require federal agencies to release
TP-Link further stated that the nearly 20 per cent of the third-party-developed custom coding as open source
change is going to be in effect for software, making it completely accessible to external developers in the open
routers produced on and after June 2, source community.
2016, though the requirements will be As stated by the US CIO, Tony Scott, in a blog post, “Through this policy and
incorporated in a gradual manner. pilot programme, we can save taxpayer dollars by avoiding duplicative customer
software purchases, and promote innovation and collaboration across federal

12 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


FOSSBYTES

agencies. We will also enable the brightest minds inside and outside of government
to review and improve our code, and work together to ensure that the code is secure, Microsoft releases
reliable and effective in furthering our national objectives.” source code for SONiC
The administration has launched the policy with the aim of fulfilling its Microsoft has released the
commitment to adopt a government-focused open source software policy, which source code for Software for
was originally proposed in its Second Open Government National Action Plan. Open Networking in the Cloud
The new law sets out open sharing guidelines for any new custom software that has (SONiC). This is used for
been built for the US government websites and systems. These could be developed running network devices like
internally or via contracted third-party vendors, apart from cyber security software. switches, and has been built
in collaboration with leading
networking industry vendors
Arista, Broadcom, Dell and
Mellanox.
This will help in building
switches that are rich in
functionality. And with SONiC,
the hardware’s functionality
and applications can reduce the
dependence on a proprietary
firmware through a traditional
networking vendor.
SONiC is based on
Microsoft’s Linux-based Azure
Cloud Switch (ACS) operating
The new policy states that prior to agencies officially commissioning a new system. ACS is also the brain
custom software development project, they must carry out a thorough ‘alternatives behind switches in Microsoft’s
analysis’ to determine whether there is a pre-existing software solution at another Azure Cloud. The code is
agency whose source code can be shared for any important task at hand. Agencies capable of running on various
will also need to find out and consider any off-the-shelf software solutions which types of hardware from different
may be available, before commissioning a custom project. equipment manufacturers and
Even though the new guidelines are not applicable for software programs which utilises a common C API – the
were contracted via third parties before the Source Code Policy got published, it Switch Abstraction Interface
strongly motivates agencies to take steps in making those applications available for (SAI) for programming the
shared inter-agency use as well. specialist chips in the networking
gear. This implies that ACS
Linux Foundation and edX bring the Free Open Source Cloud can be used for controlling and
Infrastructure Course managing network devices. It
The new Massive Open Online Course (MOOC) will permit individuals worldwide can also implement the features
to embark on lucrative careers by building and managing cloud technologies. The needed, irrespective of who has
Linux Foundation made an announcement that will enable mass innovation via open manufactured the hardware.
source. It announced its latest MOOC, which is open for registration. According to Azure’s CTO,
The MOOC is an introduction to cloud infrastructure technologies and is Mark Russinovich, “SONiC is a
conducted via edX, the non-profit online learning platform launched in 2012 by collection of software networking
Harvard University and Massachusetts Institute of Technology (MIT). The course components required to build
is free and will commence from network devices like switches.”
June 2016. The course is the SONiC can be downloaded
second edX MOOC being offered from Microsoft’s Azure GitHub
via the Linux Foundation. The repository under a mix of open
Foundation’s first course, an source licences, which include
‘Intro to Linux’, has attracted the GNU GPL and the Apache
over 500,000 students worldwide licence.
and continues to grow.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 13


FOSSBYTES

According to Jim Zemlin, executive director at The Linux Foundation, “The


response to our first edX course was powerful, demonstrating the understanding
among students and early career professionals all over the world that open source
is the pathway to a lucrative career in technology.” He added that through the
introduction to cloud infrastructure technologies, anybody can start learning the basics
of building and managing some of today’s most pervasive software, thus granting
professionals a strong position in the IT talent market.
LFS151, which is what the course is called, will offer a primer on cloud computing
and the use of open source software to maximise development and operations. The course
will cover next-generation cloud technologies including Docker, CoreOS, Kubernetes and
OpenStack. It will offer an overview of software-defined storage and networking solutions,
along with a review of DevOps and continuous integration best practices.
The course has twelve chapters. Each chapter has a short quiz at the end. One also
needs to take a final exam for completing the course. Students can take the complete
course at no cost, or add a verified certificate of completion for US$ 99.

Microsoft to open source Minecraft based AI platform


Microsoft plans to open source its AI platform, which is based on Minecraft.
Microsoft will be opensourcing a platform that is being used by researchers
for testing in artificial intelligence projects. The AIX platform is already being
utilised by Microsoft researchers. It permits researchers to utilise the unstructured
play in the Minecraft game as a testing ground for AI research.

AIX is going to be open sourced by summer this year under an open source
licence. The announcement has come at a time when Google DeepMind has gained
attention for the Go games, which are being played by its AI program AlphaGo with
a key Go player, Lee Se-dol. AlphaGo has won three straight games of the five-game
match in Seoul, losing one to Se-dol.
With AIX, Microsoft is focusing on projects which involve general intelligence.
Microsoft feels it “…is more similar to the nuanced and complex way humans learn
and make decisions.”
According to Allison Linn, a senior writer at Microsoft, AI researchers are
developing tools which involve recognising words, for example. However,
researchers are not being able to combine these skills effortlessly as humans do.
Microsoft had acquired Mojang, the developer of Minecraft, in 2014. Linn
also stated that the AIX platform, which was developed by Microsoft’s lab in
Cambridge, UK, comprises a ‘mod’ for the Java version and code, which assists
artificial intelligence agents to sense and act within the Minecraft environment.
Both these components are capable of running on Windows, Linux or Mac OS, and
researchers can use any programming language to program their agents.

14 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


In The News

PGDay India 2016:


A SUMMARY OF THE SHOW
A conference that witnessed experience-sharing on a wide range of topics by PostgreSQL users and
developers was held in Bengaluru on February 26, 2016. Here is a summary of the day.

T
he annual PostgreSQL PGDay event was held on cost savings, with no compromise in performance or
February 26, 2016, at Courtyard functionality, was well received. Speakers
by Marriott in Bengaluru. from Inmobi, a company that has an
The event was a grand success with enterprise-scale usage of PostgreSQL,
over 150 attendees representing 50+ talked about performance tips to get the
companies. This number was more best out of this database, when used on
than double the last year’s number of a large scale. The talk around Postgres-
participants. What was also interesting XL, which is a distributed multi-master
to note was that almost 80 per cent of hybrid OLTP/OLAP offering, was also
the attendees were from outside the city. well received.
Robert Haas, well-known The event concluded with the DBA
PostgreSQL contributor, made the track, in which useful insights were given
keynote address, talking about the past, present into how to detect and avoid data corruption, apart
and future of this open source database. There were talks from inputs about DBA maintenance activities.
on Vacuum, foreign data wrappers, logical replication, etc, With an impressive sponsors list comprising
in the developer track. 2ndQuadrant, EnterpriseDB, GE Predix, Ashnik,
The speakers gave some great insights into customer Tashee Linux Services and OpenSCG, the event was
use cases involving PostgreSQL. The presentation by a grand success. The enthusiastic response augurs
speakers from the National Informatics Centre got the well for the increased adoption of PostgreSQL in the
most applause, thanks to its wholehearted endorsement years to come.
of PostgreSQL in a myriad projects, ranging from
police-FIR systems and education certificates to online Reported By:
appointments at government hospitals. The government’s Ashish Mehra, Nikhil Sontakke and Pavan Deolasee.
policy of supporting open source and the tremendous All three were a part of the organising team for the event

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 15


CUSTOMER SUCCESS SNAPSHOT

YOU SAID IT
CRIS BOOKS MORE, HAPPIER
PASSENGERS WITH INFRASTRUCTURE
POWERED BY RED HAT

CHALLENGE
• Support nearly 2 million passengers a day on over 2,500 trains nationwide.

• Handle a workload of 25,000 concurrent users during peak booking periods.

• Deploy a highly scalable solution based on open standards.


2014 Innovation Award Winner
SOLUTION
• Deployed Red Hat® Enterprise Linux® as the underlying infrastructure for mission-critical systems,
increasing scalability to support online ticket bookings.

• Developed the CRIS Journey Planner (JP), an application for passengers based on open JavaTM EE 6
Centre for Railway
Information Systems (CRIS) standards, which caters to 25,000 concurrent users and provides a high-performing, user-friendly,
and highly available interface.
www.cris.org.in
• Collaborated closely with Red Hat Consulting during the planning, execution, and implementation
of the projects.
GOVERNMENT INDUSTRY
BENEFITS

CUSTOMER SINCE • Gained an operational, state-of-the-art journey planning system in the most cost-effective

2006 manner with shortest time to market.

• Avoided vendor lock-in and achieved interoperability by integrating Red Hat Enterprise Linux
into the existing infrastructure.
SOFTWARE AND
SERVICES • Increased customer satisfaction by developing and launching a reliable, streamlined train journey
Red Hat Enterprise Linux planning system.
Red Hat JBoss® Enterprise
Application Platform
Red Hat Consulting “ We look to Red Hat for technology
leadership through updates and
HARDWARE support that help us to improve
10 servers of Intel Xeon New Delhi, India our application tuning so that we
6-core processor can give our customers the best
possible experience.“
SUNEETI GOEL
CHIEF PROJECT ENGINEER,
HEADQUARTERS PASSENGER RESERVATION SYSTEM GROUP, CRIS

facebook.com/redhatinc
@redhatnews
linkedin.com/company/red-hat Learn more about Red Hat customer successes: redhat.com/customersuccess

redhat.com Copyright © 2014 Red Hat, Inc., Red Hat, Red Hat Enterprise Linux, the Shadowman logo, and JBoss are trademarks of Red Hat, Inc.,
#12022727_v3_0414 registered in the U.S. and other countries. Linux® is the registered trademark of Linus Torvalds in the U.S. and other countries.
CODE
SPORT
In this month’s column, we discuss the topic of automatic relation extraction from
Sandya Mannarswamy

text using natural language processing techniques.

I
n last month’s column, I had described certain siblings. Write an algorithm to determine whether
techniques for information extraction from the tree T contains A and B at the same level. What
unstructured texts using natural language is the execution time complexity of your code?
processing. In this month’s column, we take a break 5. You are given an array of size N. Each element of
from this and discuss a few interview questions in the array can take only one of the three values (0, 1
general computer science and data science. or -1). You are told to find the longest contiguous
1. Given an arbitrary integer N, how many unique sequence of 1s in the array. Write a C function to
binary trees can be created containing N nodes? find the start and end of the contiguous sequence
Can you come up with an algorithm which can of 1s in the array. Now you are told that you can
programmatically list all possible binary trees flip K contiguous sequence of (-1) to 1 where K is
consisting of N nodes? a dynamically provided input to the function. In
2. You are given two unsorted lists of integers. You that case, how would you determine the longest
are asked to create a sorted list consisting of all contiguous sequences of 1s in the array after
the elements in the two lists. Can you write an flipping K elements which were -1 to 1? What is the
algorithm for this? What is the running time of time and space complexity of your function?
your code, in the worst case? 6. Given an array of N integers, you are asked to find
3. You are given a list of N nodes in a tree, with each out whether the given array is an arithmetic series
node represented by an integer. For each node, you (a sequence of numbers where there is a constant
are also given the details of its parent node. You difference between any two consecutive numbers
are not given the edges of the tree explicitly, but of the series). What is the complexity of your
you are told that it is a binary tree. You are asked function? Now, you are given a modified problem
to write code to perform an in-order traversal of where your task is to convert a given array into an
the tree using this information. How would you arithmetic sequence by dropping off those array
solve this problem: (a) If you are told that you can elements which prevent it from being an arithmetic
perform pre-processing on the information and can sequence. How would you solve this problem?
use any amount of additional storage; (b) You are What is the time complexity of your function?
told that you can use only a constant amount of 7. Given an arbitrarily long character string, you are
additional storage. asked to find out the most frequently occurring
4. You are given an arbitrary tree T (not necessarily character as well as the least frequently occurring
binary), with each tree node containing a pair of character. Write a C function to solve this problem,
integers (X and Y). Each tree node can have an given a character string of size N. Now you are told
arbitrary number of children nodes, at each level that instead of being a fixed size character string,
of the tree. You are given two numbers, A and B, you are given a stream of characters which come
and are told to find out if the nodes N1 and N2 dynamically as input. At any point in time, you
containing the integers A and B are located at the should be able to print the most frequent and least
same level of the tree. Note that N1 and N2 can frequent characters in the stream (note that if two or
be two separate nodes or be the same node. It is more characters have the same highest frequency,
only required that N1 and N2 should be located you can choose to print any of them as the most
at the same level of the tree, and they need not be frequent characters). What is the time and space

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 17


CodeSport Guest Column

complexity of your algorithm? time and space complexity of your function?


8. You are given an array of N integers and told that the 13. You are given a long piece of text T and three sub-strings
array is mostly sorted, with only k elements not being in S1, S2 and S3. You are asked to find the shortest sub-string
their correct sorted position in the array, and with k being X of T such that X contains S1, S2 and S3 as sub-strings.
very small compared to N. What is the sorting algorithm What is the time and space complexity of your solution?
you would use to completely sort the array? 14. You are given an array of N integers (where N can be of
9. You are given an array of N integers. You are asked to the order of millions) which has been sorted. Given an
find the maximum and minimum elements in each of arbitrary value K, you are asked to find out how many
the sub-sequences of length K of the array, where K is times it repeats in the array? Can you write a C function
an arbitrary integer. (a) Given an array of N integers to do this in O (log N) time complexity?
and a value K, how many sub-sequences of length K are 15. Consider the following code snippet:
possible? (b) What is the complexity of computing the
maximum and minimum for each of the sub-sequences int main(int argc, char *argv[])
of length K? How would you solve this problem in {
polynomial time, for any arbitrary value of K? void *my_mem = 0;
10. You are given a list of N integers, where the numbers can int alloc_blocks = 0;
only be positive. You are now being asked to write code to while(1)
create two lists of size N/2 such that the sum of the elements {
of the two sub-lists are equal, if at all it is possible to create my_mem = (void *) malloc(1024 * 1024);
such sub-lists. Your function should return the sum of the if (!my_mem) break;
sub-list elements if it is possible to split the original list into memset(my_mem,1, (1024*1024));
two such sub-lists, or return 0 if that is not possible. What is printf(“allocated memory blocks is %d MB\
the time complexity of your function? (Note that you can n”,++alloc_blocks);
move the elements of the list.) You are now told that the }
original list can either contain positive or negative integers, exit(0);
and are told to solve the above problem. What are the }
changes you would make to your earlier solution?
11. Consider the problem given in (10). Now, instead of Can you explain what would happen when you compile
splitting the list into two sub-lists such that the elements and run the program?
of both are equal, you are asked to split the list at any If you have any favourite programming questions/
arbitrary index such that the difference of the sum of the software topics that you would like to discuss on this forum,
two sub-lists is the minimum among all possible splits of please send them to me, along with your solutions and
the original list. Unlike problem (10), you cannot move feedback, at sandyasm_AT_yahoo_DOT_com. Till we meet
around the elements of the list. What is the time and space again next month, happy programming!
complexity of your function?
12. You are given two sentences S1 and S2, each containing By: Sandya Mannarswamy
N and M words, respectively. You are now asked to
The author is an expert in systems software and is currently
convert S1 into S2 by either deleting a word or adding a
working as a research scientist in Xerox India Research Centre. Her
new word. Modifying word1 to word2 can be considered interests include compilers, programming languages, file systems
as word deletion, followed by word addition. Each insert and natural language processing. If you are preparing for systems
operation has a cost of +2 and each delete operation has software interviews, you may find it useful to visit Sandya’s LinkedIn
a cost of +1. Given two arbitrary sentences S1 and S2, group Computer Science Interview Training India at http://www.
the edit distance of (S1, S2) is the minimum total cost of linkedin.com/groups?home=HYPERLINK “http://www.linkedin.com/
groups?home=&gid=2339182”&HYPERLINK “http://www.linkedin.
operations needed to convert S1 into S2. You are asked to
com/groups?home=&gid=2339182”gid=2339182
write a function to compute the edit distance. What is the

THE COMPLETE MAGAZINE ON OPEN SOURCE Your favourite Magazine on Open


Source is now on the Web, too.
OpenSourceForU.com Follow us on Twitter@LinuxForYou

18 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Guest Column Exploring Software

Anil Seth
Visualising Data Using googleVis
Viewing statistics in motion charts can be great fun. This article
describes how googleVis software can be used to view motion
charts. The author gives the reader an interesting example of data
picked up from the Bombay Stock Exchange and how it can be
viewed as a motion chart.

V
iewing statistics can be an exciting experience. If a sample session in ‘An Introduction to R’ which is included
you are surprised, look at one of the motion chart in the R distribution. You can access the built-in help
presentations by Hans Rosling, at his TED talk https:// including the introduction as follows:
www.youtube.com/watch?v=RUwS1uAdUcI. The software
behind the presentation of motion charts was acquired by > help.start()
Google and integrated into the Google Charts API. starting httpd help server ... done
Normally, I would have experimented with the Python
API. However, I came across the R package ‘googleVis’ This will bring up HTML documentation. You can
on GitHub and decided to experiment with that instead. It navigate to ‘Appendix A: A sample session’ in ‘An
provides a motivation to experiment with R. Introduction to R’.
R is a statistical tool, which assumes considerable
significance in the context of making sense of Big Data. Exploring googleVis
Now, let’s get back to understanding and exploring the
Installing googleVis googleVis package.
You need to install R and then the googleVis package under ‘Fruits’ is a data frame included in the distribution
R. For example, on Fedora, use the following commands: of the googleVis package. The command data() will list
all the data frames available. However, we are currently
$ sudo dnf install R interested in the googleVis package only. So, find out the
$ sudo R data frames available in googleVis, load Fruits and print its
>install.packages(“googleVis”) contents as follows:

You may be prompted to select a repository from which to > data(package=”googleVis”)


install R packages. Once the installation is complete, you can Data sets in package ‘googleVis’:
test it with just two commands, as follows: Andrew Hurricane Andrew: googleVis example
data set
$ R Cairo Daily temperature data for Cairo
> suppressPackageStartupMessages(library(googleVis)) CityPopularity CityPopularity: googleVis example
> plot(gvisMotionChart(Fruits, “Fruit”, “Year”, data set
options=list(width=600, height=400))) Exports Exports: googleVis example data set
starting httpd help server ... done Fruits Fruits: googleVis example data set
….
You will get a 600x400 (width x height) motion chart > data(Fruits)
in the browser and can play around with various bubbles > Fruits
moving with time. You can find videos of googleVis charts on Fruit Year Location Sales Expenses Profit Date
YouTube in case you are interested. 1 Apples 2008 West 98 78 20 2008-12-31
2 Apples 2009 West 111 79 32 2009-12-31
Getting started with R ...
In case you are not already familiar with R, at this point, try 9 Bananas 2010 East 81 71 10 2010-12-31

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 19


Exploring Software Guest Column

You can learn more about googleVis in the R SmallCap and IT indices, the properly organised data will
interpreter, with the following: look like what follows:
ƒ help(googleVis) which will give a brief description of the
package and guide you to further information about it Date Index Open High Low Close Hi-Lo
ƒ demo(googleVis) which will guide you through 2016-02-29 Sensex 23238.5 23343.22 22494.61 23002 848.61
examples of charts included in the package, including 2016-02-29 Midcap 9584.31 9651.77 9389.35 9575.1 262.42
gvisBubbleChart and gvisMotionChart 2016-02-29 Smallcap 9567.63 9594.03 9399.43 9548.33 194.6
ƒ vignettes(googleVis) which will give you an introduction 2016-02-29 IT 10488.4 10507.62 10044.59 10229.49 463.03
to the package
Save this sheet in CSV format as BSE-indices.csv. This
Experimenting with stock market indices file can be easily imported as a data set in R, though the date
The next step is to use your own data. An easy and interesting field needs to be explicitly identified.
resource is to download historical indices data from the BSE
(Bombay Stock Exchange) site. You can save the data as CSV > D=read.csv(“BSE-indices.csv”, header=TRUE,colClasses=c
files. The daily data will look something like what follows: (Date=”Date”))
> M <- gvisMotionChart(D,idvar=”Index”,timevar=”Date”,
Date Open High Low Close sizevar=”Hi.Lo”)
1-January-2016 26101.5 26197.27 26008.2 26160.9 > plot(M)
4-January-2016 26116.52 26116.52 25596.57 25623.35
5-January-2016 25744.7 25766.76 25513.75 25580.34 Notice that the ‘Hi-Lo’ becomes ‘Hi.Lo’ in R. This value
is used as the default value for the size of the bubble in the
You can merge and organise the data from the various above chart. The chart will open in a browser window and
CSV files in a spreadsheet so that it is suitable for importing needs Flash Player. Figure 1 shows the initial image.
in R. The YMD date format is very convenient as there is The Open column is used as the x-axis and the next
no ambiguity. You may want to add a column that shows the column, High, is used as the y-axis. However, the columns
difference between the high and low values as an indicator of used for the x and y axes can be changed in the chart. The
volatility. Suppose you want to look at the Sensex, MidCap, colours for various indices can also be changed. If you

MotionChartID272865...
MotionChartID2722865...
127.0.0.1:15264/custom/googleVis/MotionChartID2728658d2e73.html
127.0.0.1:15264/custom/googleVis/MotionChartID2728658d2e73.html

Color Sensex, 1/1/16 Color


Lin
Lin

Low Unique colors


25,000 25,000
Size
20,000 Hi.Lo

10,000
Bank, 1/1/16 1000
Size
20,000 Select Deselect all
20,000 Hi.Lo
Bank
Health
1000
IT
Select Midcap
Bank Sensex
15,000 15,000
Health Smallcap
IT
Trails
Midcap IT, 1/1/16
Sensex
Smallcap
10,000 10,000
Close

Trails
High

10,000 15,000 20,000 25,000 Jan 2015 - Feb 2016

Open Lin Time


1/1/16 2/29/16

Data: D • Chart ID: MotionChartID2728658d2e73 • googleVis-0.5.10 Data: D • Chart ID: MotionChartID2728658d2e73 • googleVis-0.5.10
R version 3.2.3 (2015-12-10) • Google Terms of Use • Documentation and Data Policy R version 3.2.3 (2015-12-10) • Google Terms of Use • Documentation and Data Policy

Figure 1: Initial display of a chart Figure 2: A chart showing some trails as the bubbles move in time

20 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Guest Column Exploring Software

click on the Play option, the values and bubble sizes will
MotionChartID272865... change with time.
127.0.0.1:15264/custom/googleVis/MotionChartID2728658d2e73.html Change the x-axis column to Date, the y-axis column
to Closing and colours to Unique. Select Bank, IT and
12,500 Color
Sensex indices for showing the trails. You can see the
Lin

Smallcap, 1/1/16 Unique colors


12,000 Size results in Figure 2.
Hi.Lo You may notice that the values of some of the indices
Midcap, 1/1/16
11,500
IT, 1/1/16
1000 are too close to each other to be viewed effectively. The
11,000
Select Deselect all chart has the option to select an area and zoom in, as you
Bank
Health can see in Figure 3.
10,500
IT
Midcap
The range of data sets available in the public domain
Sensex is increasing, some of which you may explore on https://
10,000 Smallcap

Trails www.google.com/publicdata/. The data made available


9,500 Zoom Zoom out by various government departments in India is available
through https://data.gov.in/. Visualisation tools like the one
Close

9,000
Jan 2016 Feb 2016
discussed above help you make sense of it all.
Time
2/29/16
By: Dr Anil Seth
Data: D • Chart ID: MotionChartID2728658d2e73 • googleVis-0.5.10
R version 3.2.3 (2015-12-10) • Google Terms of Use • Documentation and Data Policy The author has earned the right to do what interests him. You can
find him online at http://sethanil.com and http://sethanil.blogspot.
com, or you can reach him via email at anil@sethanil.com.
Figure 3: After zooming in, a chart showing some trails of the movement of bubbles

Please share your feedback/ thoughts/


views via email at osfyedit@efy.in

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 21


Admin Let’s Try

vnStat: A Lightweight Network Traffic Monitor


vnStat is a Web based interface to view and monitor network traffic bandwidth usage in a nice
graphical format. It is a command line tool and its integration with PHP gives it a graphical frontend.
Here’s a tutorial on installing and using vnStat with a PHP frontend.

To view all the available interfaces on your system that


vnStat can monitor, use the following command:
vnStat
[root@sun ~]# vnstat --iflist
Available interfaces: lo eth0 eth1

Now add the eth0 and eth1 interfaces for monitoring


traffic, as follows:

[root@sun ~]# vnstat -u -i eth0


Error: Unable to read database “/var/lib/vnstat/eth0”
Info: -> A new database has been created

For the first time, it will display the above error. Ignore this
as sufficient data is not available for display.
Similarly, add the second interface, i.e., eth1, as follows:

T
here are occasions when it is useful to know your [root@sun ~]# vnstat -u -i eth1
bandwidth usage pattern. This not only helps Error: Unable to read database “/var/lib/vnstat/eth1”.
administrators to detect the root cause of traffic related Info: -> A new database has been created.
issues, such as a network overload, but also helps them to
keep a tab on traffic flow to and from the Internet, and the Method for updating the database
charges levied by the ISP for the bandwidth used. The database can be updated by either using cron or service.
This is an important task which requires dedicated and Cron: To use cron, just remove the comment from /etc/
effective software. vnStat is just the tool for that. It is a cron.d/vnstat, after which the entry should be as shown below:
lightweight (command line) network traffic monitor, which
provides several output options like summaries on an hourly, #vi /etc/cron.d/vnstat
daily, monthly, weekly or ‘top 10 days’ basis. It can monitor MAILTO=root
selected interfaces, stores network traffic logs in a database */5 * * * * vnstat /usr/sbin/vnstat.cron
for further analysis and can be used without root permission.
:wq save and quit
Prerequisites for installation Service: The second method uses vnStatd a program that
I have chosen a system with Centos version 6.7 (equivalent to collects all the information and is run as the daemon. You can
RHEL 6.7) with minimal installation, which acts as a gateway start it with executing below command:
through which eth0 is connected to the Internet and eth1 is
connected to the local network. First, install all the required [root@sun ~]# /etc/init.d/vnstat start
packages and dependencies for vnStat and the PHP Web
frontend using the Yum utility. Enable service at boot time

[root@sun ~]# yum install wget epel-release php httpd [root@sun ~]# chkconfig --level 345 vnstat on
php-gd
To start traffic monitoring for selected interfaces, start
Installation vnstat service.
To install vnStat, execute the command given below in a
terminal: [root@sun ~]# cd /usr/local/src

[root@sun ~]# yum install vnstat Download the latest vnStat PHP frontend

22 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try Admin

You can access vnStat with the PHP frontend by


Traffic data for public Internal (eth0) pointing your browser to http://ipaddress/vnstat, i.e.,
Summary http://192.168.1.140/vnstat/
In Out Total Allow port 80 on the server from outside, add rule in
This hour 10.52 MB 1.00 MB 11.52 MB
This day 10.52 MB 1.00 MB 11.52 MB iptables by executing below command:
This month 50.33 MB 8.64 MB 58.97 MB
All time 50.33 MB 8.64 MB 58.97 MB
Top 10 days [root@sun src]#/sbin/iptables -A INPUT -m state --state NEW
In Out Total
-p tcp --dport 80 -j ACCEPT
16 February 2016 27.08 MB 2.09 MB 29.18 MB [root@sun src]#/etc/init.d/iptables save
18 February 2016 12.15 MB 2.36 MB 14.51 MB
10 February 2016 401.00 KB 2.64 KB 3.03 MB [root@sun src]#/etc/init.d/iptables restart
03 February 2016 166.00 KB 239.00 KB 405.00 KB
09February 2016 24.00 KB 315.00 KB 339.00 KB

To secure the vnStat PHP frontend with user authentication,


Figure 1: vnStat PHP frontend main page add an entry in httpd.conf at the end as shown below:
<Directory /var/www/html/vnstat>
from http://www.sqweek.com, as follows: AuthUserFile /var/www/vnstat-htpasswd
AuthName “vnStat user”
[root@sun src]#wget -c http://www.sqweek.com/sqweek/files/ AuthType Basic
vnstat_php_frontend-1.5.1.tar.gz require valid-user
</Directory>
Extract the downloaded archive in a Web-accessible
directory, e.g. /var/www/html/ After making changes in the httpd.conf file, restart the
httpd service:
[root@sun src]## tar xzf vnstat_php_frontend-1.5.1.tar.gz
[root@sun src]## mv vnstat_php_frontend-1.5.1 /var/www/html/ [root@sun src]#service httpd restart
vnstat
Create file /var/www/vnstat-htpasswd
Edit the config.php file and set the following parameters as
per your set-up: [root@sun src]#touch /var/www/vnstat-htpasswd
[root@sun src]#htpasswd -cm /var/www/vnstat-htpasswd
[root@sun src]#vi /var/www/html/vnstat/config.php vnadmin
$language = ‘en’;
This will add the user, with Force SHA encryption
$iface_list = array(‘eth0’, ‘eth1’); of the password.
$iface_title[‘eth0’] = “Public Interface”; Now point your browser to http://192.168.1.140/vnstat/. It
$iface_title[‘eth1’] = “ Local interface”; will ask for the user name and password. After entering both,
you will be able to see the vnStat PHP frontend main page as
If you are using SELinux, restore the file(s) default shown in Figure 1.
SELinux security contexts by using the restorecon command: You can get details like the summary, hourly, daily and
monthly graphical reports of bandwidth usage for configured
[root@sun src]# restorecon -Rv /var/www/html/vnstat/ interfaces.

Make sure that the correct local time zone is set in php.ini.
Edit php.ini and set date.timezone =”Asia/Kolkata” References
[1] http://humdi.net/vnstat/
[root@sun src]#vi /etc/php.ini [2] http://linuxaria.com/recensioni/monitot-your-bandwidth-
date.timezone =”Asia/Kolkata” on-linux-with-vnstat>
[3] http://www.sqweek.com/sqweek/index.php
[4] http://www.thegeekstuff.com/2011/11/vnstat-network-
Now enable web service at boot
traffic-monitor/

[root@sun src]#chkconfig --level 345 httpd on


By: Suresh M. Jagtap
Start the Web service httpd as follows: The author is a Linux enthusiast who loves to explore various Linux
distributions and open source software. You can contact him at
suresh@linuxsecret.com
[root@sun src]#service httpd start

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 23


Admin Let’s Try

Use Sync for Stress-free Data Backup


A disk crash or a corrupted file can have disastrous consequences. A safe bet is to
backup one’s data. The author narrates his personal experience with data backup issues
and gives a tutorial in the use of Sync, which he recommends for home users.

M
ost of you are probably active users of various maintain two backup copies of data. In other words, I have the
services like Gmail, Flickr, Google Photos, Apple actual data in local storage, as well as a backup of the data on two
iPhotos and Instagram, or at least, must be aware different portable HDDs (hard disk drives). Due to this approach,
of them. Many of you might also be users of digital storage I need to use a well-defined and simple backup process.
options like Dropbox. While these services do give you the
flexibility of easy availability of data anywhere in the world A survey of personal backup solutions
(assuming you have the bandwidth to access the data), you Over the years, I have used many methods to maintain backup
need to maintain a local copy (on your laptop or desktop) copies of data. The first method obviously was to maintain a
of all your data, much as vendors may try to convince you copy of each file. Then I started storing compressed versions
that all the data is safe with them. In case the local copy gets of the data and using a timestamp for each backup. Just as
corrupted, one can always depend on cloud storage. enterprises have had to address the Big Data explosion, I too
While data stored in the cloud is mostly safe, there have been have not been immune to the problem of large volumes of
cases where the cloud service provider has accidentally deleted data. Due to data volumes and the frequency of changes, the
user data. Additionally, data can be lost in case there is a data timestamp method of maintaining backups is not practical.
breach and the cloud provider’s service gets compromised or Additionally, each zip archive ends up with one copy of the
passwords get stolen. Considering these possibilities, some of us file and when a particular copy is needed, it is difficult to
are not comfortable in relying entirely on cloud services. figure out the most relevant copy.
Personally, I make limited use of cloud-based storage I also tried using version-control software like SVN.
services and mostly depend on keeping data in local storage. The problem with version control is that it’s sub-optimal
Having made the decision of ‘going local’ I need to take steps to for binary data. In other words, version control is best
ensure that the data is properly backed up, so that I will have a suited for text files, where versions are stored using the
copy of the data, in case the primary storage fails. For important ‘difference’ method. In the case of binary files, most such
things like photographs, email and financial documents, I solutions simply copy the binary file into the repository, as

24 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try Admin

finding a ‘difference’ between two binary files is not simple command line. I can almost hear you grumble: “Ew! The
and straightforward. Ideally, version control software, when command line. How can you even recommend a command
used, should be hosted on a separate server, which is not an line application in these days of GUI systems?” But, in my
option at home. Additionally, maintaining the repository on an experience, this command line interface provides me the
external HDD was next to impossible. maximum flexibility.
Then I started searching for personal backup software that On my local machine, I always have a JVM instance
would allow me to maintain backups of data and that, too, on available either as a JDK or a JRE, along with a properly
external media like a portable HDD. I tried multiple solutions, configured PATH variable and a JAVAHOME variable. To
two of them being Microsoft SyncToy and Cobian Backup. make the task of backing up easier, I have created batch files
While I liked each of these tools (with a preference for Cobian that contain the backup command (using Sync). The batch files
Backup), I found that the GUI interface, and the way these tools are then stored on the portable device. Thus, whenever I wish to
stored the backup commands, created a problem that needed take a backup, I simply connect the portable device and execute
additional effort each time I wished to use the tools for backup. the backup script. As the source directory does not change
While using these tools, I faced a problem with the backup (being local storage) and the destination directory is specified
commands due to the way Windows uses external media. using a relative path, the batch scripts do not need to be edited –
Whenever external media is connected to a computer, there is irrespective of what drive letter is assigned to it. When launched,
no guarantee that it will be assigned the same drive letter as the script executes in a dedicated command prompt and I can
the one assigned the last time the media was connected. For continue with other activities. I only need to monitor the progress
example, if the connected media is assigned the drive letter once in a while. Once the backup is complete, I go through the
H: today, there is no guarantee that it will be assigned the log file to check for errors and my backup work is done.
same drive letter tomorrow. If you have other media already By using a Windows batch file, my backup activity is
connected, the portable HDD will get assigned the next reduced to a double click or an ‘Enter’ key-press, which is all
available drive letter. As the backup commands configured that’s needed to launch the script. The scripts have made the
in these tools refer to an absolute path that includes the drive task so simple that even after maintaining two backup copies
letter, the command does not work if the drive letter is different. of my personal photographs, email and other documents, the
If the external HDD is assigned a different drive letter, you will only trouble I face is that of maintaining a regular schedule to
have no choice but to edit the backup command to point it to keep the backup copies suitably fresh.
the present drive letter. Now imagine the effort needed if such
changes have to be applied to multiple backup commands, each Using Sync
time you wish to take a backup. To use Sync, the command format is as follows:
While searching for more tools, I came across a simple
backup tool, namely, Sync (alternately named Syncdir). java -jar Sync.jar <switches> [“Source”] [“Target”]

Introducing Sync Using this command synchronises the [“Target”] to


Sync is a Java application that needs to be invoked from the match the [“Source”]. It should be noted that only the

Figure 1: Taking backups for the first time Figure 2: Updating the backup. New files added

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 25


Admin Let’s Try
--synctime:[y|n] Synchronise the time of matched target
files
--overwrite:[y|n] Overwrite existing target files/directories
--delete:[y|n] Delete unmatched target files/directories
--force Equivalent to the combination:--rename:y
--synctime:y --overwrite:y --delete:y

Batch script: sync-generic.bat


Given below is a ‘generic’ script that can be invoked by other
backup scripts. The reason for using such a script is that the
actual backup script ends up with only minimal details and is
thus easier to maintain. Additionally, changes to the commands,
if any, when done in a central script work for all dependent
scripts, reducing maintenance efforts.

@echo off
Figure 3: Updating the backup. A file has been updated set SYNC_HOME=./Sync.jar
rem deleting existing log file
[“Target”] is modified and, by default, the file name, size, del log\%1.log
last-modified time and a CRC-32 checksum of the file are java -jar Sync.jar --log:log/%1.log --force %2 %3
used to match files. If [“Source”] is a directory, the source echo Backup for %1 done.
and target directories are matched recursively. Matched
target files are time-synced and renamed if necessary; Batch script: backup-doc.bat
unmatched source files are copied to the target directory Given below is the script that takes a backup of the ‘doc’
and unmatched target files and/or directories are deleted. directory and uses the generic script for this purpose.
If [“Source”] is a file, source and target files are matched,
while ignoring the file name. If the files match, the target file @echo off
is time-synced and renamed if necessary; if the target does set NAME=doc
not exist, the source file is copied to the target. call sync-generic.bat %NAME% D:\%NAME% .\%NAME%
pause
A few switches
Some of the switches that can be used with Sync are: While there are many personal backup solutions, my
-s, --simulate Simulate only; do not modify target tool of choice is Sync – a Java based command-line tool.
Though it does not have a fancy GUI, it helps me maintain
--ignorewarn- Ignore warnings; do not pause backups with little effort. To make the task of backing up
ings easier, I use Windows batch files, which only require me to
-l, --log:<”x”> Create log file x; if x is not specified, double-click (or press an Enter key) the script, to kick off
sync.yyyyMMdd-HHmmss.log is used the backup process.

-r, --norecurse Do not recurse into sub-directories


-n, --noname Do not use file name for file-matching References
[1] Syncdir, http://syncdir.sourceforge.com/
-t, --notime Do not use last-modified time for file- [2] Best free backup software: http://www.in.techradar.com/
matching news/software/applications/Best-free-backup-software-11-
programs-we-recommend/articleshow/38877922.cms
-c, --nocrc Do not use CRC-32 checksum for file- [3] 13 Best backup software: http://www.pcadvisor.co.uk/
matching test-centre/software/13-best-backup-software-2015-2016-
uk-3263573/
--time:[x] Use an x-millisecond time-tolerance for
file-matching (0-millisecond time-toler-
ance is used by default; use --time:1000 By: Bipin Patwardhan
or more to avoid mismatches across
The author is a senior technical architect at Capgemini India. He
different file systems) believes in the value of open source and has developed many
--rename:[y|n] Always[y]/never[n] rename matched customer solutions using open source Java libraries like Apache
XMLBeans.
target files

28 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Overview Admin

Sharing Resources Across


Diverse OSs with Samba
Welcome to the world of Samba. Being software that can be run on multiple platforms such
as Windows, UNIX, Linux, etc, it enables you to share your files and printers between systems
running various OSs. It works as an alternative to NFS.

S
amba is an open source network file system. It allows A Microsoft RPC call, the name service switch (NSS),
access to shared resources across the different OSs and pluggable authentication modules (PAM) allow this
such as UNIX, Linux, IBM System 390, OpenVMS, daemon to achieve this. This allows Windows NT domain
etc. It is an open source implementation of Microsoft’s users to access a UNIX machine as UNIX users. The
SMB (Server Message Block)/CIFS (Common Internet File winbind service controls the behaviour of the winbindd
System) protocol. daemon. It doesn’t require the smb service to be started.
It is a client side service, which is used to connect to
The main components of Samba Windows users.
Samba is made of three daemons (smbd, nmbd and Samba also contains several important files.
winbindd). There are two services (smb and winbind) that ƒ smb.conf: This basic configuration file can be edited
control the behaviour of daemons. directly with a text editor or via a utility program like
ƒ smbd: The server message block daemon provides SWAT or Webmin.
sharing and printing services over the network. It also ƒ smbpasswd: This contains the names of users to which
controls resource locking, user authentication and data smbd will refer when it is time to enforce user-level
sharing through the SMB protocol. The Samba server access control.
listens for SMB traffic on the default TCP ports, 139 ƒ smbusers: This provides the names of Samba’s
and 445. administrators and of users with specialised Samba access.
The smb service controls the behaviour of the smbd
daemon. Samba’s features
ƒ nmbd: This provides NetBIOS names over the network. The primary features of Samba are shown in Figure 2.
It allows the browsing protocols to make up the Windows The other features are:
Network Neighborhood view. The Samba server listens ƒ Establishes shares on UNIX that are accessible to
for NMB traffic on the default UDP port 137. The smb Windows users
service also controls the behaviour of the nmbd daemon. ƒ Shares printers
ƒ winbindd: This resolves user and group information ƒ Creates a naming scheme so one can use user-friendly
on a server running Windows NT 2000 or Windows names rather than IP addresses
Server 2003. It renders Windows user/group information ƒ Makes UNIX a Windows Internet Naming Service
in a format that is understandable by UNIX platforms. (WINS) server

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 29


Admin Overview
To create a Samba users (smbgrp) group, execute the
● Provides sharing and printing servces following command:
smbd ● Controlled by smb service

groupaddsmbgrp

● Provides NetBIOS names over the network To add a user into a group, use the command
nmbd ● Controlled by smb service
given below:

● Resolves user and group information on a usermod -G smbusersmax


server running Windows NT 2000
Winbindd ● Controlled by the winbind service
Samba configuration
The smb.conf file stores all the configurations for Samba.
Figure 1: The main components of Samba It provides runtime configuration information for the
Samba programs. It determines which system resources
Authenticate
you can share with the outside world and what restrictions
Provide
Windows
Windows
Domain Loging
Assist in network
you wish to place on them. The file consists of sections
and parameters. The name of the section must be enclosed
Internet Name browsing (with
Service (WINS) or without
name server NetBIOS)
resolution

in square brackets. Each section continues until the next


Samba
Features one begins.
Act as a Backup Server directory
Each section of the file starts with a section header such
Domain Controller trees and printers
(BDC) for a
Samba-based
to Linux, UNIX,
and Windows
clients
as [global], [homes], [printers], etc.
PDC
Act as an Active
Directory
The [global] section has the variables to define sharing
domain member
server
for all resources.
The [homes] section allows remote users to access
Figure 2: Features of Samba their (and only their) home directory on the local (Linux)
machine. That is, users can access their share from
Why use Samba? Windows machines and they will have access to their
Here are a few reasons: personal home directories.
ƒ Samba easily replaces a Windows NT Samba file/print server. The [printers] section works the same as the [homes]
ƒ It offers more stability/performance compared to section, but for printers.
Windows NT. If you change this configuration file, the changes do not
ƒ It allows interoperability between UNIX and Windows apply until you restart the Samba daemon with the command
workstations. service smb restart.
ƒ It can serve as a Windows primary domain controller. To view the status of the Samba daemon, use the
ƒ Samba can serve as a highly stable WINS server. following command:
ƒ It can be used to stabilise Windows browsing services.
/sbin/service smb status
Installation and configuration
To start the daemon, use the following command:
Samba installation
To install Samba on a Linux machine, use the /sbin/service smb start
following command:
To stop the daemon, use the following command:
yum -y install samba
/sbin/service smb stop
To create a new Samba user (the new Samba user must exist
on the Linux machine as a user), use the following command: To start the smb service at boot time, use the
following command:
smbpasswd -a <username>
/sbin/chkconfig --level 345 smb on
For example, to add the user max, execute the
command given below: Access management using Samba
Table 1 gives the access control options.
smbpasswd -a max

30 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Overview Admin
Option Parameters Function Default Scope
String
Admin users Users who can perform operations as the root None Share
(list of usernames)
String
Valid users Users who can connect to a share None Share
(list of usernames)
String
Invalid users Users who will be denied access to a share None Share
(list of usernames)
String
Read list Users who have read-only access to a writable share None Share
(list of usernames)
String
Write list Users who have read/write access to a read-only share None Share
(list of usernames)
Max
Numeric Maximum number of connections for a share at a given time Zero Share
connections
Guest only (only
Boolean If yes, it allows only guest access No Share
guest)
String
Guest account UNIX account that will be used for guest access Nobody Share
(name of account)
Table 1
Table 2 gives the username options.
Option Parameters Function Default Scope
Username map String (filename) Sets the name of the username mapping file None Global
Indicates the number of capital letters to use when trying
Username level Numeric Zero Global
to match a username
Table 2

The authentication of clients is shown in Table 3.


Option Parameters Function Default Scope
Indicates the type of security that the Samba
Security Domain, server, share or user User Global
server will use
Table 3
The password configuration options are listed in Table 4.
Option Parameters Function Default Scope
Encrypt passwords Boolean If yes, enables encrypted passwords No Global
If yes, updates the standard UNIX password
UNIX password sync Boolean database when users change their encrypted No Global
password
String (chat com- Sequence of commands sent to the password See earlier section
Password chat Global
mands) program on this option
Password chat  If yes, sends debug logs of the password-change
Boolean No Global
debug process to the log files with a level of 100
String (UNIX com-
Password program Program to be used to change passwords /bin/passwd%u Global
mand)
Number of capital-letter permutations to attempt
Password level numeric None Global
when matching a client’s password
If yes, updates the encrypted password file when
Update encrypted
Boolean a client connects to a share with a plain-text No Global
password
password
Null passwords Boolean If yes, allows access for users with null passwords No Global
/usr/local/samba/
smbpasswd file String (filename) Name of the encrypted password file private/smb- Global
passwd
Name of a file that contains hosts and users that
hosts equiv String (filename) None Global
can connect without using a password
Name of a rhosts file that allows users to connect
use rhosts String (filename) None Global
without using a password
Table 4

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 31


Admin Overview
Samba server types and
security modes Active
There are three different types of servers Directory
available: Security
ƒ Domain controller Mode
• Primary domain controller (PDC) Domain Server
• Backup domain controller (BDC) Security Security
• ADS domain controller
ƒ Domain member server
Mode Mode
• Active directory domain server
• NT4 style domain server
User
ƒ Standalone server Level
There are two types of security modes Security
available for Samba: share-level and user-level,
known as security levels.
Share-level security: This can be Figure 3: User level security modes
implemented in only one way. With share-level
security, the server expects a password for each share. It is not
dependent on a particular user name. compatible authentication protocols.
In smb.conf, the security=share directive that sets share- In smb.conf, this can be achieved by using the following
level security is: directives:

[GLOBAL] [GLOBAL]
security = share ...
security = ADS
User-level security: This is the default setting for Samba. realm = test.COM
It is used by Samba, even if the security = user directive is password server = kerberos.test.com
not listed in the smb.conf file. Once the server accepts the
user name/password, the user can then access multiple shares Server security mode: This is used when Samba is not
without entering the password for each instance. capable of acting as a domain member server. In smb.conf,
In smb.conf, the security = user directive that sets user- this mode is achieved by using the following directives:
level security is:
[GLOBAL]
[GLOBAL] ...
security = user encrypt passwords = Yes
security = server
There are different implementations of user level security password server = “pwd”
(see Figure 3).
Domain security mode: In this mode, the Samba
server has a domain security trust account, which ensures
all authentication requests are handled by domain References
controllers. [1] https://www.samba.org/samba/
[2] https://docs.fedoraproject.org/en-US/Fedora/12/html/
It can be achieved by using the following directives in
Deployment_Guide/ch-samba.html#samba-rgs-overview
smb.conf: [3] http://www.fredshack.com/docs/samba.html
[4] https://access.redhat.com/documentation/en-US/Red_Hat_
[GLOBAL] Enterprise_Linux/5/html/Deployment_Guide/ch-samba.html
[5] https://help.ubuntu.com/lts/serverguide/samba-fileserver.html
...
[6] http://www.linuxtopia.org/online_books/network_
security = domain administration_guides/samba_reference_guide/index.html
workgroup = MARKETING

By: Palak Shah


Active directory security mode: In the active directory
environment, this mode is used to join as a native active The author is a senior software engineer. She loves to explore new
technologies and learn innovative concepts. She is also fond of
directory member. The Samba server can join an ADS using philosophy. She can be reached at palak311@gmail.com
Kerberos, even if a security policy restricts the use of NT-

32 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Overview Admin

Ceph: A Storage and Backup


Solution for Every Need
Backup and storage solutions assume paramount importance these days when data piles up in
terabytes and petabytes, and the loss of it can be catastrophic. Ceph is the next generation, open
source, distributed object store based low cost storage solution for petabyte-scale storage.

S
torage is a key requirement these days and has to be entity. With distributed storage, Ceph allows the distribution
thoroughly planned, keeping in mind performance, of the load, which delivers the best possible performance
future enhancements, scalability and availability, from low cost devices. All this contributes to low cost
as all these factors directly impact the functioning of the hardware, high data efficiency, broader storage use cases and
system. With the recent advances in cloud computing, the greater performance. These are some of the major benefits of
need for storage has been broadly classified into three Ceph, because of which it has been widely adopted recently
major types—object store, block store and file system by enterprises and cloud service providers. Major storage
store. Ceph is an open source, reliable and easy-to-manage, vendors like SanDisk and other enterprises like Red Hat have
next-generation distributed object store based storage contributed significantly to Ceph’s development. So let us
platform that provides different interfaces for the storage take a closer look at it.
of unstructured data from applications.
Ceph, being an object store, provides an interface for Architecture and design
all three types of data storage, i.e., object store, block store Ceph is a scalable storage cluster based on RADOS (reliable,
and file based store, hence giving a unified storage platform. autonomic distributed object store), which is a self-healing
Ceph is the best example of software-defined storage (SDS). and self-managing distributed object storage system. It has
Analogous to the principles of cloud computing, Ceph scales two types of nodes, Ceph Monitor and Ceph OSD. A Ceph
out using commodity hardware, considering the challenges Monitor node maintains the state of the cluster map to check
of scaling up. This helps us define different hardware for for node availability, and provides the cluster map to the
different workloads, and manage multiple nodes as a single cluster clients. A Ceph OSD Daemon is an intelligent peer,

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 33


Admin Overview

Client a RESTful HTTP API as the object gateway, which


is compatible with Amazon’s S3 and Openstack’s
Swift services. Ceph can be used to make a cloud
RADOSGW RBD CephFS object storage solution like Dropbox or Google
LibRADOS Drive using the Openstack cloud platform.
HDFS, GFS, Gluster-Swift, EMC Atmos and
NetApp StorageGRID Webscale are a few of the
RADOS
leading proprietary object store systems.
OSD OSD OSD OSD
Ceph as block storage
Block store is the traditional form of disk data
OSD Monitor OSD OSD
storage where the data is divided into blocks and
stored using a file system. Block store is best suited
for VM disk volume storage needs, where we
store large singular files with higher read and write
OSD OSD OSD OSD
frequencies. Ceph block storage can be well served
as a SAN (storage area network) solution.
Figure 1: Ceph architecture One added advantage of Ceph is that it
has thin-provisioning enabled, which helps in faster data
which stores the data, checks its own state and other OSDs duplication and efficient storage space utilisation. When a disk
and reports back to monitors. In case an OSD goes down, the stored in Ceph is replicated, it doesn’t occupy any memory
Ceph OSD Daemon automatically triggers data rebalancing for the replicated copy. Ceph uses the COW (copy-on-write)
based on the new cluster map from Ceph Monitor. Data is method where the block is replicated and updated only on
always transferred from the OSD nodes and never from the write, else the block is not replicated.
monitors to the clients. Amazon’s EBS (Elastic Block Storage) is a popular block
This architecture of Ceph might seem familiar to those storage service used generally for volumes of VM instances.
who know about Google File System (GFS) and Hadoop
Distributed File System (HDFS) but it is also very different Ceph as file system storage
from them in multiple ways. Ceph uses an algorithm known A file system based storage is like any NAS (network
as CRUSH (controlled, scalable, decentralised placement attached storage) system, where the file system is managed
of replicated data) for random and distributed data storage by a remote storage device. Ceph uses the Ceph FS (Ceph
among the OSDs. Ceph doesn’t need two round-trips for data file system), which provides a POSIX-compliant file system
retrieval like HDFS or GFS, in which one trip is to the central as an interface. A client system can mount this file system
lookup table to find the data location and the second trip is to and access the file storage.
the located data node. Every bit of data stored in Ceph OSDs Ceph FS uses Ceph MDS (Ceph metadata server) in the
is self-calculated using the CRUSH algorithm and stored storage cluster to store the metadata about the file system. This
independent of any other attribute. When a client requests data metadata consists of the file system tree structure, timestamps,
from Ceph, this CRUSH algorithm is used to find the exact permissions and other POSIX-compliant data. Ceph MDS can
location of all the requested blocks, and the data is transferred also be distributed by dividing the tree structure and storing it
by the responsible OSD nodes. in different Ceph MDSs. This metadata is stored like any other
As and when any OSD goes down, a new cluster map object in the Ceph OSDs, and is replicated for backup and
is generated and the duplicate data of the crashed OSD recovery purposes. FreeNAS, NAS4Free are some of the NAS
is transferred to a new node based on results from the solutions available. The Lustre file system is a famous open
CRUSH algorithm. source parallel file storage solution. Oracle ZFS Storage and
Dell Storage SCv2000 are a few of the popular NAS solutions
Ceph as object storage from proprietary vendors.
Object stores have certain advantages over traditional block
storage. Every stored object is given an object ID, which Ceph in cloud platforms
makes it faster to search/access and easier to manage. An Storage is an important component in the cloud service
object store is highly scalable for a large number of objects providers’ (CSP) list of services. Major CSPs provide
but has limitations on the size of a single object. multiple forms of cloud storage services, which include rapid
Ceph provides the LibRADOS native library, which allows storage, archival storage, object store, block store, NAS,
direct access to RADOS using apps with support for C, C++, SAN, VM disk storage, etc. All these solutions generally
Java, Python, Ruby and PHP. Ceph also features RADOSGW, require different dedicated hardware and software in order to

34 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Admin Overview
maintain QoS and performance.
Ceph as a single storage cluster allows multiple data
storage interfaces, which can utilise different hardware for
different storage needs, managed by a single SDS system.
Ceph divides the OSDs into placement groups for the
CRUSH algorithm. These placement groups can be combined
together to form a pool, which is like a logical partition for
storing the objects in Ceph. Pools can help differentiate
between the storage hardware based on performance. Ceph
also has cache-tiering, which helps in creating a pool of faster
storage devices as cache storage for expensive read/write
operations. This helps in improved performance and efficient
utilisation of the storage hardware.
With OpenStack as the cloud platform, Ceph can be used
as a Swift object store and Cinder block store utilising the same
storage hardware for multiple needs. Ceph can be used with other
cloud platforms like CloudStack, Eucalyptus and OpenNebula.

Ceph as a backup solution


We have been using RAID systems for redundancy and
data backup purposes. With Ceph as the storage platform,
data redundancy and recovery is managed using the
RADOS storage system. RADOS has features like self-
healing and self-managing, which help in recovering
the data from lost OSD nodes using instant replication
and rebalancing. Ceph also allows taking an incremental
snapshot of block storage using RBD.
OpenStack has a Ceph backup driver, which is an
intelligent solution of VM volumes. It backs up the volumes
to Ceph’s backend storage, and regularly performs an
incremental backup on the volumes to maintain consistency.
CloudBerry Backup for Ceph is a popular tool,
which helps in versatile control over Ceph’s backup and
recovery mechanism.
According to several research organisations like
Gartner Inc., Ceph has made a strategic entry into
the enterprise IT space and will prove to be the next
big evolution in storage technology. With the current
adoption rate, Ceph will soon surpass the existing storage
solutions at enterprises. There is a lot of development
expected to happen with Ceph, which will bring about
significant performance improvements to match the
current proprietary solutions. Even if you don’t take
future enhancements into consideration, Ceph is a storage
platform that definitely needs to be looked at for your
next big deployment.

By: Krishna Modi


The author has a B. Tech degree in computer engineering from
NMIMS University, Mumbai and an M. Tech in cloud computing
from VIT University, Chennai. He has rich and varied experience
at various reputed IT organisations in India. He can be reached at
krish512@hotmail.com

WWW.EFYMAG.COM
WWW.EFYMAG.COM
36 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com
Insight Admin

Manage File Storage to Give the


Best Customer Service

“There’s never enough of storage, no matter how big a disk you have,” seems to be the refrain of
all computer users. There are, of course, various solutions to tide over this problem, especially
now, with the arrival of cloud computing. This article takes the reader through the steps required
to create a file storage system and its management. It could well set you up as a STaaS (Storage
as a Service) provider in cloud computing.

W
e can’t deny the fact that the traditional model large organisations and even small businesses to centralise
of data management and hosting has changed, not only their storage but also everything they came across
and users now want their data/resources to be in the IT world. Due to the rapid increase in structured and
centralised. This is possible because of the innovation of non-structured data, there is the need for good management
virtualisation technology. It has become much easier for of storage, without a waste of resources.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 37


Admin Insight

If we talk about Storage as a Service (STaaS) in cloud not even take 1MB of your disk space but in its property, it
computing and if you are a cloud service provider, then will show 10GB as that is the allocated space. So let’s start
a lot of your resources get wasted once fixed storage the procedure.
space is allotted to a client. Most of this space is rendered A sparse file can be created by either using the truncate
useless since the client doesn’t use some of it and you are or the dd utility in Linux (other tools are also available).
not allowed to use it either. To solve this or to minimise
problems like this, it is preferable to use file data storage $truncate --size=1GB test.img
as a shared storage solution rather then object storage or a
storage attached network (SAN). The above command will create a 1GB sparse image test.img.
In this article, I will try to lead you through the To get the same result with dd, use the following command:
procedure of using and managing a file storage solution and
also qemu-img virtualisation. I will even try to take you $dd if=/dev/zero of=test.img bs=1024 count=0
through some new techniques of disk virtualisation, which seek=$[1024*1000]
can be really useful at some point for your cloud server. I
have used simple techniques, which can be easily understood Here, bs is block size and the size is provided by seek.
and are not very striking but will really help you at different You can also use the following code for simplicity:
levels of storage system development.
All the tools used are generally available on most $dd if=/dev/zero of=test.img bs=1 count=0 seek=1G
Linux systems; if not, these can be easily downloaded
via your default online repository. The tested system for If you want to allocate the whole disk space at once
these commands is RHEL 7.0, but they will work on other (which is not a good solution in our case), you can use
Linux systems in the same way, with a little modification. fallocate, as follows:

File storage: Solving the storage $fallocate -l 1G test.img


problem
If you are a STaaS provider, you will one day definitely
Note: Here’s a little trick: Can I create a 2TB partition
come across the problem of a lot of your storage being
on my 1TB hard disk? The answer is, “Yes!” Since sparse
wasted. This is because, if a user purchases 10GB of files have the property of not taking up space during creation.
storage space from you, then you are forced to allocate
him his 10GB, regardless of whether he will use all $truncate –size=2TB mylargefile.img
this space at once. Another challenge you will face
frequently is that of maintaining the right amount of Create any desired file system on it, as follows:
storage to provide the best service to your customers.
As an example, if your data storage has 100GB of space $mkfs.ext2 mylargefile.img
left, someone may unexpectedly demand 200GB of $mount mylargefile.img /mnt/
storage from you, and you may have to refuse. But a good
provider should fulfil all user requests and provide 24/7 You now have a 2TB partition.
service. So a good solution is to use file storage, with
which you can scale your storage according to the data After creating the file, you can format it and directly
inside it and allocate space that you don’t even have. mount it in order to use it as your storage, but to enable it to
File storage relies on the fact that everything in your do lots of disk operations it first has to connect to the loop
operating system is a file. In file storage, we create a file device using the losetup utility. To attach the file to your loop
and use it as our virtual partition, then format it in the device, use the following command:
desired file system and mount it. All these operations
are done in our real disk partition (/dev/sda*) with some $losetup -f test.img
small tricks or manipulation. For this procedure, we use
sparse images as our disk. …where test.img is your formatted file. To see all such
Sparse files: A sparse file is a specific type of file files or block devices connected to the loop device, use the
that aims to use the file system space more efficiently following command:
by using metadata to represent empty blocks. In sparse
files, blocks are allocated and written dynamically as $losetup -a
the actual data is written, rather than at the time the file
is created. So, if you create a 10GB sparse file, it will You can easily grep the last created loop device by using
the command below:
38 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com
Insight Admin

$losetup -a | tail -1 You will find that there is no change in the mounted space,
since the loop device has not detected the increased size.
After creating the loop device, it’s time to format it. You To make the loop device detect this, it will first have to be
can use any file system, but I recommend using the ext2 file checked by e2fsck, using the following command:
system (the reason will be explained later).
To format, use mkfs utils: $e2fsck -f test.img

$mkfs.ext2 /dev/loop0 After that, since the file system is only made on a 1GB
partition, it has to be resized. However, we don’t want data
If you are working on big data management, then the xfs to be lost or deleted and can’t handle the break in service
file system is preferred, because it works well on handling large by unmounting it for even a few seconds. So we need to do
files and supports larger inode data. Now you can mount it to online resizing by using the command given below:
use as your virtual storage, which can be shared, as follows:
$resize2fs test.img
$mount /dev/loop0 /mnt/
We have specially used the ext2 file system, since
To share it across the network using a service like nfs, online resizing and file checking only works perfectly for
open /etc/exports and use the following command: this file system with resize2fs utility. This will make our
file partition ready for use. However, all these changes
/mnt/ *(rw,rsync,no_root_squash) have not yet been detected by the loop device; so the same
operations have to be done on it.
Close it and restart the nfs service by using the command
given below: $losetup -a |grep mnt
$losetup -c /dev/loop0
$service nfsd restart

Once you put the data inside the device, you can
check its original occupied size by using the qemu-
img utility, as follows:

$qemu-img info test.img

…or by using du, as shown below:

$du -h test.img

General disk operation


Scaling up the size: If a user wants more storage, it can get
difficult for you since this type of storage doesn’t support disk
operations like resizing (especially online resizing) and you
can’t unmount or put the storage offline. Using a small trick
and with a bit of manipulation, however, you can achieve this.
Just follow the exact procedure I have outlined. Suppose a user
wants his space to be increased by 1GB (i.e., 2GB total), then
you first need to increase the size of the file by 1GB. For that
you can use the same truncate tool, since it works on already
created files and also creates the file of the desired size.

$truncate --size=2GB test.img

…or:

$qemu-img resize test.img +2GB

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 39


Admin Insight
With the -c operation, losetup will detect the increase storage image, it will get copied exactly in the backup image.
in the size but you still need to make the change in the file So a better solution can be to rsync the mounted path:
system for the loop device.
$rsync -avz /mnt/ /mnt_backup
$resize2fs /dev/loop0
To make the backup process automatic, you can use
With this, the new resized storage is ready to be used. lsyncd for live synchronisation. Install lsyncd in an RHEL
Decreasing the storage: To decrease the partition size, it system, as follows:
is not required to decrease the size of the file with the ‘truncate’
utility, since it may leave some half data chunks or bad blocks, $yum install -y lsyncd
and due to these bad blocks you will not be able to use it until
you format it again. So, the best solution can be to just decrease Edit the configuration file lsyncd.conf, as follows:
the file system layer on the file using the resize2fs command:
$cat /etc/lsyncd.conf
$resize2fs test.img 1G ----
-- User configuration file for lsyncd.
…where 1G is the new decreased size. You do not need --
to worry about the size shown in metadata since the user -- Simple example for default rsync.
will only be able to use the formatted space. Since the file is --
a sparse file, the remaining space will not take any space in settings = {
your system. In the looped device, we just need to detect the logfile = “/var/log/lsyncd.log”,
changed size, as follows: statusFile = “/var/log/lsyncd.stat”,
statusInterval = 2,
$losetup -c /dev/loop0 }
sync{
Now, your storage of decreased size is ready. default.rsync,
source=”/mnt/”,
Tip: To detach the storage from the loop device use the
target=”192.168.1.15:/backup/”,
following command:
rsync={rsh =”/usr/bin/ssh -l root -i /root/.ssh/id_rsa”,}
$ losetup -d /dev/loop0 }

Backing up your data: Features that every storage service


provider must have are backup, snapshot and clustering. Data Note: You need to first connect to the backup
snapshots seem to be the less sensible option, so let’s not machine by ssh, using ssh-keygen.
concentrate on them and discuss the topic at a later stage. But
talking of backing up data, this feature creates a backup file If you want to automate the backup process, you can use
which saves all the useful data for future use. fsmonitor npm, as follows:
So let’s take a backup, which can be successfully done by
using the rsync utility: $fsmonitor rsync -azP /mnt/ /mnt_backup

$rsync -avz test.img test_backup.img You can even use rsnapshot for incremental backup.
Snapshots of the storage data: Snapshot is a facility
This will create a test_backup.img file with the same data available in different Linux distros to save your storage
blocks. So every time you run this command, new changes made at particular stages. It can be a useful feature for STaaS
since the last backup will get saved in test_backup.img. providers, because it will help your storage to revert back to
any previous state. Snapshot is different from backup because
Warning: it doesn’t take up your storage space until changes are made
in the storage. To save space, it just copies the files that have
If you create the backup image of your file system
been deleted. For our file storage, we will be using the qemu-
through rsync, the backup file will not be a sparse file; so
it will allocate all of the space at once. img utility for a snapshot. So first create a new snapshot of
your storage, as shown below:
Although this looks simple, it is not a very efficient solution
since once data starts overwriting in some blocks in the original $qemu-img snapshot -c backup_snapshot test.img

40 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Insight Admin

-c is used for creating a new snapshot. To revert back to a # cryptsetup close my_encp.raw
particular state, use the following command:

$qemu-img snapshot -a 5 test.img Note: You can do the same disk scaling and
other operations on disk, but now you need to make
…where 5 is the snapshot ID. To see all the available changes to /dev/mapper/my_encp.raw rather then /dev/loop0.
snapshots, use the command given below:
Using a file storage virtual machine: A great benefit of
$qemu-img -l test.img file storage is its use as a base storage for virtual machines.
Once you decide to use a file storage for an OS run on a
To delete a snapshot, use the command shown below: virtual machine, other operations like scaling and encryption
can also be applied to it. To create a virtual machine instance
$qemu-img snapshot -d 2 /images/sles11sp1.qcow2 from our already created storage, let’s use the qemu-kvm
utility, as shown below:
Securing your virtual storage: The other advantage of
using file storage is that it’s easy to ship like a container. But with $qemu-kvm -name “my_os” -m 1024 -smp 2 -drive file=test.
shipping comes the responsibility of securing your storage. So a img,if=virtio,\
good solution for securing your virtual data storage is to protect index=0,media=disk,format=raw -drive file=ubuntu-14.04.
it by using encryption, for which you will need a password every iso,index=1,media=cdrom
time to mount it. For encrypting your storage, let’s use dm-crypt.
We will try encryption on fresh file storage, as follows: This will start a new virtual machine with the minimal
option selected. Here, -m defines the amount of RAM
$truncate encrypted.raw --size=2GB allocated and -smp refers to the number of cores. You can read
about more available options in the qemu-kvm man page or
Next, set up a LUKS header, as follows: by using qemu-kvm –help. To start an already created virtual
machine, use the following command:
$cryptsetup luksFormat encrypted.raw
$qemu-kvm -name “my_os” -m 1024 -smp 2 -drive\ file=/images/
sles11/hda,if=virtio,index=0,media=disk,format=raw
Warning:
Don’t try this with an already formatted partition because File storage can be a good solution for your enterprise
it will delete all the previous data inside the partition. environment or personal use, but what matters is how you make
it useful at a different level of your cloud solution. Storage
This will prompt you to enter a fresh password. To gain management in the case of file storage doesn’t end here since
access to the device, use the command given below: there is a lot more you can do with it. It can be a powerful as
well as flexible solution for storage management, but requires
$cryptsetup open encrypted.raw my_encp.raw thorough knowledge to get the job done.

my_encp.raw is the name of the file where our partition is


mapped in /dev/mapper/. Now you can create a file system on
References
top of it, as follows:
[1] For lsyncd: http://www.linuxtechi.com/install-and-use-
lsyncd-on-centos-7-rhel-7/
$mkfs.fstype /dev/mapper/my_encp.raw [2] For qemu utility: https://www.suse.com/documentation/
sles11/book_kvm/data/book_kvm.html
Mount the newly created partition anywhere with the [3] For Dm-crypt encryption: https://wiki.archlinux.org/index.
php/Dm-crypt/Encrypting_a_non-root_file_system
following command:

$mount -t ext2 /dev/mapper/my_encp.raw /mnt/ By: Shubham Dubey


The author is a B. Tech student at the LNM Institute of Information
Once use of the storage is finished, you can unmount Technology, Jaipur. Besides being a Linux enthusiast, he works in
it, as follows: the fields of cloud computing, virtualisation and cyber security. You
can contact him at sdubey504@gmail.com or at his LinkedIn page
https://in.linkedin.com/in/shubham0d
# umount /mnt/

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 41


Admin How To

Bare Metal System Backup and


Recovery Using Open Source Tools
Disaster often strikes when least expected. When it’s a question of safeguarding data against disk crash
or corruption, it’s best to have a backup and recovery system. Bare metal system recovery is particularly
advantageous in that there is no need to even load the operating system to effect data recovery.

B
ackup and recovery of heterogeneous data files ƒ Configurability with scope for extension using plugins
is one of the key aspects of personal as well as ƒ Command line as well as GUI panels for usability
corporate computing. In our routine tasks, we create ƒ Transaction fault tolerance to avoid data loss
and update a number of ƒ Being volume oriented
files in our systems. But to support compression,
the major problem occurs splitting and merging
when the operating system for multiple devices and
crashes or any media stalls. platforms
This issue generates huge ƒ Malware scanning
complications and, many ƒ Universal view and
a time, our work stops. updates
Often, we have to create the ƒ Cross platform
files using backup, recovery ƒ Support for multiple
and disaster management data formats
tools and techniques. ƒ Support for multiple
A number of tools are databases
available in the open source ƒ Generation of reports,
as well as the proprietary alerts and logs
domain, which provide
secured backup, fast Free and proprietary
recovery and fault tolerance. software backup
The key challenge in tools
disaster management arises A number of software
when the operating system crashes and when we are not able tools are being used in personal as well as commercial
to load the OS to execute the recovery tool. In such scenarios, domains for backup of files and documents. Table 1 shows
we can work with bare metal backup and recovery tools so Linux based free software with command line interfaces.
that there is no need to even load the operating system. Using
these tools, the backup, recovery and disaster management Program/Tool Base language
can be achieved at the BIOS level itself. AMANDA C and Perl
Areca Backup Java
Features of effective backup and duplicity Python
fault-tolerant tools rdiff-backup Python
Some of the major features generally required for personal as Attic Python
well as corporate computing are: Bacula C++
ƒ Bare metal recovery and fault tolerance Back In Time Python
ƒ Non-redundant or de-duplicated backup DAR C++
ƒ Security and privacy using multi-level encryption luckyBackup C++
ƒ Dynamic compression and archiving Mondo Rescue C and UNIX Shell
ƒ Dynamic repository management obnam Python
ƒ Automatic/scheduled backups Syncthing Go
ƒ Cloud based backup and recovery Unison OCaml
ƒ Import and export for multiple platforms
ƒ Version tracking Table 1

42 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


How To Admin

Table 2 features proprietary backup packages with a


GUI (graphical user interface).
@MAX SyncUp Argentum Acronis True
Backup Image
ARCserve Backup Backup4all BackupAssist
Backup Exec Catalogic DPX Bitser
Bvckup 2 ChronoSync CloudJunxion
Comodo Backup Crashplan Dmailer Backup
Dolly Drive Double Image Druva InSync
Backup
Druva Phoenix EMC Net- Genie Backup
Worker Manager
Handy Backup HP Data Pro- IASO Backup
tector
IBM Tivoli Storage InMage DR- Image for Win-
Manager Scout dows
Iperius Backup KeepVault Langmeier
Backup
LazySave .Mac Backup Macrium Reflect
aka MobileMe
Memopal Mozy NetVault Backup
Novabackup NTBackup NetBackup
Norton 360 Norton Ghost EMC Recover-
Point
Retrospect ShadowPro- Simpana
tect
System Center SpiderOak SyncBack
Data Protection
Manager
SyncToy Time Machine Tonido Backup
TotalRecovery Pro UltraBac Ventis Backup-
Suite 2008
Windows Home Windows Yosemite Server
Server Computer Backup and Backup
Backup Restore
Table 2

Free and open source tools for


bare metal recovery
Redo Backup and Recovery
URL - http://redobackup.org
Redo Backup and Recovery is very user friendly and an
effective tool for overall backup and disaster management
with bare metal data restoration. With this software, disk
based disaster management and restoration at the bare
metal level is very effective in case of hardware failure. It is
also used as an anti-virus. The complete system backup can
be taken even if the hard drive gets burned, or damaged by a
virus or due to malicious applications.
Features of Redo Backup and Recovery include:
ƒ No need for installation
ƒ Very small download size of 250MB for live CD
ƒ Boots in a few seconds
ƒ Access to files and folders even without logging in

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 43


Admin How To
ƒ Multi-casting, which
involves the imaging
Backup of multiple systems in
Step 3: Select Destination Drive parallel
Where is the destination drive?
ƒ Cross platform
Connected directly to my computer
Shared over a network
(Linux, Windows, Mac
and multi-boot setup)
Selected network-shared storage destination:
Shared Folder specified below
ƒ Superior scalability,
Server or share location:
which does not degrade
Username (Optional): the performance even
Password (Optional):
Figure 6: FOG installer for the imaging of
Cancel Next > 50,000 computers
WELCOME TO HONDO RESCUE

Figure 4: Network based storage in Redo Backup and Recovery Backup to:
Mondo Rescue
CD-R disks CD-RW disks
URL - http://www.
ƒ Configuration of disk drives can be done graphically with DVD Disk USB Key/Disk mondorescue.org/
the number of inbuilt tools Tape drive NFS mount
Mondo Rescue
Besides the enormous features for backup, the tool is a reliable and
includes a number of additional and powerful programs, widely used disaster
Hard disk Exit

featured in Table 3. Pleasechoose the backup media to which you want to archive data.
recovery software.
Chromium fsarchiver It is used for the
baobab Chntpw Figure 7: Different options and backing up of GNU/
cryptsetup drivereset features of Mondo Rescue Linux based systems
hdparm lshw-gtk to different types
geany gnome-disk-utility of storage media. Mondo is used by Lockheed-Martin,
gparted Smartctl Siemens, Nortel Networks, IBM, HP, NASA JPL, the US
photorec Rsync Department of Agriculture and many other organisations
scp Grsync worldwide. Mondo supports RAID, LVM 1/2, ext2, ext3,
partclone Ssh ext4, XFS, JFS, ReiserFS, VFAT and other assorted file
partimage Testdisk systems without any issues. It supports software as well
as hardware RAID. This tool is packaged with many
Table 3
distributions and flavours including Fedora, SLES,
FOG project RHEL, openSUSE, Ubuntu, Gentoo Mandriva, Mageia
URL - https://fogproject.org and Debian.
FOG is another very powerful system imaging tool that is
free, open source and based on Linux. FOG can be used with Rear (Relax-and-Recover)
PXE and TFTP without any boot disk or CDs. With it, the URL - http://relax-and-recover.org
system is booted using PXE and automatically a small Linux Relax-and-Recover or Rear is another bare metal
client is downloaded, after which many operations can be disaster recovery tool that is very easy to install and set up
performed on the system. without any maintenance overheads.
The features of FOG include: The features of Rear include:
ƒ It is free and open source ƒ Modular design, written in Bash
• Easy to extend with custom functionality
• Targeted primarily at sysadmins
ƒ Offers the ‘Set up and forget’ style of operation
• Easy to install and set up
• No maintenance issues
ƒ Recovers the images based on original distribution using
original tools
ƒ Two-step fault tolerance and recovery
ƒ Bare metal recovery on different types of hardware
ƒ Support for
• Physical-to-virtual (P2V)
Figure 5: The FOG project’s official portal • Virtual-to-physical (V2P)

44 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


How To Admin

Working with the Rear tool: You can clone Relax-and-


Documentation Downloads Support Development Events Recover from GitHub as follows:
Relax-and-Recover is a setup-and-forget Linux bare metal disater recovery solution. It is easy to set up
and require no maintenance so there is no excuse for not using it.
$ git clone https://github.com/rear/rear.git
Learn more about Relax-and-Recover form the slected usage scenarios below:
Home user Enterprise user $ cd rear/

You can also prepare your own USB media. Relax-and-


Recover will ‘own’ the device in the following example.
• recover from a broken hard disk using a bootable USB stick • collact small ISO images on a central server
• recover a broken system form your bootlader •

integrate with your backup solution
recover a broken system form your bootlader
This will destroy all data on that device.
$ sudo usr/sbin/rear format /dev/sdb
Figure 8: The Relax-and-Recover official page
Relax-and-recover confirm to format the device -
• Physical-to-physical (P2P) Yes / No
• Virtual-to-virtual (V2V)
• Assorted virtualisation technologies The device has been labeled REAR-XXXXX by the
ƒ Support for assorted boot media “format” workflow.
• USB Edit “etc/rear/local.conf” configuration file:
• PXE $ cat > etc/rear/local.conf <<EOF
• eSATA OUTPUT=USB
• ISO
• OBDR/bootable tape BACKUP=NETFS
ƒ Support for different transport protocols
• HTTPS BACKUP_URL=usb:///dev/disk/by-label/REAR-000
• HTTP EOF
• SFTP
• NFS Now, create a rescue image.
• FTP
• CIFS (SMB) $ sudo usr/sbin/rear -v mkrescue
ƒ Extensive and exhaustive implementation for disk layout
• SWRAID Using log file: /home/myuser/tmp/quickstart/rear/
• LVM var/log/rear/rear-fireflash.log
• HWRAID Creating disk layout
• multipathing Creating root filesystem layout
• iSCSI WARNING: To login as root via ssh you need to
• DRBD setup an authorized_keys file in /root/.ssh
• LUKS (encrypted partitions and file systems) Copying files and directories
ƒ Support for third party technologies for backup Copying binaries and libraries
• EMC NetWorker (Legato) Copying kernel modules
• HP DataProtector Creating initramfs
• CommVault Galaxy Writing MBR to /dev/sdb
• IBM Tivoli Storage Manager (TSM) Copying resulting files to usb location
• Symantec NetBackup
• Bacula Reboot the system to boot from the USB device.
• SEP Sesam Initiate the backup as follows:
• Bareos
• Duplicity/Duply $ sudo usr/sbin/rear -v mkbackup
ƒ Assorted methods for internal backup
• rsync By: Dr Gaurav Kumar
• tar The author is the MD of Magma Research and Consultancy Pvt Ltd,
ƒ Multi-phase disk layout recovery Ambala. He is associated with a number of academic institutes,
ƒ Effective and easy monitoring where he delivers lectures and conducts technical workshops
on the latest technologies and tools. You can contact him at
ƒ Integration with high performance scheduler
kumargaurav.in@gmail.com or www.gauravkumarindia.com.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 45


Admin How To

Creating a Storage System using

A storage system for data is one of the things that most computer users need. Solutions
vary from the DIY kind to expensive cloud storage. This article points the reader to
Rockstor, a DIY network attached storage (NAS) system.

C
ommon complaints from prosumers and small Hardware requirements for building a
business owners with respect to storage systems are: storage system
a server or hard disk tends to crash when it matters When it comes to building your Linux based storage system,
most; one often runs out of storage space at the wrong time; there are three approaches you could take:
and the challenges in selecting the best storage solution from 1. Assemble a NAS from components
myriad options. Eventually, many users end up choosing an 2. Build on top of pre-built servers or old hardware, or even
inappropriate storage solution for their needs. a hypervisor
This article guides you on how to select an appropriate 3. Buy a pre-built Linux NAS box from a Linux NAS distro
storage solution. It also describes an easy way to build a provider
complete storage system using inexpensive hardware and Assembling a DIY NAS from components: This
Rockstor, a CentOS based Linux distro. approach is recommended for prosumers who are tinkerers
By the end of this article: at heart and like to experiment, build and break stuff. The
1. Prosumers will learn how to build a storage system advantage of this approach is its low cost, a high degree of
using Rockstor. control and the thrill of building a NAS. The disadvantages
2. Small and medium business users will be able to of this approach are that it’s time consuming and needs some
determine whether a Linux based DIY NAS or a dedicated degree of technical expertise.
NAS server (Rockstor Pro 8) is appropriate for their needs. If you decide to take this approach, the following are

46 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


How To Admin

Figure 3: Rockstor dashboard

Figure 1: Rockstor installation

Figure 4: Create storage-available hard disks

A note on minimum system requirements: Each


of the Linux NAS distros will have specific system
requirements depending on the underlying file system.

With Rockstor, you’ll need the following:


ƒ 64-bit Intel or AMD processor
ƒ 2GB RAM or more (recommended)
ƒ 8GB hard disk space for the OS
ƒ One or more additional hard drives for data
Figure 2: Rockstor installation—select date and timezone, and installation destination
(recommended)
ƒ Ethernet interface (with Internet access, for updates)
some of the components you’ll need: ƒ A UPS system (if desired) that is supported by Network
1. Case (e.g., SilverStone DS380B) UPS Tools
2. Power supply unit (e.g., SS-350FE) ƒ DVD drive and a blank DVD, or a USB port and
3. Motherboard and CPU combo (e.g., ASrock MB minimum 1GB USB key (for the installation media)
C2550D4I and the Intel Avoton C2550 or C2750 quad- Buying a complete NAS solution from an open source
core processor) Linux NAS distro provider: This solution is recommended
4. RAM (e.g., Kingston 8GB 1600MHz DDR3LDIMM) for SMBs or even prosumers who would like to save time
5. Boot drive (a USB or a PCIemSATA) and don’t mind paying the extra cost. The advantage of this
6. HDDs (e.g., Western Digital hot pluggable — 8.89cm solution is its plug-and-play nature. Rockstor Pro 8 DIY NAD
HDDs or 6.35cm HDDs) Build is one such solution with BTRFS at its core.
Building NAS on pre-built servers: This approach is This solution is superior to other Linux based solutions
recommended for prosumers or SMBs. The advantage of available in the market as users have access to advanced
this approach is its cost-effectiveness and flexibility in the features at a fraction of the cost. Some of these features are:
choice of hardware (as long as it meets the minimum system ƒ Simple and secure browser based management
requirements) or hypervisors. ƒ Supports different sized HDDs and has online capacity
You can use any of the following four options to build scaling. Adds more drives as and when you need them
your storage system: ƒ Bitrot protection, checksums, compression and other
1. Old hardware such as an old laptop or an old server advanced file system (BTRFS) features
2. Appliances such as Intel NUC, Asus Vivo PC, HP ƒ Efficient Copy-on-Write (CoW) snapshots of shares (on
Proliant, etc demand or on schedule)
3. Hypervisors such as KVM, Proxmox or VirtualBox ƒ File sharing and access from Linux, Mac, Windows and
4. A combination of virtualisation and appliances mobile devices (Android and iOS)

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 47


Admin How To

Create Pool Create Share

Name vinima_photos Name Photos-2013

Raid configuration Single Pool vinima_photos

Compression Dont enable compression Compression Inherit from pool wide configuration

Mount options Size 102.1 GB 932.0 GB

0.0 GB 500.0 GB
Selected disks summary 1 x 931.50 GB 931.50 GB

Total Raw Capacity 931.50 GB

Total Usable Capacity 931.50 GB 102.1 GI

(931.5 GB) Space that is completely free and unprovisioned

Select disks No. Disk Name Capacity Select all (0.0 GB) Space that is provisioned for other shares, but currently free

1 sdb 232.90 GB
(0.0 GB) Space that is provisioned for other shares and is in use
2 sdc 931.50 GB
Cancel Submit

Cancel Submit

Figure 6: Create storage-create, a Rockstor share


Figure 5: Create storage-create, a Rockstor pool
of BTRFS over other file systems is that pools can be created
ƒ Must have prosumer apps for media streaming (Plex), by combining disks of different sizes, but these disks have to be
backup (ownCloud) and file synchronisation (Syncthing) whole (not partitioned). To create a pool using the Web UI, go to
ƒ Apps for developers and small businesses such as Storage->Pools. Click on ‘Create Pools’ and specify the ‘Name’,
JenkinsCI, Gitlab, Discourse, etc ‘RAID Configuration’ and ‘Select Disks’ that should be the part
ƒ Privacy and productivity enhancing apps for everybody of this pool. Click ‘Submit’ to create the pool. Advanced users
ƒ Efficient Rockstor ->Rockstor replication for backup and can also apply different compression and BTRFS mount options
data recovery while creating a pool (or afterwards). You can see the pool
‘vinima_photos’ in Figure 5.
Building a NAS storage system using bare metal Share: A share is a space carved out of a pool and provides
hardware and Rockstor 3.8-12 storage for user generated data. It behaves similar to a directory
I am going to focus on building a NAS on a pre-built server in a file system and can be exported using protocols like Samba/
(option 2 mentioned earlier) and use a HP Proliant server with CIFS, AFP, SFTP, NFS, etc, from a Rockstor machine. A share can
4x1TB HDDs. I’ve used a USB as a boot drive to install Rockstor be resized, cloned and deleted. To create a share using the Rockstor
3.8-12, but you can also use a PCI-Express mSATA III boot drive. Web UI, go to Storage->Shares and click ‘Create Share’. Specify
You need to prepare a bootable USB with the Rockstor3.8-12.iso. the ‘Name’, ‘Pool’ and ‘Size’, and then click ‘Submit’. Similar to a
Instructions on Rockstor documentation are available at http:// pool, advanced users can apply compression options. You can see
rockstor.com/docs/quickstart.html#minsysreqs. Then proceed to the ‘Photos-2013’ share in Figure 6.
install Rockstor in the HP Proliant server. The instructions are Advanced features: Rockstor leverages other Linux
given at http://rockstor.com/docs/quickstart.html#installation. technologies, such as Docker, to go beyond the NAS paradigm.
Once Rockstor 3.8-12 is installed, you’ve set up a basic Rockstor provides an app, also called the Rock-ons hosting
storage system, and are now ready to add or remove disks, framework. Using Rock-ons, media content can be streamed
create a pool (BTRFS volume), create shares (BTRFS sub- from Rockstor, files from various smart devices can be
volume) and use advanced BTRFS features. I will briefly synchronised or backed up to Rockstor, etc. Almost any
describe these features. containerised application can be deployed to take advantage of
Disks: Rockstor supports whole disks, not partitions. My the solid storage platform provided by Rockstor. As discussed
HP Proliant box has 4x1TB HDDs that you can see on the earlier, Rock-ons are Docker-based applications that help extend
Rockstor Web UI, under the ‘Storage’ tab. functionality and run applications like syncing and streaming.
I can add additional space by connecting USB and external The installation procedure for all Rock-ons is similar.
hard drives to my box. I cannot add more HDDs as there are I hope that most readers now have a basic understanding
no empty bays, but if your case has empty bays, you could add on how to choose or build their own storage system. Please
more disks. Also, note that disk sda3 has the system-created refer to Rockstor documentation to understand a range
rockstor_rockstor pool containing the OS. It’s recommended that of applications that can be run on your storage system
this pool is not touched except for small shares, and even that, (https://rockstor.com/docs).
only if absolutely necessary. Resizing, compression or deletion of
By: Vinima Aggarwal
the rockstor_rockstor pool is not permitted.
Pool: A pool is space carved out of a disk or disks. You can The author is a passionate marketer, and loves technology and
use multiple disks to create a single pool. Pools can be resized marketing. She is currently head of marketing at Rockstor and is based
in Silicon Valley, California. You can reach her at vinima@rockstor.com.
in capacity by adding or removing disks. One of the advantages

48 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Insight Developers

JavaScript: The New Parts


This eighth article in our series on JavaScript takes us to a feature known as Promises. This
feature can be used in place of callbacks and to avoid its side effects. Promises are called
futures or deferreds in other languages.

B
efore we get started, a word of caution. instructions written from top to bottom.
Understanding Promises could be as difficult as Another drawback of the callback function is called
making sense of the Christopher Nolan movie Pyramid of Doom. As we go on nesting multiple callbacks,
‘Inception’. This article is an attempt to simplify the it becomes difficult to debug. Some nested callbacks
topic as much as possible. The attempt is to go might result in an exception, and tracing
beyond syntax and show examples on why exceptions is a nightmare.
Promises are indeed needed.
Promises are required if your Promise
application performs operations such Promise is a place holder for a future
as database access, file access or value. This is because we want to
fetching user data from the online store the results of an asynchronous
APIs of Twitter. These operations operation. Promise is an object. The
could take variable amounts of time. syntax below is used to create a
Waiting for these calls to complete Promise.
could stall the application for a few The syntax is:
seconds or freeze the UI. The result
of the operation could even be an new Promise (function)
error, after a long wait and another retry
attempt. Hence, it is better to make these calls Given below is an example of how to create
asynchronous (not wait for the results). a function that returns a Promise. We have converted
an asynchronous fs.readFile() operation into a Promise.
JavaScript callbacks
JavaScript began as a browser language, with the primary task 1. function readFile(filename) {
of handling user events such as a mouse click. The language 2. let p = new Promise(function(resolve, reject) {
has excellent support for event-driven programming. In this 3. fs.readFile(filename, function(err, contents) {
language, functions are first class citizens, which means you 4. if (err) {
can use a function where any other primitive data type fits. For 5. reject(err); // error case
example, a function can be passed as an argument. A function 6. }
can also return a function. A callback is a function that 7. resolve(contents);
will get executed upon an event. It is common for frontend 8. });
developers to write many callback functions on mouse events 9. });
like onClick, onmousedown, etc. The functions (callback) get 10. return p;
called upon mouse events. 11. }
Another important point to note is that JavaScript is
single-threaded. Any operation (function call), which takes Line 2: Creates a new Promise object. The object ‘p’ gets
time, should be asynchronous. a value after reading the file.
Let’s take an example of a callback. Line 3: A Promise takes a function. The function in turn
takes two functions ‘resolve’ and ‘reject’ as parameters. For
1. var fs = require(‘fs’); all successful cases, it executes ‘resolve’. For error scenarios,
2. fs.readFile(‘readme.txt’, printfile); the second parameter function is executed.
3. function printfile(err, filecontents) {
4. console.log(filecontents.toString()); Promise states
5. } Promise has states. Initially, when a Promise is created,
it is in the pending state. This is the initial state. Once the
printfile() is a callback function, which would be called asynchronous operation completes, the Promise moves to the
after a read operation. One drawback of a callback function resolved state. There are two possible states within ‘resolve’.
is that the flow of execution is different from the sequence of These are: ‘fulfilled’ and ‘rejected’. A successful result of a

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 49


Developers Insight
resolved Promise is fulfilled. An error case of a Promise is
resolved
the rejected state.
Using the readFile() function, we can see how the
settled
Promise code looks.
fulfill(...)
1. #!/usr/local/bin/node
2. “use strict”; resolve(...) fulfilled
3. console.log(‘first line’);
4. let fs = require(“fs”);
5. function readFile(filename) { pending
rejecte(...)
6. let p = new Promise(function(resolve, reject) {
7. fs.readFile(filename, function(err, contents) { rejected
8. if (err) {
9. reject(err); // error case Figure 1: Promise states diagram
10. }
11. resolve(contents); promise.then(fulfill).then(process).catch(reject);
12. });
13. }); Promise all() and Promise.race()
14. return p; There are two more methods that Promise provides. These
15. } two are used when you have to iterate through multiple
16. let promise = readFile(“readme.txt”); Promises (async operations). Instances of it could be useful
17. when you have to gather content from multiple sources/APIs.
18. promise.then(fulfill, reject);
19. Promise.all([p1, p2, p3]).then(function(value) {
20. function fulfill (contents) { console.log(value);
21. console.log(contents.toString()); }, function(reason) {
22. } console.log(reason)
23. });
24. function reject (err) { Promise.all() takes multiple Promises as an array. The
25. console.log(err.message) first function of .then is executed when all Promises (p1, p2
26. } and p3) are fulfilled. Even if one fails, the second function is
27. console.log(‘last line’); called for error handling.
Promise.race() is similar, except that the parent Promise
The output of the above program is shown below: will be fulfilled even if only one of the three Promises are
fulfilled. This could be used when fetching dictionary data
first line from multiple sources. The result from whichever source is
last line fetched faster, and is good to display. This method can also be
README file contents used to timeout after predefined milliseconds.

Line 16: This invoked the function readFile(), which Promises with arrow functions
returns a Promise. In previous articles in this series, we covered another feature
Line 18: After the file contents are read, the Promise of ES6 called arrow functions. They come handy when used
comes to the resolved state. For a successful file read, fullfill() in combination with Promises.
gets executed. For an error scenario like the file not existing, The earlier program, from Lines 8 to 16, can be modified
‘reject’ gets executed. to use anonymous functions, as follows:
One of the advantages of Promise is chaining, which is
done by appending multiple .then() functions. You can also promise.then(function (contents) {
have one error handling function using .catch(). Line 18 can console.log(contents.toString());
be changed to what’s shown below: }, function (err) {
console.log(err.message)
promise.then(fulfill).then(process).then(processFurther); });
This can further be enhanced to:
…or, what follows: promise.then((contents) => {

50 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Insight Developers

console.log(contents.toString()); browsers. In non-browser environments, it is supported in the


}, (err) => { latest versions of Node and Babel. For detailed support, check
console.log(err.message) the Kangax link in References.
}); Many popular JavaScript libraries and frameworks have
adopted Promise due to its merit. Some popular ones are
Useful fetch module jQuery, Ember.js, etc. Understanding Promise is essential for
the async feature, which will come with future versions of
#!/usr/local/bin/node ECMAScript.
‘use strict’;
References
[1] An excellent detailed article on Promise with code examples:
var fetch = require(‘node-fetch’);
http://www.codeproject.com/Articles/1079322/Learn-How-
fetch(‘http://api.icndb.com/jokes/random’).then(function to-Make-ES-Promises-with-Executable-Exam
(res) { [2] An introduction to JavaScript Promise http://dev.
return res.json(); paperlesspost.com/introduction-javascript-promises/205
[3] Promise in JavaScript https://www.youtube.com/
}).then(function (json) {
watch?v=oa2clhsYIDY
console.log(json.value.joke); [4] A detailed chapter on Promise in ExploringJS book
}); http://exploringjs.com/es6/ch_promises.html
[5] Kangax support matrix http://kangax.github.io/
compat-table/es6/
node-fetch is a lightweight module to get the contents of
a file or URL. It returns a Promise. The above code is a small By: Janardan Revuru
node.js program that fetches a joke from the Internet database
The author believes that ‘JavaScript as first language’ for
and prints it on the console. new programmers and encourages developers to learn pure
JavaScript. He is a co-organiser of the ‘JavaScript meetup –
Support matrix Bangalore’ group at meetup.com. http://www.meetup.com/
JavaScript-Meetup-Bangalore/
Promise is supported in the Chrome, Firefox and Edge

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 51


Developers Let’s Try

GNU Emacs: How to Work with HTML


Mode, Indentation and Magit

In this latest article in our


well-documented series on

<html>
GNU Emacs, the reader is
taken on a journey through
HTML mode, indentation in
HTML code and Magit —
the magic interface to Git,
inside Emacs.

T
his article in the GNU Emacs series takes readers on <a href="mailto:user@hostname">shakthi</a>
how to use HTML mode, do indentation, and use the </address>
Magit interface. </body>
</html>
HTML mode
You can use HTML mode to effectively edit HTML and CSS Tags
files using GNU Emacs. To start the mode, use M-x html- You can enter HTML tags using C-c C-t. GNU Emacs will
mode. You will see the string ‘HTML’ in the mode line. prompt you with the available list of tags. A screenshot of the
available tags is shown in Figure 1.
Default template The anchor tag can be inserted using ‘a’. You will then
A default HTML template can be started by opening a receive a message prompt: ‘Attribute:’. You can provide the
test.html file, and using C-c C-t html. It will produce the value as ‘href’. It will then prompt you for a value, and you can
following content: enter a URL, say, http://www.shakthimaan.com. The anchor
tag will be constructed in the buffer as you input values in the
<html> mini-buffer. You will be prompted for more attributes. If you
<head> want to finish, simply hit the Enter key and the anchor tag will
<title> be completed. The final output is shown below:

You will then be prompted with the string ‘Title:’ to input <a href=”http://www.shakthimaan.com”></a>
the title of the HTML page. After you type ‘Hello World’, the
default template is written to the buffer, as follows: You can insert a h2 tag by specifying the same after C-c
C-t. You can also add attributes, as required. Otherwise,
<html> simply hitting the Enter key will complete the tag. The
<head> rendered text is as follows:
<title>Hello World</title>
</head> <h2></h2>
<body>
<h1>Hello World</h1> You can insert images using the alt tag. You can specify
<address> the src attribute and a value for the same. It is also a good

52 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try Developers

practice to specify the alt attribute for the image tag. An Accents
example is shown below: In HTML mode, you can insert special characters, accents,
symbols and punctuation marks. These characters are mapped to
<img alt=”image” src=”http://shakthimaan.com/images/ShakthiK- Emacs shortcuts. Some of them are listed in the following table:
workshop-A4-poster.png”>
Shortcut Character
Unordered lists can be created using C-c C-t followed by
C-x 8 ‘ a á
ul. It will then prompt you for any attributes that you want
included in the tag. You can hit the Enter key, which will C-x 8 “ e ë
prompt you with the string ‘List item:’ to key in list values. An C-x 8 / E Æ
example of the output is shown below:
C-x 8 3/4 ¾

<ul> C-x 8 C ©
<li>One C-x 8 L £
<li>Two C-x 8 P ¶
<li>Three
C-x 8 u µ
</ul>
C-x 8 R ®
You can neatly align the code by highlighting the above C-x / / ÷
text and indenting the region using C-M-\. The resultant
output is shown below: Indentation
Consider the following paragraph:
<ul> “When we speak of free software, we are referring to
<li>One freedom, not price. Our General Public Licenses are designed
<li>Two to make sure that you have the freedom to distribute copies
<li>Three of free software (and charge for this service if you wish),
</ul> that you receive source code or can get it if you want it, that
you can change the software or use pieces of it in new free
If you wish to comment out text, you can select the region programs; and that you know you can do these things.”
and type M-q. The text is enclosed using ‘<!--’ and ‘-->’. For You can neatly fit the above text into 80 columns and 25
example, the commented address tags in the above example rows inside GNU Emacs using M-q. The result is shown below:
look like what follows:
When we speak of free software, we are referring
<!-- <address> --> to freedom, not
<!-- <a href="mailto:shakthi@achilles">shakthi</a> --> price. Our General Public Licenses are designed to
<!-- </address> --> make sure that you
have the freedom to distribute copies of free software
A number of major modes exist for different programming (and charge for
environments. You are encouraged to try them out and this service if you wish), that you receive source code or
customise them to your needs. can get it

Figure 1: HTML tags

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 53


Developers Let’s Try
if you want it, that you can change the software or use are many ways in which you can install Magit. To install it
pieces of it from the Melpa repository, add the following to your ~/.emacs:
in new free programs; and that you know you can do these
things. (require ‘package)
(add-to-list ‘package-archives
You can also neatly indent regions using the C-M-\ ‘(“melpa” . “http://melpa.org/packages/”) t)
shortcut. For example, look at the following HTML snippet:
When you do M-x list-packages, you will see ‘magit’
<table> in the list. You can press ‘i’ to mark Magit for installation,
<tr> followed by ‘x’ to actually install it. This will install Magit
<td>Tamil Nadu</td> in ~/.emacs.d/elpa. The version installed on my system is
<td>Chennai</td> magit-20160303.502.
</tr> When you open any file inside GNU Emacs that is version
<tr> controlled using Git, you can start the Magit interface using
<td>Karnataka</td> M-x magit-status. I have bound this key to the C-x g shortcut
<td>Bengaluru</td> in ~/.emacs using the following:
</tr>
<tr> (global-set-key (kbd “C-x g”) ‘magit-status)
<td>Punjab</td>
<td>Chandigarh</td> The default Magit screenshot for the GNU Emacs project
</tr> README file is shown in Figure 2.
</table> Pressing ‘l’ followed by ‘l’ will produce the history log in

After indenting the region with C-M-\, the resultant


output is shown below:

<table>
<tr>
<td>Tamil Nadu</td>
<td>Chennai</td>
</tr>
<tr>
<td>Karnataka</td>
<td>Bengaluru</td>
</tr>
<tr>
<td>Punjab</td>
<td>Chandigarh</td>
</tr>
</table>
Figure 2: Magit
If you have a long line which you would like to split, you
can use the C-M-o shortcut. Consider the quote:
“When you’re running a startup, your competitors decide
how hard you work.” Paul Graham
If you keep the cursor after the comma, and use C-M-o,
the result is shown below:

“When you’re running a startup,


your competitors decide how hard you work.” Paul
Graham

Magit
Magit is a fantastic interface to Git inside GNU Emacs. There Figure 3: History

54 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try Developers

Figure 5: Branch
Figure 4: Staged changes
can enter the commit message.
the Magit buffer. A screenshot is provided in Figure 3. You can create and check out branches using the ‘b’
You can make changes to the project sources and stage shortcut. A screenshot of the Magit branch pop-up menu is
them to the index using the ‘s’ shortcut. You can unstage the shown in Figure 5.
changes using the ‘u’ shortcut. After making changes to a All the basic Git commands are supported in Magit -
file, you need to use M-x magit-status to update the Magit diffing, tagging, resetting, stashing, push-pull, merging and
buffer status. rebasing. You can read the Magit manual (http://magit.vc/ ) to
A sample screenshot of the modified files and staged learn more.
changes is shown in Figure 4.
You can hit TAB and Shift-TAB to cycle through the By: Shakthi Kannan
different sections in the Magit buffer. To commit a message, The author is a Free Software enthusiast and blogs at shakthimaan.com.
press ‘c’ followed by ‘c’. It will pop up a buffer where you

OSFY Magazine Attractions During 2016-17


MONTH THEME
March 2016 Open source Databases

April 2016 Backup and Data Storage


May 2016 Web Development
June 2016 Open Source Firewall and Network security
July 2016 Mobile App Development
August 2016 Network Monitoring
September 2016 Open Source Programming Languages
October 2016 Cloud Special
November 2016 Open Source on Windows
December 2016 Machine Learning
January 2017 Virtualisation (containers)
February 2017 Top 10 of Everything

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 55


Developers How To

Get Started with Developing


MS Office Add-ins

You can spruce up Microsoft Office, and add custom commands and new features to Office
programs that help increase your productivity by using add-ins. Further, you can develop your
own custom add-ins to give your documents style and class. In this tutorial, you can learn how to
build a word cloud generator add-in for Microsoft Word.

A
dd-ins have been an important part of the mails, calendars, meeting requests and appointments.
Microsoft Office suite as they allow users and third Office currently has three types of add-ins that
party developers to extend the capabilities and you can build:
functionalities of Office, and with more than 1.2 billion ƒ Task pane add-in: This appears at the side of the
users worldwide (https://news.microsoft.com/bythenumbers/ document and allows you to provide contextual and
ms_numbers.pdf), there is quite a large market for apps on this functional features in it.
platform. ƒ Content add-ins: These appear in the body of the
Previously, one had to use things like VBA, Com add-ins, document.
VSTO, etc, to write add-ins for Office, but now all that you ƒ Outlook add-ins: These appear next to an Outlook item
need is the Open Web Platform. when it’s being viewed or edited. They require Exchange
2013 or Exchange Online to host the user’s mailbox.
A high level overview of the platform
The Office Add-ins platform allows us to build apps using Anatomy of an Office add-in
existing Web standards like HTML 5, CSS, JavaScript and An Office project consists of a mainifest.xml file bundled
RESTful Service that can interact with office documents, with a Web application. The Manifest file can be submitted

56 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


How To Developers

Figure 1: Examples of task pane and content add-ins


napacloudapp.com
to the Office app store or SharePoint app catalogue,
and contains metadata about the application like Building a word cloud generator add-in
the developer’s name, target platform, etc. The Web In this section, we will build a simple word cloud generator
application can be hosted on any Web server. task pane add-in for Microsoft Word, which allows users to
select a bunch of words from which it generates a word cloud
Prerequisites for developing add-ins for Office image. It also allows users to insert that word cloud into the
The recommended IDE for add-ins development document. For the actual word cloud generation, we will take
is Visual Studio 2015. There is a free version for the help of a JavaScript library called wordcloud2.js (https://
individual developers and open source contributors github.com/timdream/wordcloud2.js). So, let’s get started with
(Visual Studio Community Edition) that can be the interesting parts.
downloaded from https://www.visualstudio.com/en-us/ 1. Fire up Visual Studio and go to File > New > Project.
products/visual-studio-community-vs.aspx 2. Although almost all of the development is done in HTML
Download and install the Office Developer Tools 5, CSS and JavaScript, for some reason, the template
extension for Visual Studio from https://www.visualstudio. for the project is located under the Visual C# section. So
com/en-us/features/office-tools-vs.aspx navigate through the drop down hierarchy to Visual C# >
Last of all, but most important, you will need Microsoft Office/SharePoint> Office Add-in.
Office 2013, 2016 or Office 365. If you don’t already have 3. Select Office Add-in Project, name it ‘Word Cloud
one of these, you can download a trial version from the Generator’, click OK and in the next screen, select Task
Office website or sign up for the Office 365 Dev program Pane Add-in as the type.
(http://dev.office.com/devprogram). Now, if you navigate to the Solution Explorer, you will
Tip: Alternatively, Napa Development Tools can also see two projects — ‘Word Cloud Generator’ and ‘Word Cloud
be used for Office development. Napa is a free, quick, Generator Web’. The former contains the Manifest.xml file and
in-browser tool for developing Office add-ins. Napa the latter contains all our HTML 5, CSS and JavaScript files.
tools can be accessed from the website https://www. 4. Navigate to the Word Cloud Generator Project and open

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 57


Developers How To

Figure 2: Screenshot of our word cloud add-in


class=”canvas”></canvas>
the Manifest file in the editor. Select the Activation </div>
tab, and in the Application section, choose Word as the <div id=”word-cloud-btn-container”>
only option as we are targeting only Word. Now, in the <button id=”cloud-from-selection”>Generate Cloud From
Required API sets section, add the ImageCoercion API Selected Text</button>
set. It is required to insert images into the document. <button id=”insert-cloud-to-document”>Insert Word Cloud To
5. Now navigate to the Word Cloud Generator Web Project. Document</button>
In this article, we will only work with the standard project </div>
structure, except for downloading the wordcloud2.js file </div>
from Github.com and adding it to the Scripts folder.
6. Open the Home.html file located in the Addin> Home 8. Open the home.js file in the same directory. Here, we will
directory and remove all the contents from inside the define a draw() function that will take a string as parameter,
<body> tag. Then add wordcloud2.js inside the <head> split it into words and generate a word cloud out of it.
tag as follows:
function draw(words) {
<scriptsrc=”../../Scripts/wordcloud2.js”></script> var list = [[‘This’, 20],[‘is ‘, 18],[‘the’,
16],[‘default’, 15],[‘Word’, 22],[‘Cloud’, 20]];
7. After this, we will create a header in the page with the if (words) {
application name, a <canvas> element to contain the var newList = [];
word cloud, with two buttons to generate a new word var regex = /[^\s\.,!?]+/g;
cloud from the selected text and to add the word cloud var individualWords = words.match(regex); //
into the document. It can be achieved by adding the Split individual words from sentences
following code in the markup inside the <body> tag. if (individualWords) {
individualWords.forEach(function(word) {
<div id=”content-header”> var wordEntry = [];
<h1 style=”text-align:center;font-weight:bold;padding- wordEntry.push(word);
top:10px;”>Word Cloud Generator</h1> wordEntry.push(Math.floor(Math.random() *
</div> 15) + 1);
<div id=”content-main”> newList.push(wordEntry);
<div id=”word-cloud-container”> });
<canvas id=”word-cloud-placeholder” width=”250” height=”250” list = newList;

58 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


How To Developers

} else { imageHTML, { coercionType: Office.CoercionType.Html },


app.showNotification(‘No data was selected. function (asyncResult) {
Please select some data in document’); if (asyncResult.status == “failed”) {
} app.showNotification(‘Error: ‘ +
} asyncResult.error.message);
WordCloud(document.getElementById(‘word-cloud- }
placeholder’), { });
list: list, }
fontFamily: ‘Finger Paint, cursive, sans-serif’, function setBase64Image(imageBase64Data) {
gridSize: 16, Office.context.document.setSelectedDataAsync(imageBas
weightFactor: 2, e64Data, {
color: ‘#f0f0c0’, coercionType: Office.CoercionType.Image,
backgroundColor: ‘#001f00’, },
shuffle: false, function (asyncResult) {
rotateRatio: 0 if (asyncResult.status === Office.
}); AsyncResultStatus.Failed) {
} app.showNotification (“Action failed with
error: “ + asyncResult.error.message);
9. Now, we will use the Office.js function to get the selected }
text from the document and pass it to the draw() method, });
on clicking the button (with the ID cloud-from-selection). }

$(“#cloud-from-selection”).on(‘click’, function () { Finally, we will call these functions on clicking the button
Office.context.document.getSelectedDataAsync(Office. (with the ID insert-cloud-to-document).We will use some
CoercionType.Text, feature detection to call the appropriate function, as follows:
function (result) {
if (result.status === Office. if (Office.context.requirements.
AsyncResultStatus.Succeeded) { isSetSupported(‘ImageCoercion’, ‘1.1’)) {
if (result.value == “”) { var $canvas = $(‘#word-cloud-placeholder’);
app.showNotification(‘No data var imageData = $canvas[0].toDataURL();
was selected. Please select some data in document’); imageBase64Data = imageBase64Data.replace(/^data:image\/
} else { (png|jpg);base64,/, “”); setBase64Image(imageData);
draw(result.value); } else {
} // get the imageURL from a server side script that saves the
} else { image on a server and return the //URL.
app.showNotification(‘Error:’, var imageHTML = “<img “+”src=’” + imageURL + “’ img/>”;
result.error.message); setHTMLImage(imageHTML)
} }
}
); 11. That’s pretty much it. Now you can hit the Start button
}); in Visual Studio and see the application running, and if
you want to download the complete source code, it is
10. Then, we will need a function to insert the word cloud available on GitHub (https://github.com/shivasaxena/
image into the document. There are two ways of Word-Cloud-Generator). The final add-in looks like
achieving this. We will have to check if the running what’s shown in Figure 2.
version of Word supports the insertion of base64
encoded images, in which case we will directly insert
it in the document, or else we will send the image to a By: Shiva Saxena
server side script, save the image on the server and then The author is a FOSS enthusiast. He is currently working as
a consultant involved in developing Enterprise Application
insert an <img> tag with the URL of the saved image. and Software as a Service (SaaS) products. He has hands on
The functions to do this will be as shown below: development experience with technologies like Android, Apache
Camel, C#.NET, Hadoop, HTML5, Java, OData, Office, PHP and
function setHTMLImage(imageHTML) { React, among others, and loves to explore new and bleeding edge
technologies. He can be reached at shivasaxena@outlook.com
Office.context.document.setSelectedDataAsync(

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 59


Developers How To

Build a Personal Safety Application


in App Inventor 2

Astound your friends by building your own Android personal safety app! This tutorial in our on-going
series on App Inventor 2 guides the reader in building such an app. Reading previous articles would be
a great help to those just joining us in this quest to build Android apps.

P
ersonal safety is of prime concern to each one of The purpose of the application
us. Many a time we are in difficult and dangerous As explained earlier, the purpose of this application revolves
situations and need a helping hand to extricate around the need to extricate the mobile device user from a
us from harm’s way. As the proverb says, “Prevention dangerous situation. We hope to build an Android mobile
is better than cure.” So it is better to be prepared with a application which will send the location of the user, along
solution before a potentially harmful situation arises. with some predefined text, to certain people/agencies and
In this article, we are attempting to build a place a call to them to describe the situation. We will enable
personal safety application, which in times of trouble users to set the contact numbers of the people they wish to
can help us. Our goal here is to inform the ones who contact in case of emergencies. You are already aware of all
care for us about where we are and the activities the components that will be used for this application like the
we are involved in. We will be building a mobile texting component, the phone call component, location sensor,
application using which, on pressing a single button, buttons, labels, Tiny DB, etc.
we can send critical information about ourselves to
the people we intend to send it to. GUI requirements
This tutorial series requires some basic as well as Every application requires a graphical user interface or GUI,
a little advanced knowledge about building Android which helps the user to interact with the on-screen components.
applications with App Inventor. So let’s use the How each component responds to user actions is defined in the
knowledge acquired from the past articles in this blocks editor section. As per our current requirements, we hope
series, to build this app. to have a single button which will trigger all the events, and a

60 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


How To Developers
dedicated screen where the user can fill Component’s
Purpose Location
certain details like phone numbers to name
contact in case of an emergency. Since Palette-->User Interface-
1 Button To trigger events
->Button
we are using multiple screens this
time, let’s note down their components Palette-->User Interface-
2 Label To display a label
->Label
separately. The very first screen, which
Palette-->User Interface-
will have the trigger button, is Screen1 3 Text box To get user inputs
->Text Box
and the other will be Screen2. How to
For SMS related
add a new screen into the application 4 Texting Palette-->Social-->Texting
functions
will be explained as we move further. For call related func- Palette-->Social-->Phone
5 Phone call
tions Call
GUI requirements for Screen1 To save data persis- Palette-->Storage-->Tiny
1. Button: The button will let you 6 Tiny DB
tently DB
trigger the event and is a very Horizontal To arrange the child Palette-->Layout--
7
essential component. arrangement components >Horizontal Arrangement
2. Texting component: This is essential To get the device’s Palette-->Sensors--
8 Location sensor
to SMS/text message related location >Location Sensor
functionality. It enables you to To display on-screen Palette-->User Interface-
9 Notifier
read and write text messages to the information ->Notifier
desired phone numbers.
3. Phone call component: This visible on the screen. horizontally aligned. Whichever
is essential for call related 4. Tiny DB: When you need to store component you put into it will be
functionality. It can detect your some data persistently within the kept horizontally aligned.
incoming calls and place new device, Tiny DB comes into the Given above are the components
outgoing calls, as required. picture. It enables you to store that we require for this application.
4. Tiny DB: When you need to store values, corresponding to unique Let’s drag them on to the designer from
some data persistently within the tags, in the phone database. the left hand side palette.
device, Tiny DB comes into the 5. Notifier: This is used to display Drag and drop the components
picture. It enables you to store some instructions or give controls mentioned in the table above the viewer.
values corresponding to unique tags, over your existing components. You Visible components can be seen by
into the phone database. can see how it functions in more you while the non-visible components
5. Location sensor: This will sense detail as we will be implementing it will be located beneath the viewer
your current location in terms of in the app. under the tag ‘non-visible’.
latitude and longitude. 6. Horizontal arrangement: This Put separate components for
6. Notifier: This is used to display component plays a crucial role Screen1 and Screen2.
some instructions or give controls in keeping the child components To create a new screen, click on
over your existing components. You the ‘Add Screen’ button just above the
will understand its functionality Screen1 designer viewer.
better as we implement it in the app.

GUI requirements for Screen2 Press For help Screen2

1. Button: This will let you trigger Emergency Contacts


the event and is a very essential Text Contact:

component. Call Contact:


2. Label: The label component gives
you control to write static text or Save
headings. The text of the label
component remains as it is until you
change it in the blocks editor.
3. Text box component: This allows
users to give their inputs as required Non-visible components
Non-visible components
by the application. As soon as the
PhoneCall1 Texting1 LocationSensor1 TinyDB1 Notifier1
text box is highlighted, it activates TinyDB1

the device keyboard to make it Figure 1: Designer screen (Screen1) Figure 2: Designer screen (Screen2)

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 61


Developers How To
Components Components understand them.
Screen1: When the Help button is
Screen1 Screen2 clicked, the following things will happen
(depicted in Figures 5 and 6).
Help_Button Label1 It will generate a custom text
PhoneCall1 HorizontalArrangement1 message for the texting component,
taking the current latitude and longitude
Texting1 Label2 values from the location sensor.
LocationSensor1 Next, it will decide which number
TextBox1
the text message should be sent to, and
TinyDB1 HorizontalArrangement2 will then fetch the phone number from
Notifier1 the database.
Label3
Similarly, for the call component, we
TextBox2 will need a phone number to which the
call should be placed.
Save_Button It is a good practice to think of all the
TinyDB1 user scenarios and be prepared in advance;
so if such a scenario occurs, the user can be
notified to take the necessary action.
The next lines will check whether the
Rename Delete fields returned by the DB component are
empty. If they are, then the user should
Figure 3: Component viewer (Screen1) be notified to take appropriate action.
Rename Delete If the phone number fields are not
If you have dragged and placed empty, we should proceed with the text
everything, the layout will look Figure 4: Component viewer (Screen2) message and phone call facility.
something like what’s shown in Figures When the Help button is given a long
1 and 2. like to go over what we want from press, it will open Screen2 where the user
Make the necessary property changes our application. Listed below are the has set the emergency numbers.
like we did when changing the text functionalities that we are looking for: Don’t forget to create Screen2 in
properties for label and button components ƒ On pressing the button, a custom advance though.
in previous articles in this series. SMS should be sent to the number Screen2: When the Save button is
Renaming the components helps to already saved in the database. clicked, the following things will happen
identify them in the blocks editor. ƒ On pressing the same button, a phone (depicted in Figures 7 and 8):
So now, your graphical user interface call should be placed to another 1. First, it will check whether the user
is ready. Figure 1 shows exactly how the number saved in the database. inputs given are blank.
application will look after the installation. ƒ By any means, the user should 2. If they are, then nothing will be
The hierarchy of the components be able to change the emergency saved and the user will be notified.
that we have dragged to the designer is contact numbers. 3. If both the fields are filled, it will
illustrated in Figures 3 and 4. ƒ If the people whose numbers have allow the user to save the numbers
After having read the previous been saved for emergencies are not into the database.
articles on App Inventor, I am sure you available, the user should be duly When the back button is pressed
are pretty clear about how to prepare notified. from Screen2, it will close the current
a GUI. It isn’t a complex task — the screen and will take the user to Screen1.
components just have to be placed on the Blocks in the blocks editor You are now done with the blocks
viewer and a few property changes are I have already prepared the blocks for editor. The next step is to download and
required to make them look clearer. you. All you need to do is to drag the install the app on your phone to check
So let’s move on and add these relevant blocks from the left side palette how it works.
behaviours using the blocks editor. I and drop them on the viewer. Arrange the
hope you remember how to switch from blocks exactly as shown in the images Packaging and testing
designer to the blocks editor. There (Figures 5, 6, 7 and 8). I will explain what To test the app, you need to get it on your
is a button available right above the each does and how it is called. phone. First, download the application to
Properties pane to do this. The blocks in these figures may your computer and then move it to your
But before we move on, I would look a bit daunting; so let me help you phone via Bluetooth or a USB cable. I’ll

62 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


How To Developers
your SD card, you need to install it.
Follow the on-screen instructions
to install it. You might get some
notification or warning saying,
“Install from untrusted source.”
Allow this from the settings and
after successful installation you will
see the icon of your application in
the menu of your mobile. Here you
will see the default icon, which can
be changed, and we will tell you
how to do this as we move ahead on
this course.
Figure 5: Blocks editor Image 1 (Screen1) I hope your application is working
exactly as per your requirements.
When Help_Button .LongClick Now, depending upon your needs and
do open another screen screenName Screen2
customisation, you can change various
things like the image, sound and
Figure 6: Blocks editor Image 2 (Screen1) behaviour also.

Debugging the application


We have just created the prototype of the
application with very basic functionality.
What else might the user be interested in?
Now, let’s look at various cases which
require serious thought so as not to annoy
the user. Then let’s ensure that your app
addresses these requirements. Consider
the following cases:
1. Wouldn’t it be nice to add some
background for Screen1?
2. What other ways could we use to
trigger the SMS and call functions
rather than clicking the button?
3. Wouldn’t it be nice if we could
provide a recording feature to capture
Figure 7: Blocks editor Image 1 (Screen2) all that is happening around the user
during that period of danger?
When Screen2 BackPressed These are some of the possible
scenarios that could occur and users will
do close screen be pretty happy seeing these problems
addressed.
Figure 8: Blocks editor Image 2 (Screen2) Think about all these scenarios and
consider how you could integrate these
tell you how to download it. set for it. into the application. Do ask me if you fail
1. In the top row, click on the Build 3. Now you need to get this APK file to address any of the above requirements.
button. It will show you the option to to your mobile phone either via You have successfully built another
download the APK to your computer. Bluetooth or via USB cable. Once useful Android app for yourself. Happy
2. While downloading this, you you have placed the APK file on Inventing!
can see its progress and after
a successful download, the By: Meghraj Singh Beniwal
application will be placed in the The author has a B. Tech degree in electronics and communication, is a freelance writer
download folder of your directory and an Android app developer. He is currently working as an automation engineer at
or the preferred location you have Infosys, Pune. He can be contacted at meghrajsingh01@rediffmail.com

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 63


Developers Overview

A Guide to Material Design, a


Modern Software Design Language
Material Design is a new and continuously evolving design language. The principles and concepts
introduced by it appear realistic and pleasing, and it is platform and language independent.

A
design language or design vocabulary is an overarching design of user interfaces. The most popular examples of
scheme or style that guides the design of a suite of software design languages are:
complementary products or architectural settings. ƒ Skeuomorphism
Designers wishing to give their suite of products a unique but ƒ Metro Design
consistent look and feel define a design language for it, which ƒ Flat Design
can describe choices for design aspects such as materials, colour
schemes, shapes, patterns, textures or layouts. They then follow Skeuomorphism
the scheme when designing each object in the suite. In simple A skeuomorph is a derivative object that retains ornamental
words, software design is a set of guidelines, which provides design cues from structures that are necessary in the original.
or suggests the design aspects in your software for a consistent In simple words, skeuomorphism enforces the UIs to be
look across entire applications. Here’s an example: the themes designed to look like the original object of the real world
for all activities in Android have the same look and feel. and behave in that way too. Many computer programs have
Design languages are only guidelines; they are not rules. These a skeuomorphic graphical user interface that emulates the
guidelines are designed such that users broadly follow them, aesthetics of physical objects.
but one can surely innovate and bend all the rules of a design An example of this trend was the 1998 RealThings
language. People tend to consider them hard and fast rules and package. A more extreme example is found in many music
never do anything outside those guidelines, which is not a good synthesis and audio processing software packages, which
thing in developing engaging user interfaces. These guidelines closely emulate physical musical instruments and audio
were also generated by innovation. equipment. Functional input controls like knobs, buttons,
switches and sliders are often careful duplicates of those
The history of design languages found on the original physical device being emulated. Some
The concept of a design language has existed ever since the software even include graphical elements of the original

64 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Overview Developers

Figure 4: One of the goals of Material Design, which is consistent design


across different screen sizes

Figure 1: An application showing the use of skeuomorphism it easier for those familiar with the original device to use the
digital emulation by making certain affordances stronger. On
Metro the other hand, people opposing this design language say that
Just another WordPress site HOME MY IMAGES SLIDESHOWS JUMBO
it requires resemblance of the original object, so it limits the
Sample Page About Us Contact Us
creativity of a programmer or of UI designers to design or try
Home Some images anything new in their interfaces.
Sample Post 10 New Post
Metro Design
It is a long established fact that a reader will be
distracted by the readable content of a ...

Metro is a typography- and geometry-focused design language


tablished fact that a
reader will be dis...

Sample sample post 8 created by Microsoft primarily for user interfaces. A key
Pos...
“On the other hand we denounce with
righteous indignation and dislike men who are
design principle is better focus on the content of applications,
relying more on typography and less on graphics (content
so beguiled and...
It is a long
established fact that

before chrome). Early examples of Metro principles can


a reader w...

Figure 2: WordPress theme showing the use of Metro Design be found in Encarta 95 and MSN 2.0. The design language
evolved in Windows Media Centre and Zune, and was
USERNAME
formally introduced as ‘Metro’ during the unveiling of
PASSWORD
SUNNY AMSTERDAM
Windows Phone 7. It has since been incorporated into several
TODAY TUE WED
LOG IN
of the company’s other products, including the Xbox 360
26˚/ 34˚ 29˚/ 21˚ 30˚/ 24˚ system software, Xbox One, Windows 8, Windows Phone,
SUN

28
MON

29
TUE

30
WED

31
THU

1
FRI

2
SAT

3
and Outlook.com under the names Microsoft Design language
and Modern UI after Microsoft discontinued the name ‘Metro’
SEARCH 4 5 6 7 8 9 10
11 12 13 14 15 16 17

supposedly because of trademark issues.


18 19 20 21 22 23 24
DOWNSTATE UPSTATE
25 26 27 28 29 30 31 ITEM 1

ITEM 2

22312 331 1352


ITEM 3
Flat Design
Flat Design is a minimalist UI design genre, or design
ITEM 4

ITEM 5
70%
ITEM 6 language, currently used in various graphical user interfaces
(such as websites or Web applications). Flat Design is a style
Figure 3: An illustration of different components in Flat Design of interface design without any stylistic elements that gives
the illusion of three dimensions (such as excessive use of drop
design that serve no user interface function, such as handles, shadows, gradients or textures) and is focused on a minimalist
screws and ventilation holes. Even systems that do not employ use of simple elements, typography and flat colours. Designers
literal images of some physical object frequently contain may prefer Flat Design because it allows interface designs to
skeuomorphic elements, such as slider bars that emulate linear be more streamlined and efficient. It makes it easier to quickly
potentiometers and tabs that behave like tabbed file folders. convey information while still looking visually appealing and
Skeuomorphs need not be visual. The shutter-click sound approachable. It also makes it easier to design an interface
emitted by most camera phones when taking a picture is an that is responsive to changes in browser size across different
auditory skeuomorph; it does not come from a mechanical devices. With minimal design elements, websites are able to
shutter, which camera phones lack, but from a sound file in load faster and resize easily, and still look sharp on high-
the phone’s operating system. Another example is the swiping definition screens. As a design approach, it is often contrasted
hand gesture for turning the ‘pages’ or screens of a tablet. to skeuomorphism and rich design. (Though it should be noted
People who are in favour of this language argue that it makes that Flat Design can and does use skeuomorphs just as much

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 65


Developers Overview
ƒ Various guidelines for shadows, such as different shadows
for resting and the active state of objects
Motion: Motion is great. We all love motion and
animations in interfaces. It makes the application feel alive,
which is what motion in Material Design is all about. Motion
is a very important part of design nowadays. If an object is
to be placed on a screen, making it appear with some motion
(such as fade or reveal), rather than just popping it in front
Figure 5: The material shown in Mate- Figure 6: Motion in Material Design of the user, would make it easier to understand for the user.
rial Design, which lives in the 3D world
However, putting random animations anywhere does not
help users either; rather, it could irritate them. Motion is to
provide continuity between different states of the application.
System Status Bar
Motion should be applied so that it guides the user
about where the flow of the application is headed and
Background
what’s the result in response to the input of the user. For
example, suppose there is a card on the screen, showing
Floating Button
some information; when the user clicks on the card, it
Content card
expands to fill the whole screen and shows more information.
This provides continuity between these two states of the
System Navigation Bar application. Going back would shrink the card again, and
users will get continuity back and forth in the application.
Layout: Layout in Material Design is composed of many
Figure 7: Stacking in layouts following Material Design components. Mobiles, tablets and desktops—all share almost
the same elements such as toolbars, bottom bars, floating action
as a realistically designed UI can.) buttons, SideNav bars, etc. The Material Design specification
shows different layout arrangements for these elements on the
Material Design different screen sizes of phones, tablets and desktops.
Material Design is a modern design language which enforces It also explains each element, and shows some Android-
a consistent look and feel of the application across devices specific elements and how these can be used in user interfaces.
such as a phone, tablet and a computer. It is a very broad In layout, a very important thing to note is the stacking of
concept now, and has guidelines for almost all the user elements. As stated before, objects in Material Design move
interfaces and patterns. Material Design is a live process; it is and exist in a 3D space. In 3D space, material is stacked one
continuously evolving and making itself better by including on top of the other rather than every object being on the same
other design patterns in the document. This language is plane. So the layout section shows how all the elements, such
mainly about concentrating on three things: as the background plane, toolbar, floating action button, etc, are
ƒ Material placed on one another, and how they cast crisp or soft shadows
ƒ Motion according to their position along the z-axis.
ƒ Layout
Material: In Material Design, one of the goals is to make
the UIs natural. In nature, everything is in a 3D space, so References
elements in Material Design exist and behave like objects in [1] https://www.google.co.in/design/spec/material-design/
introduction.html
the real world. It provides meanings and visual cues to the
[2] http://papyros.io/qml-material/
user. Material in Material Design instantly reacts to user input. [3] https://github.com/samvidmistry/PyMaterial
An example would be the button; when the button is pressed, [4] http://www.materialup.com/
it rises from its position to make contact with the finger of the [5] https://www.materialpalette.com/
[6] http://www.hongkiat.com/blog/material-design-resources/
user. In that process, the button also casts a shadow as it rises,
giving the impression that the button is in a 3D world. This also
implies that the material is opaque and solid, so that touch or By: Samvid Mistry and Prakash Patel
any kind of input events cannot pass through the material and Samvid Mistry is in the final year of his diploma in computer
interact with the material behind the foremost material sheet. engineering at Nirma University. He is an Android developer.
Other guidelines are: He can be contacted at mistrysamvid@gmail.com,
http://www.samvidinfotech.in
ƒ To show stacking of material, usage of shadows to
separate them Prakash Patel is an assistant professor in computer engineering at
Nirma University. He can be contacted at prakash.patel@nirmauni.ac.in
ƒ Material can grow or shrink, but it cannot bend or fold

66 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try OpenGurus

The Z File System: It’s Honest and Different


The Z File System, or ZFS, is an advanced file system designed to overcome many of the major
problems found in previous designs. Initially developed by Sun Microsystems, its development has
now been moved to the OpenZFS Project.

D
ata is priceless. Small ventures to the fortune 500 back, the checksum is calculated again. If the checksums do
spend crores to maintain backups and ensure data not match, a data error has been detected. ZFS will attempt to
integrity. The loss of even a byte of data can impact automatically correct errors when data redundancy is available.
huge MNCs and could possibly threaten a nation’s security. Pooled storage: Physical storage devices are added to
Ever since the evolution of computers and file systems, a pool, and storage space is allocated from that shared pool.
experiments and research have been going on in an effort to Space is available to all file systems, and can be increased by
find an efficient, trustworthy backup system. adding new storage devices to the pool.
Performance and scalability: It is a 128-bit file system
What is the Zettabyte File System (ZFS)? that’s capable of managing zettabytes (one billion terabytes)
ZFS is an advanced, open source file system and logical of data. Multiple caching mechanisms provide increased
volume manager designed by Sun Microsystems for use in performance. Examples include ARC, an advanced memory
its Solaris operating system, and licensed under the Common based read cache; L2ARC, a second level disk based cache;
Development and Distribution License (CDDL). The name and disk-based synchronous write cache, ZIL.
ZFS is a registered trademark of the Oracle Corporation. In addition to these, ZFS supports multiple RAID levels
(redundant array of independent disks) that further improves
Why should you use ZFS? performance and diminishes redundancy.
The answer to this question lies in the three main design goals,
which have translated into ZFS’most appealing features. Installing ZFS in Ubuntu
Data integrity: Integrity is the keyword that any serious In Ubuntu, ZFS can be installed by running the following code:
user searches for. In ZFS, all data includes a checksum of the
data. When data is written, the checksum is calculated and $ sudo add-apt-repository ppa:zfs-native/stable
written along with it. When that data is later read

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 67


OpenGurus Let’s Try
$ sudo apt-get update three drives and uses striping to divide data across all the
$ sudo apt-get install ubuntu-zfs hard drives, with additional parity data divided across
all disks. If one of the hard drives dies, you won’t lose
Once the installation is over, run the command any of your data. RAID 5 offers data redundancy with a
shown below: lower storage cost than RAID 1). RAID Z voids the ‘write
hole’ by using copy-on-write. If a single disk in your pool
$ sudo zfs list dies, simply replace that disk, and ZFS will automatically
rebuild the data based on parity information from the
This should give an output that states ‘No datasets other disks. To lose all the information in your storage
available’, which ensures that the installation is fine. pool, two disks would have to die. To make things even
Now we have just one hard drive. more redundant, you can use RAID 6 (RAID-Z2 in the
case of ZFS) and have double parity.
Configuring ZFS To achieve the above, run the following command:
As mentioned earlier, our installation has just one hard drive.
Let’s suppose that we need to add six more hard drives to $ sudo zpool create -f <storage pool name> raidz <drive1>
our system. <drive2> <drive3>
Now, let us assume that we have finished adding the six
hard drives. But since none of them is partitioned, they are
currently unusable. What makes ZFS different?
This is where the benefits of ZFS come in, freeing us from ZFS is significantly different from any previous file
the burden of creating partitions (although we have the liberty system because it is more than just a file system.
of creating these, if we want to). Combining the traditionally separate roles of volume
Let’s now create a storage pool using two of our hard drives: manager and file system provides ZFS with unique
advantages. The file system is now aware of the
$ sudo zpool create -f <storage pool name> </path/to/drive1> underlying structure of the disks. Traditional file systems
</path/to/drive2> could only be created on a single disk — one at a time.
If there were two disks, then two separate file systems
zpool create is the command used to create a new storage would have to be created. In a traditional hardware RAID
pool, and -f overrides any errors that occur (such as if the configuration, this problem was avoided by presenting the
disk(s) already have information on them). operating system with a single logical disk made up of the
To see the newly created pool, run the following space provided by a number of physical disks, on top of
command: which the operating system placed a file system. ZFS’s
combination of the volume manager and the file system
$ sudo zfs list solves this, and allows the creation of many file systems
— all sharing a pool of available storage.
Now let us look at where RAID levels come into play. One of the biggest advantages of ZFS’s awareness of the
What if we create a 2KB file in the storage pool that we physical layout of the disks is that existing file systems can
just made? 1KB would automatically go to the first drive be grown automatically when additional disks are added to
and 1KB to the second one. Then when we read the 2KB the pool. This new space is then made available to all the file
file, each hard drive would present 1KB to us, combining systems. ZFS also has a number of different properties that
the speed of the two drives. Here we have used RAID 0. can be applied to each file system, giving many advantages
When you use RAID 0, your multiple disks appear to be a to creating a number of different file systems and datasets
larger and faster hard disk. One disadvantage of RAID 0 is rather than a single monolithic file system.
that it is fragile. If any one of the drives dies, we lose our
entire data. But ZFS supports higher RAID levels that can References
overcome this disadvantage. [1] https://www.freebsd.org/doc/handbook/zfs.html
To get a feel of how ZFS overcomes data loss as in the [2] http://www.howtogeek.com/175159/an-introduction-to-
above case, let us look at another scenario. the-z-file-system-zfs-for-linux/
First, let’s delete the pool that we have created, as follows:

$ sudo zpool destroy <storage pool name>


By: Athira Lekshmi C.V.
The author is pursuing a Bachelor’s degree in computer science and
engineering at Amrita Vishwa Vidyapeetham in Kollam, Kerala. She is a
Now we are going to create a RAID Z pool, which is an
FOSS enthusiast and an active member of FOSS club in her college.
improved version of a RAID 5 pool (RAID 5 needs at least

68 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try OpenGurus

SimpleCV: Making Vision


Computing Easy and Effective
SimpleCV makes computer vision tasks simpler and quicker to implement by providing a wrapper
over the underlying complex code in the OpenCV library. In addition to this, SimpleCV provides
functionalities such as optical character recognition (OCR) through easy-to-use functions.

C
omputer vision is a domain that’s under active OpenCV have been taken care of in SimpleCV by providing
research, with applications spreading across a direct-to-use functions for performing repeatedly used
spectrum of fields such as robotics, human computer computer vision tasks. SimpleCV is written in Python and
interaction (HCI), etc. Computer vision involves tasks such can be installed in all major operating systems such as Linux,
as image reception, processing, reasoning, video stream Windows and Mac. It is available with the BSD licence.
handling, etc. These tasks involve a lot of mathematical SimpleCV allows developers to handle both images and
processing, which is not easy to implement. To handle videos. The source image/video is acquired directly using
this problem and make the life of the developer simpler, various input sources as listed below:
frameworks like SimpleCV become handy. ƒ Web cams
ƒ Kinects
SimpleCV ƒ FireWire and IP cameras
SimpleCV is an open source framework for performing ƒ Mobile phone cameras
computer vision tasks. And like its name suggests, it has made SimpleCV facilitates not only making changes to the
computer vision simple. The complexities associated with images but also allows them to be understood.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 69


OpenGurus Let’s Try
files/opencv-win/2.3.1/OpenCV-2.3.1-win-superpack.exe/
Pyt
download

Op
ho

SciP
n2

en
CV
.7 After the downloading has been completed, execute the

Y
SimpleCV Dependencies
.exe file and extract to the folder C:\OpenCV2.3\. It can be
Pyth

Eas
Num
extracted to any other folder too, but the environment paths
on S

y_in
PY

are to be set accordingly.


stall
etup

After the installation is over, set the ‘PATH’ variables


Too
ls

accordingly, as shown below:


Figure 1: SimpleCV dependencies
SETX PATH C:/Python27/;C:/Python27/Scripts/;C:/OpenCV2.3/
Installation opencv/build/x86/vc10/bin/;%PATH%
SimpleCV can be installed on all major operating systems. SETX PYTHONPATH C:/OpenCV2.3/opencv/build/
As it makes the computer vision code simple by providing python/2.7/;%PYTHONPATH%
a wrapper over existing libraries, it has many dependencies.
Hence, during installation, all those dependencies need to To update the ‘PATH’ variables, it is required that you
be taken care of for making SimpleCV ready to run. The close the current command prompt and reopen it. Then
process of installation varies in different operating systems. execute the following commands:
The dependencies for SimpleCV are shown in Figure 1.
If you prefer to install SimpleCV in Ubuntu, use the easy_installpyreadline
following commands: easy_install PIL
easy_installcython
sudo apt-get install ipython python-opencv python-scipy easy_install pip
python-numpy python-pygame python-setuptools python-pip pip install ipython
sudo pip install https://github.com/ingenuitas/SimpleCV/ pip install https://github.com/ingenuitas/SimpleCV/
zipball/master zipball/1.3

Once the installation has been successfully completed, run After the successful installation by executing the above
simplecv from the console. commands, type ‘simplcv’ from the command prompt at the
For Windows operating systems, there are two different installation folder, to open the simplecv shell. Otherwise, you
ways of installing SimpleCV. One method is to download the may click on the SimpleCV icon on the desktop.
Windows Superpack from http://www.simplecv.org/download/
. This super pack includes all the dependencies required for SimpleCV shell
SimpleCV installation. The SimpleCV shell allows us to execute the commands and
However, if your system already has some of the get the results in an interactive manner. Some of the shell
dependencies and you want to keep them as they are, then interaction commands are as listed in Table 1.
individual components can be downloaded separately, as
shown below: Command Description
1. Download and install Python 2.7 using http://www.python. Exit Enables users to exit the SimpleCV shell.
org/ftp/python/2.7.3/python-2.7.3.msi The same thing shall be achieved by press-
ing Ctrl + D.
2. The Python set-up tools for Windows can be downloaded
tutorial Allows users to start the SimpleCV interac-
from http://pypi.python.org/packages/2.7/s/setuptools/
tive tutorial
setuptools-0.6c11.win32-py2.7.exe
clear Performs the cleaning of the SimpleCV shell
3. Installation of the SciPy super pack can be carried
example Provides a collection of examples that we
out using http://sourceforge.net/projects/scipy/files/
can run
scipy/0.9.0rc5/scipy-0.9.0rc5-win32-superpack-
python2.7.exe/download Forums Launches a Web browser for the help forums
4. Installation of the NumPy super pack can be done walkthrough Launches a Web browser with a walk-through
by using http://sourceforge.net/projects/numpy/ Table 1: SimpleCV shell interaction commands
files/NumPy/1.6.2/numpy-1.6.2-win32-superpack-
python2.7.exe/download The SimpleCV console is shown in Figure 2.
5. Installation of Pygame for Windows can be done using SimpleCV incorporates a detailed and elegant help system.
http://pygame.org/ftp/pygame-1.9.1.win32-py2.7.msi The help on any of the components/libraries can be retrieved
6. Installation and configuration of OpenCV can be done by using the Help command. An example of a command and
by using http://sourceforge.net/projects/opencvlibrary/ its output is listed below:

70 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try OpenGurus

SimpleCV HelloWorld
As stated in the beginning of this article,
the core idea of SimpleCV is to make
machine vision simple. A HelloWorld
example is shown below:

from SimpleCV import Camera


# Initialize the camera
cam = Camera()
# Loop to continuously get images
while True:
# Get Image from camera
Figure 2: SimpleCV console img = cam.getImage()
# Make image black and white
SimpleCV:1> help(Image) img = img.binarize()
# Draw the text “Hello World” on image
The successful response for the above command img.drawText(“Hello World OSFY !!!”)
would be: # Show the image
img.show()
Help on class Image in module SimpleCV.ImageClass:
class Image As can be observed from the code sample, SimpleCV
| **SUMMARY** provides direct functions for performing the vision tasks. The
| The Image class is the heart of SimpleCV and allows you to first step is to get the image source from a camera, which
convert to and can be done using the cam.getImage(). To convert into black
| from a number of source types with ease. It also has and white, the direct function img.binarize() should be used.
intelligent buffer Similarly, drawing the text also can be carried out using the
| management, so that modified copies of the Image required simple method img.drawText(). The output image can be
for algorithms displayed using img.show().
| such as edge detection, etc can be cached and reused when SimpleCV provides mechanisms to capture the live
appropriate. camera feed using simple methods as shown below:
|
| Images are converted into 8-bit, 3-channel images in RGB from SimpleCV import Camera
colorspace. It will cam = Camera()
| automatically handle conversion from other representations cam.live()
into this
| standard format. If dimensions are passed, an empty image An example of machine learning
is created. Apart from performing the simple image manipulation tasks
| as shown in the HelloWorld example, it allows us to perform
| **EXAMPLE** comparatively advanced tasks such as image classification.
| The classification is based on the machine learning. A
| >>>i = Image(“/path/to/image.png”) sample code for performing classification is available at the
| >>>i = Camera().getImage() official code repository of SimpleCV (https://github.com/
| sightmachine/SimpleCV/blob/develop/SimpleCV/examples/
| You can also just load the SimpleCV logo using: machine-learning/machine-learning_nuts-vs-bolts.py). This
| example classifies the given image into ‘Nuts’ and ‘Bolts’
| >>>img = Image(“simplecv”) with Scikits-learn liberary. The image features such as
| >>>img = Image(“logo”) “area”, “height” and “width” are extracted from the imges
| for the classification purpose:
| Or you can load an image from a URL:
| SimpleCV Blob handling
| much more here ... Blobs are regions of similar pixels. Blobs are used to
detect items of interest in the image. Once the blobs
A detailed tutorial on interactive shells is provided at are identified, many operations shall be performed as
http://tutorial.simplecv.org/en/latest/examples/shell.html shown in Figure 3.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 71


OpenGurus Let’s Try

Counting Number of Blobs Count Pixels Motion Detection Example


Motion detection is an important task in many
Color Computation Calculate Width, Height machine vision applications. SimpleCV allows the
SimpleCV Blob Operations developer to perform the motion detection task using
Angle Computation Find Mid Point less than ten lines of code as shown below:

Geometric Shape Computation Blob Shap Comparison from SimpleCV import *


cam = Camera()
Figure 3 : SimpleCV Blob operations threshold = 5.0 # setting an adjustable threshold for
motion detection
Blob Detection - Input while True:
previous = cam.getImage() #get the current image
time.sleep(0.5) #pause for half a second. The 0.5 can
be adjusted
current = cam.getImage() #get another image
diff = current – previous # compute the image
difference
matrix = diff.getNumpy()
Blob Detection - Output mean = matrix.mean()
diff.show()
if mean >= threshold:
print “Some Movements Detected”

As it can be observed from the aforementioned


code, the motion detection is carried out by subtracting
Figure 4 : Blob detection input and output images the current image from the snapshot taken half a second ago.
The SimpleCV facilitates direct subtraction of these image
A simple code for blob handling is as shown below: shots as shown in the code.
A detailed capability demo of SimpleCV is available at
from SimpleCV import Image https://youtu.be/UZSm7Q2bZoc. In Summary, SimpleCV
coins = Image(“coins.png”) really makes the Vision computing easy by providing a
binCoins = coins.binarize() wide collection of methods. As it is Python based, the
blobs = binCoins.findBlobs() seamless integration shall be performed with other useful
blobs.show(width=8) Python Libraries.

After binarizing the image, the blobs are identified By: Dr K.S. Kuppusamy
by directly applying the function “.findBlobs()”. The
The author is assistant professor of computer science at the
example program loads an image with coins in it. The School of Engineering and Technology, Pondicherry. He has
findBlobs is used to identify the pixels representing the 10+ years of teaching and research experience in academia and
coins leaving the background pixels. The screenshot of industry. His research interests include accessible computing,
the input and output images for the blob detection are as Web information retrieval, mobile computing, etc. He can be
reached at kskuppu@gmail.com
shown in Figure 4.

Your favourite Magazine on


THE COMPLETE MAGAZINE ON OPEN SOURCE
Open Source is now on the Web, too.

OpenSourceForU.com
Follow us on Twitter@LinuxForYou

72 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Overview For U & Me

A Beginner’s Guide to Linux


There are many valid and compelling reasons to opt for Linux as the operating system for a
desktop computer, the most important being that it can be tailored to the user’s needs and tastes.
This article, aimed at noobs, describes the salient features of the Linux operating system.

L
inux was designed for the sole purpose of providing kernels have several pros and cons, some of which include:
desktop users with an affordable but functional Pros
operating system in a world where the most commonly ƒ Direct and faster approach for hardware to access
used OS comes with a high price tag. programs
Linux was originally created in 1991 by Linus Torvalds, a ƒ Simplicity and ease of in-process communication
Finnish student from the University of Helsinki, for his own ƒ Quick processing of requests
personal computer, and with zero plans for expansion beyond Cons
one platform. However, Linux grew to become adaptable ƒ Lack of security due to processes running solely on
with several computer architectures and now runs on every supervision mode
platform imaginable. ƒ Enormous memory and installation footprints
Linux has fascinated developers ever since it was created However, kernel developers did a good job by
and, as a result, its open source code has seen several overcoming the flaws associated with the monolithic kernel
contributions from more than 12,000 programmers. by modelling modules of the kernel, which can be both loaded
The Linux kernel (the main part of the OS located at and unloaded at any time. Modules that are able to support
the core) has been released under the GNU General Public features such as low level virtualisation can also enable you
License (GPL) version 2. to replace your kernel without rebooting the entire system,
although this happens only in rare cases.
Linux kernel architecture
The Linux kernel operates on a monolithic architecture, which Common features of the Linux kernel
basically means that the complete operating system works on Let’s look at some of the common features of the Linux kernel.
the kernel space and on the solo supervisor mode. Monolithic Multi-user support: A Linux system supports several

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 73


For U & Me Overview
user accounts at once as well as multiple users logged in on with other commercially available operating systems like
a single system at a given period of time. Users also enjoy Microsoft Windows.
the privilege of arranging their environment in a way that Freedom: One of the reasons why we love Linux is the
suits them best with the aid of customisable desktop icons freedom that comes with it. With Linux, you have the liberty
and menus. Additionally, to ensure privacy, user accounts are to control each aspect of the operating system, including the
password protected (at the discretion of the user). desktop interface, windows manager and task bars.
Multi-tasking capabilities: With a Linux OS, several Software: With Linux, you are able to use different
programs are capable of running at the same time, both software for different tasks, all at once. Interestingly, because
on the interface and at the background. The ability to run Linux supports free and open source software, you can modify
several programs at once enables the Linux kernel to act as a the source code when and how you want to.
server, with background processes (also known as daemons)
responding to user requests. The disadvantages of Linux
Graphical user interface (GUI): Linux supports a Not easy to master: Mastering the Linux operating system
powerful framework for working with graphical applications requires time and patience as well as a willingness to learn.
called the X Windows System or simply X, which ensures that Otherwise you may have to hire the services of a dedicated
X-based GUIs receive adequate support and run flawlessly. Linux coach, which may be costly.
Other than X, you are also provided with an X-based desktop Compatibility issues: By being free, Linux has put itself
setting to give your GUI processes a lively look and feel. at a disadvantage by not being able to catch up with new
Hardware management and support: Every kind of hardware. Although developers are trying their best to ensure
device and hardware that you connect to your desktop has pre- that Linux meets the compatibility requirements of new
configured support from Linux. All removable devices such hardware, they can do little when people are provided with
as CD-ROMs, sound and memory sticks, tapes and so on, can several other operating systems as an alternative.
be supported by Linux. Lacks some key features: Although Linux is good in
Networks: Linux is popular at ensuring all client most respects, it lacks some key features which are readily
computers on a LAN or on the Internet are provided with available in other operating systems.
adequate networking services. Custom tailored software is Not compatible with several Windows programs: Linux
readily available, which enables Linux to don different hats may be great in its own right; however, one major setback
including that of a file server, mail server, Web server, etc. with it is that it is not compatible with all programs. It lacks
Application compatibility and support: One of the key the equivalent to many popular Windows programs, due to
features of Linux is its compatibility with a wide array of which users opt for other operating systems.
application programming interfaces (APIs) and POSIX, which While open source technology has its perks, it is not
allow support for more varieties of freeware and shareware altogether better than other operating systems due to the
software. Almost every kind of GNU software is compatible limited support it receives.
with Linux, though some may require slight modifications. In conclusion, the Linux operating system continues to
evolve and widen its support for several architectures and
Advantages of Linux frameworks. Installation of Linux on your PC is a bit of a
Cost: Unlike other operating systems which come with a huge technical process but after you are done, you will be able to
price tag and monthly fees, Linux can be obtained for zero enjoy using it.
cost. Again, while other OSs like Microsoft only allow one
licence per system, it is the opposite with Linux, which allows By: Usman Hassan
distribution over series of computers.
The author is a digital marketing psychopath, who is passionate
Security and robustness: Since it was created, right up to about inbound marketing, SEO, CRO, email marketing and
the present day, the Linux OS has withstood malware, ad ware e-commerce. He is a writer at Mobiwoz.com
and virus vulnerabilities. The case is the complete opposite

Know the Leading Players ACCESS ELECTRONICS


in Every Sector of the B2B INDUSTRY WITH A
Electronics Industry
www.electronicsb2b.com
Log on to www.electronicsb2b.com and be in touch with the Electronics B2B Fraternity 24x7

74 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try For U & Me

GNU Linux Mint 17.3:


The Friendly Distro

Linux Mint strives to be an elegant, modern, easy-to-use Linux distro. Based on Ubuntu and
Debian, it tries to overcome the shortcomings of Ubuntu by packing in some proprietary software.
The author takes the reader through the process of installing Linux Mint and gives a first hand
account of how he has configured his own system.

S
ome time ago, an article titled “The free software at have a distro that's easy to install and to configure, and has
work" was published in Byte magazine (see Reference 1). almost all the software I need installed by default, in order to
In this four-page article, the author discusses GNU Linux have a completely functioning system in a short span of time.
and some applications, making some interesting comments In the past, I have mainly used GNOME and, sometimes,
mainly about their use in a research environment (at that KDE. I'm not a big fan of GNOME’s transition to version 3
time, the author was systems administrator in a university’s and KDE’s move to the Akonadi technology, because I like
physics department). The first figure, in the article, shows to have a desktop environment which is reasonably light,
GV (the PDF and PS viewer), LyX and the text editor Jed, fast and easy to back up; so my choice is Xfce. This article is
all under KDE. The second figure shows a table and a graph based on GNU Linux Mint 17.3 Xfce 32 bit ‘Rosa’.
made with StarOffice. Finally, the last figure shows Netscape The article presents some basic customisations of the
and a terminal window opened under the FVWM desktop desktop environment, and information about the installation
environment (I'm not too sure about this though, since the and configuration of a few daily-used applications, with the
article appeared many years ago). So I decided to give Linux help of the terminal window. Please note that the following
a chance, because the author was very optimistic, on the off are only examples because a lot of other ways are possible.
chance that what he’d written was indeed true.
The next step involved choosing a distro — the first Settings and systems managers
was Red Hat 5.0, and then SuSE from the version 5.3 (with Figure 1 shows the Settings manager with a yellow
Applixware Office 5.0) to the version 10.0. For the last few background, for the following options:
years, my preferred distro has been Mint, because I like to ƒ The Appearance option, as suggested by the name, is to

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 75


For U & Me Let’s Try
Settings

configure how the desktop looks. The Style tab controls Settings
Customize your desktop

the visual appearance of the controls such as the buttons Personal

and menus. The second tab, called Icons, is about how Appearance Desktop Desktop Settings File Manager Notifications

the icons are shown, for example, in the panel, on the Panel Preferred Applications Screensaver Window Manager
Window Manager

desktop, in the file manager and in the menus. Because I


Tweaks

think the default font size is too big, I need to use the third
Workspaces

Hardware

tab, called Fonts, to select a smaller size. A more detailed Display Keyboard Mouse and Touchpad Power Manager
Removable Drives and

explanation about this aspect can be found at http://docs.


Media

System

xfce.org/xfce/xfce4-settings/appearance Accessibility Driver Manager Firewall Configuration Input method Languages

ƒ I use the Background option only to change the default Login Window MIME Type Editor Session and Startup Software Manager Software Sources

background, and the Icons tab to select a smaller font


Update Manager

size and to select the default icons for the visualisation Other

on the desktop. Passwords and Keys Settings Editor

ƒ The Screensaver, as its name suggests, is about selecting a


screen saver. The option ‘Blank after' is to set the time, in
Help All Settings Close

minutes, after which the screen saver activates. Figure 1: Settings manager
ƒ The only thing that I did with the Window manager option
was to select a smaller font size for the window’s title.
More information about this can be found at http://docs.
xfce.org/xfce/xfwm4/preferences
ƒ The Firewall configuration starts Gufw, if previously
installed. This is a GUI for the Linux default firewall.
Simply, change the Status to ‘On’.
ƒ In the Login window, you can choose the theme for it. My
preference was ‘Elegance’, though you can also change its
default background, which is located in /usr/share/mdm/
themes/Elegance/background.jpg with another picture
having the same name.
ƒ Usually, it’s not necessary to use the Session and Startup
options. Sometimes, these can be used to uncheck some
applications that will be started automatically when
you log in to the desktop — for example, ‘Blueman’, Figure 2: LibreOffice language settings
‘MintUpdate’, ‘mintUpload’, ‘mintWelcome’, etc.
ƒ The installation of a printer is easy. On System devices such as MP3 players or mobile phones via a
> Printers simply click on ‘Add’ and follow the USB cable — see Reference 2).
instructions on the screen. The most important thing is ƒ Office group: qpdfview (a fast PDF reader),
that the recommended driver may not be the best. So libreoffice-style-tango (I’m not a big fan of the
some further tests may be needed. default human icons), hyphen-[your-language],
mythes-[your-language], myspell-[your-language] (the
Software manager default language is selected during the installation
In this section, I would like to explain the applications I added process and if ‘English’ is chosen, it may be necessary
after the default installation: to add one or more languages to LibreOffice; an
ƒ Accessories group: mc (Midnight Commander), vim-gtk. example is shown in Figure 2).
ƒ Graphics group: eog also known as the Eye of GNOME ƒ Settings group: gufw (see the previous section).
(because it’s not possible to print a picture from the ƒ Other packages: efax, fonts-inconsolata, fonts-sil-
default Ristretto application), Imagemagick (a nice gentium-basic, p7zip-full, ttf-mscorefonts, texlive-
application especially for batch operations) and Inkscape. full. Some information about 7-Zip and efax will be
ƒ Internet group: Skype. presented in the next two sections.
ƒ Multimedia group: Audacious (but the default Banshee
is also a nice player), k3b (probably the best CD burner 7-Zip
available on Linux), system settings (KDE system settings This software is very useful to zip (with encryption) one or
- by selecting Application appearance > Icons > Mint- more files. In the following example, an encrypted file called
X-Aqua you can have k3b with Xfce icons), mtp-tools, archive.zip containing two text files is created. The syntax
mtpfs, gmtp (gMTP is a player that can communicate with for the terminal is as follows:

76 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try For U & Me
Select file P=$(zenity --entry --title=’Password’ --text=’Enter
password:’)
astonfe Documents Projects Scilab latex
7za e -p$P $F
Places Name Size Modified

Recent
alkanes.pdf 52,5 kB 08/12/2014 I’m used to using only 7-Zip to zip, because the simplest
Home
alkanes.tex 2,2 kB 08/12/2014 way to unzip an archive is to use the default archive manager
Desktop
alkanes.xls 8,2 kB 08/12/2014 and also because, in the case of an encrypted archive, a
tetrachloroethene.pdf 33,7 kB 04/25/2014
Documents
prompt for the password is shown.
tetrachloroethene.tex 771 bytes 04/25/2014
Downloads

Music
tetrachloroethene.xls
xls2tex.sce
6,1 kB 04/25/2014
916 bytes 08/12/2014
Efax
Pictures
The first thing to know is which port your modem is
Videos
available on. In this example, an external USB modem is
Batch considered. From the terminal window, type lsusb and look
Devices
if the modem is recognised. It’s probably recognised on the
ttyACM0 port. Sometimes it may not be so easy and further
Computer
tests may be needed. According to Reference 5, some
Cancel OK privilege changes are required:

Password sudo chmod +s /usr/bin/fax


Enter password: sudo chmod +s /usr/bin/efax
qwerty
sudo chmod +s /usr/bin/efix

Cancel OK Then edit the configuration file with sudo gedit (or gvim)
/etc/efax.rc and change some options as follows:
Figure 3: Two examples of Zenity dialogue boxes
DEV=ttyACM0
7za a -mem=AES256 -pPASSWORD -tzip archive.zip file1.txt file2. FROM=”[your-faxnumber]”
txt NAME=”[your-name]”
PAGE=a4
To unzip, use 7za e archive.zip (the ‘e’ option stands
for ‘extract’), and to clear the terminal history, simply In the following example for the terminal window, a file
use history –cw. called filename.ps is sent to the number 0123456789:
Zenity is free software and a cross-platform program
that allows the execution of GTK+ dialogue boxes in the fax send -v 0123456789 filename.ps
command line and shell scripts (see References 3 and 4), and
in Mint, it’s installed by default. The following bash script can With Zenity, the previous operation can be done with the
be used to zip (with encryption) one file or one directory: following bash script:

#!/bin/bash #!/bin/bash
# File archive with AES encryption # Fax send
# F=$(zenity --file-selection --directory --title=’Select F=$(zenity --file-selection --title=’Select file’)
directory’) N=$(zenity --entry --title=’Fax number’ --text=’Enter the
F=$(zenity --file-selection --title=’Select file’) fax number:’)
P=$(zenity --entry --title=’Password’ --text=’Enter fax send -v $N $F
password:’)
7za a -mem=AES256 -p$P -tzip $F.zip $F I use Efax only to send a fax, so I haven’t tested it to
receive one. More sophisticated faxing solutions are achieved
An example is shown in Figure 3 — one dialogue box is with the Hylafax fax server and with the YajHFC client
about the file selection and the other for entering the encryption (Yet another Java HylaFAX client) as reported on http://
password. The following script is to unzip an archive: www.yajhfc.de, but some testing is required mainly for the
configuration of Hylafax.
#!/bin/bash Another example of Zenity’s usage is password
# File extract generation. In Mint, the APG (automated password
F=$(zenity --file-selection --title=’Select file’) generator) utility is installed by default, and can be

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 77


For U & Me Let’s Try
used directly in a terminal window or via a bash script Freeplane

as reported in the following code, which generates N Freeplane 1.3.15

passwords of M length: Application Details


Command /home/astonfe/freeplane-1.3.15/freeplane.sh
#!/bin/bash
Working Directory
# Password generator
M=$(zenity --entry --title='Password' --text='Minimum Scilab
password length:')
Scilab 5.5.2
N=$(zenity --entry --title='Password' --text='Generate n
passwords:') Application Details
Command /home/astonfe/scilab-5.5.2/bin/scilab
apg -a 1 -m $M -n $N
Working Directory

Freeplane, NetBeans and Scilab


jEdit
Because the software repositories are a bit outdated, it can be
interesting to download the latest version directly from the jEdit 5.3.1

developers’ website for some applications. Presented here are Application Details
three examples. Command java -jar /home/astonfe/jEdit/5.3.1/jedit.jar
Freeplane is an interesting Java based mind mapping Working Directory
software. Download the latest version freeplane_bin-1.3.15.zip
from https://sourceforge.net/projects/freeplane, unpack it in your Figure 4: From top to bottom, the launchers for Freeplane, Scilab and jEdit
home directory and then run it using /home/[your-username]/
freeplane-1.3.15/freeplane.sh. See the top part of Figure 4 But it’s possible to install jEdit 5.3.1 (a development
(Settings > Menu editor > Add launcher). The application icon is trunk). Download jedit5.3.1install.jar from http://www.
/home/[your-username]/freeplane-1.3.15/freeplane.svg. tellurianring.com/projects/jedit-daily and then install it
A Java Development Kit must be installed before from the terminal with java -jar jedit5.3.1install.jar. To run
NetBeans. An easy way is to install openjdk-7-jdk from the it, simply use java -jar /home/[your-username]/jEdit/5.3.1/
Software manager. The last version for Linux netbeans- jedit.jar (see Figure 4, bottom). The icon is /home/[your-
8.1-linux.sh can be downloaded from https://netbeans.org/ username]/jEdit/5.3.1/doc/jedit.png. If you have your
downloads. The installation involves the following two steps own syntax-highlighting file, you can add it in the modes
from the terminal: directory /home/[your-username]/.jedit/modes. In this case,
you must also add a row to the catalogue file, for example:
chmod +x netbeans-8.1-linux.sh
./netbeans-8.1-linux.sh <MODE NAME=”scilab” FILE=”scilab.xml” FILE_NAME_GLOB=”*.sce” />

During the installation process, be


careful to install the NetBeans IDE in /
home/[your-username]/netbeans-8.1
and the JDK location that’s /usr/lib/
jvm/java-7-openjdk-i386
The latest version of Scilab scilab-
5.5.2.bin.linux-i686.tar.gz can be
downloaded from http://www.scilab.
org, which you can then unpack to
your home directory. Last of all, run
Scilab using /omhe/[your-username]/
scilab-5.5.2/bin/scilab (see Figure 4,
middle). The application icon is /home/
[your-username]/scilab-5.5.2/share/
icons/hicolor/256x256/apps/scilab.png

jEdit, Emacs and Vim


It’s not possible to install the last
stable version jEdit 5.3.0 due to
issues with the preinstalled Java. Figure 5: From left to right, jEdit, Emacs and Vim

78 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try For U & Me
You can download the last stable release of GNU Emacs let b:loaded_lstlisting=”v1b”
emacs-24.5.tar.gz from http://ftp.gnu.org/gnu/emacs and then syn region texZone start=”\\begin{lstlisting}” end=”\\
unpack it to your home directory. Before installing Emacs, it’s end{lstlisting}\|%stopzone\>”
necessary to install the following software: syn region texZone start=”\\lstinputlisting” end=”{\s*[a-
zA-Z/.0-9_^]\+\s*}”
sudo apt-get install gtklp cups-pdf build-essential libx11-
dev xaw3dg-dev libjpeg-dev libpng12-dev libgif-dev libtiff4- Because the default configuration file is almost empty,
dev libncurses5-dev libxft-dev librsvg2-dev libmagickcore- edit gvimrc with sudo gedit (or gvim) /etc/vim/gvimrc and
dev libmagick++-dev libxml2-dev libgpm-dev libghc-gconf-dev paste it as a basic configuration, for example, as in the
libotf-dev libm17n-dev libgnutls-dev following lines:

The Emacs installation process involves the following set cryptmethod=blowfish


steps from the terminal window: set encoding=utf-8 “ The encoding displayed
set fileencoding=utf-8 “ The encoding written to file
cd /home/[your-username]/emacs-24.5 set foldmethod=marker
./configure set guifont=Inconsolata\ Medium\ 11
make set guioptions-=T
sudo make install set number
set textwidth=0
The two programs, gtklp and cups-pdf, are very useful to set wrapmargin=0
simplify the printing from Emacs by adding the following two source $VIMRUNTIME/mswin.vim
lines in your dotemacs file /home/[your-username]/.emacs: behave mswin

(setq lpr-command “gtklp” The last two lines are useful to make the cut, copy and
ps-lpr-command “gtklp”) paste commands work as in Windows with Ctrl+X, Ctrl+C
and Ctrl+V (see Reference 7).
This way, the printing process is managed by a GTK Installation and configuration of Linux is not as difficult
GUI in which you can print to a PDF file or to a printer. One as is sometimes reported. It is possible for GNU Linux to
more thing about the colour themes — as an alternative to the be the only system installed on your computer along with a
adoption of a specific colour theme, I prefer to change only wide range of great programs. Almost 20 years have passed
some colours by adding the following code to the dotemacs file: since the article cited in the introduction was published, and
during this time, developers have taken great steps forward.
(custom-set-faces For the future, I hope that LibreOffice will continue to
‘(font-lock-keyword-face ((t (:foreground have its present GUI. I also hope that Xfce will have more
“#0000CD”)))) ;; Medium blue customisation options.
‘(font-lock-type-face ((t (:foreground
“#DC143C”)))) ;; Crimson References
‘(font-lock-constant-face ((t (:foreground [1] Gonzato, Il free software al lavoro (The free software at
“#FF8C00”)))) ;; Dark orange work), Byte (Italian edition), 1998, 8, 96-99.
‘(font-lock-builtin-face ((t (:foreground [2] http://en.wikipedia.org/wiki/Media_Transfer_Protocol, last
“#1E90FF”)))) ;; Dodger blue visited on 26/02/2016.
[3] http://en.wikipedia.org/wiki/Zenity, last visited on
‘(font-lock-function-name-face ((t (:foreground 26/02/2016.
“#800080”)))) ;; Purple [4] http://help.gnome.org/users/zenity/stable, last visited on
‘(font-lock-string-face ((t (:foreground 26/02/2016.
“#808080”)))) ;; Gray [5] http://www.aldovarotto.org/computer/linux/fax.htm, last
visited on 26/02/2016.
‘(font-lock-comment-face ((t (:foreground [6] Arnote, Just the fax Ma’am... and it works in Linux!,
“#3CB371”))))) ;; Medium sea green PCLinuxOS Magazine, 2013, 72, 38-47.
[7] http://superuser.com/questions/10588, last visited on
About Vim, the directory for the colour themes is /usr/share/ 26/02/2016.
vim/vim74/colors and for the syntax-highlighting files is /usr/
share/vim/vim74/syntax. So, if you have some of them, you must By: Stefano Rizzi
copy your files to these directories. If you want to recognise the
The author works in the analytical chemistry and textile chemistry
lstlisting environment as in Vim 7.3, you can use the following areas. He has been a Linux user since 1998.
code in /usr/share/vim/vim74/syntax/tex/lstlisting.vim:

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 79


For U & Me Let’s Try

Cool FFmpeg Tricks


FFmpeg is a free software project that creates libraries and programs to handle multimedia
data. It includes libavcodec, an audio/video codec library. FFmpeg is a complete solution for
recording, converting and streaming audio and video files.

S
ometimes, fans of free software have to jump through would be a risk of a licence violation. Recently, FFmpeg
many hoops to get to the good stuff. FFmpeg is one released its Version 3 with a totally native AAC encoder, but
such example. Like the VLC media player, FFmpeg more work remains. The solution was for users to do their own
can handle almost any audio or video format you throw at it. compilation. The FFmpeg Wiki site has compilation steps for
Yet, most people have not even heard about it. Once you have several distributions. The compilation process does take a while
compiled and installed it, it will prove to be pretty useful. to finish, but it will eventually create the four binary executable
FFmpeg bills itself as a complete solution to record, files. Copy them to your /usr/bin directory. If you don’t want
convert and stream audio and video. The FFmpeg project to disturb the FFmpeg installation that came with your Linux
provides four simple apps - ffprobe, ffmpeg, ffplay and distribution, then you can access the binaries by providing their
ffserver. Many audio/video applications use it internally. If full path from a different directory.
not directly, then via mencoder, FFmpeg is used by lots of While this FFmpeg installation covers most formats,
media players, browser plugins, audio/video editors and other I found that it would not encode to the audio codec of the
multimedia software. This article will focus on the ffmpeg undocumented AMV format. AMV is used by many cheap
binary executable, which can be used to convert audio/video Chinese-made media players. These tiny devices are primarily
files from one format to another. FM and MP3 players with the video option added as a
‘please ignore it’ extra. The video resolution is a luxurious
Compiling FFmpeg 160x120! I bought one of these players without realising all
Because many audio/video codecs are encumbered by software this. Fortunately, there was a Google Code project called
patents, GNU/Linux distributions have had to omit FFmpeg amv-codec-tools that provided support for AMV using an
or bundle a less-capable FFmpeg version of it. Even if it were old version of FFmpeg. If you need to work with this format,
compiled with support for non-free encoders/decoders, there then build a custom FFmpeg binary, specially made for AMV,

80 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try For U & Me
using the following commands. Copy the amv-ffmpeg that The audio stream is set to LAME MP3 codec, a 44.1kHz
the following steps generate to the /usr/bin directory. sampling rate and a 96K bit rate. Here, mpeg4 and
libmp3lame are the codec names.
git clone https://github.com/vsubhash/amv-ffmpeg.git
cd amv-ffmpeg ffmpeg -i duffy.ogv \
./configure -vcodec mpeg4 -s 352x480 -r 30 -b:v 500k \
make -acodec libmp3lame -ar 44100 -b:a 96k \
duffy.mp4
So, what can FFmpeg do?
ƒ List supported codecs: The -codecs or -formats options ƒ Extract audio from a video file: You can use the -vn
will list all the codecs that are supported. Each codec option to disable the video stream and save the remaining
is referred to by a special name, which will have to be audio stream to an MP3 file, as follows:
used in your FFmpeg commands when a particular codec
needs to be used. The -encoders option will list all codecs ffmpeg -i duffy.ogv \
which FFmpeg can convert to, i.e., the output formats. The -vn \
-decoders option will list all codecs which FFmpeg can -acodec libmp3lame -ar 44100 -b:a 96k \
convert from, i.e., the input formats. duffy.mp3
ƒ Check codecs used in a media file: FFmpeg uses the -i
option to specify the input file. This will display the streams ƒ Help listen to YouTube and other online video
available in the file and the settings with which each stream sites: I use the youtube-dl program to ‘listen’ to YouTube
was encoded. Do remember that file extensions can be videos. Many TV programmes, such as the Keiser Report,
misleading. MP4, AVI or WMV files are merely containers upload their shows to YouTube so that viewers who miss
for multiple content streams. There is no guarantee that a the broadcast schedule can catch up later. With an audio
WMV file will have a stream in Windows Media Audio file, you can continue to work while you listen. To install
format. It could very well have MPEG-3 audio. youtube-dl, use the command sudo apt-get install
youtube-dl. The first youtube-dl command here uses the -F
ffmpeg -i duffy.ogv option (uppercase F) to list all formats in which a particular
video is available. The second youtube-dl command uses
ƒ Convert a file from one format to another: Linux the -f (lowercase F) to download the video in a particular
programs typically create OGV/OGG files. However, very format identified by a number obtained from the previous
few consumer electronic devices support these formats. command. Later, FFmpeg is used to convert the audio to
You can use FFmpeg to convert to a device-compatible OGG format so that it can be played with the Audacious
format, as it can guess the output format based on the file player. Audacious has an alternative Winamp interface,
extension you have used. which can be minimised to a teeny-weeny but entirely
functional floating toolbar. If you would rather view the
ffmpeg -i duffy.ogv duffy.mp4 video, then you can try the YouTube downloader JavaScript
given in my article carried in the February 2016 issue of
ƒ Convert a file to AMV format: First you have to OSFY. You can choose that option if your Net connection
convert the video to WMV formate. Then, you can use the does not support streaming videos very well.
AMV FFMPEG binary.
# list the available formats
ffmpeg -i duffy.ogv duffy.wmv youtube-dl -F \
https://www.youtube.com/watch?v=_juHR5uodjU
amv-ffmpeg -i duffy.wmv -f amv \
-s 160x120 -r 16 -ac 1 -ar 22050 -qmin 3 -qmax 3 \
tank.amv # download the video in M4A audio format
youtube-dl -f 140 \
ƒ Explicitly specify conversion settings: When you let https://www.youtube.com/watch?v=_juHR5uodjU
FFmpeg guess the output format and do the conversion,
it will choose its own default codec settings. This may ffmpeg -i *.m4a play.ogg
not always be what you want. Sometimes, you may audacious play.ogg & # return to prompt
have very different requirements. In this example, the
container format is MP4. The video stream is set to the ƒ Fixe errors: recordMyDesktop is a great screen-
MPEG4 codec, 30 frames per second and a 500K bit rate. capturing program but it sometimes creates video files

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 81


For U & Me Let’s Try
ƒ Append a video to another video: FFmpeg offers several
methods for merging. The simplest is to use input videos
that use the same codec and, preferably, the same resolution
and frame rate. The following commands first create a text
file containing the path names of the input files. The FFmpeg
commands merge the files specified in the text file.

echo “file ‘tank-cut-recoded.mp4’” > file-list.txt


echo “file ‘tank.mp4’” >> file-list.txt
ffmpeg -f concat -i file-list.txt -c copy tank-mix.mp4

ƒ Perform two-pass encoding: With two-pass encoding, you


can get a higher quality video file with fewer artifacts. The data
obtained from the first pass is recorded in a log file and then
Figure 1: Add right-click menu options for media files using Caja Actions used in the second pass for a more efficient conversion.
and FFmpeg bash scripts
ffmpeg -y -i duffy.ogv -f mp4 \
with incorrect key frames. These errors can make -pass 1 -passlogfile duffy-log-file \
conversions or editing difficult or impossible. FFmpeg can -vcodec mpeg4 -s 320x240 -r 25 -b:v 512K \
fix these errors if you instruct it to simply copy the audio -an /dev/null
and video streams to a new file.
ffmpeg -i duffy.ogv -f mp4 \
ffmpeg -i duffy.ogv \ -pass 2 -passlogfile duffy-log-file \
-c:v copy \ -vcodec mpeg4 -s 320x240 -r 25 -b:v 512K \
-c:a copy \ -acodec libmp3lame -ar 44100 -b:a 96K \
duffy-fixed.ogv duffy.mp4

ƒ Rotate video: Sometimes, the videos that people take Klaatu (of the GNU World Order podcast) once provided a
from a mobile phone have been rotated 90 or 180 degrees. great video-editing tip. First, down-sample (lower the bit rate
You can fix this by specifying a transpose filter, as with the same resolution) the input video and create your video
follows. The transpose value is in multiples of 90. edits. When the video editing project file is finalised, replace
the down-sampled video file with the original, bigger video file.
ffmpeg -i tank.mp4 \ This way, the video editing process and your test renderings are
-vf “transpose=1” \ faster. The final rendering can take hours to finish, but you do
tank-rotated.mp4 not waste much time waiting before that. You can use FFmpeg
to down-sample the video.
ƒ Cut a part of a video: To extract a part of a video, you
need to specify the start time using the -ss option, and the FFmpeg automation
duration using the -t option (use the hh:mm:ss format). If If you have Caja Actions or Nautilus Actions Configuration
you don’t want the file to be re-encoded, then ‘copy’ the installed, then you can simply right-click media files to convert
streams, as shown below: them. You will have to first write some BASH scripts that
will automate the FFmpeg conversions. Nautilus Actions
# start from 1 min 12 sec and go for 2 min 50 sec Configuration (GNOME 2/3) is available in most software
ffmpeg -i tank.mp4 \ repositories. My March 2016 article has steps for compiling
-ss 1:12 \ Caja Actions (Mate).
-t 2:50 \
tank-cut-recoded.mp4 References
[1] http://ffmpeg.org/ “FFmpeg.org”
[2] http://trac.ffmpeg.org/wiki/CompilationGuide “FFmpeg
ffmpeg -i tank.mp4 \
compilation guide”
-c:v copy \
-c:a copy \
-ss 1:12 \ By: V. Subhash
-t 2:50 \ The author is a writer, programmer and FOSS fan (www.vsubhash.com).
tank-cut.mp4

82 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Insight For U & Me

Getting Familiar with GRUB


Everything you ever wanted to know about GRUB, but were afraid to ask, is
incorporated in this very informative article.

W
henever a make it cleaner, safer
computer is and more powerful than the
switched on, the boot previous version, which was known as
loader is the first software program to run on it. Preliminary Universal Programming Architecture
The boot loader is responsible for the loading of the kernel (PUPA for GNU GRUB). PUPA was eventually renamed
software, which in turn loads the operating system. GNU GRUB 2, and the original GRUB was named GRUB
GRUB is a multi-boot boot loader and is one of the most Legacy. The last stable release for GRUB Legacy came
used. Most of the popular operating systems have GNU out in 2005.
GRUB because of its user-friendly design and support for GRUB 2 was widely accepted by the computer science
multiple boots. community. Most of the Linux distributions started using
GRUB 2 and by the end of 2009, multiple major distributions
History of GRUB were installing it by default.
It all started in 1995, when Erich Boleyn was trying to
boot the GNU Hurd with the GNU Mach (earlier known The booting process
as the University of Utah’s Mach 4 microkernel). The Ever thought about what really happens when you switch on
aim of developing GRUB was mainly to offer support for a computer and how the operating system is loaded? Booting
compatibility on different machines and operating systems. can be defined as a series of processes that help to start a
The idea of modifying the FreeBSD boot loader was dropped computer and put it into a state of readiness for any operation
when Boleyn realised that it was easier to develop a boot by the user. Booting is said to be complete when the normal,
loader rather than modify the current one. GRUB had a lot of operative, runtime environment is attained. Booting of a
features added by Boleyn, which makes it stand out from all computer can be explained in several phases.
the existing boot loaders. The BIOS phase: BIOS (Basic Input Output System)
Later, in 1999, GRUB was adopted as an official GNU is a firmware interface. The BIOS software is stored on a
package by Gordon Matzigkeit and Yoshinori K. Okuji. The ROM chip on the motherboard and is the first software run
latest sources were made available through Anonymous CVS by a PC when powered on. It carries out the process called
for the development of GRUB. POST (Power on Self-Test) which checks, identifies and
In later years, GRUB was further developed and more initialises the hardware to find that every device has the
features were added. But it soon became obvious that the right amount of power, and performs a memory test to see
design of GRUB was incompatible with the new features that whether the memory is corrupted. The BIOS cannot load
were planned for it. the kernel directly because it is unable to access enough of
Around 2002, Okuji started rewriting GRUB to the large hard disk space to load the kernel code. Instead,

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 83


For U & Me Insight
BIOS detects and executes the boot loader software located
in the boot device, which is capable of loading the kernel,
thus giving control to the PC.
The boot loader phase: The completion of self-tests
initialises the master boot record or MBR, located on the
first sector of the bootable disk, which is very small. GRUB
is responsible for the execution of the kernel and the initrd
images in the file system. The MBR executes the GRUB
boot loader, which allows us to choose between the multiple
kernel images installed on the machine. GRUB then loads
the user-selected kernel and passes control on to it. The
GRUB entry and default entry time can be changed by
editing the grub.cfg file or using any third party application.
The kernel phase: Once the control is passed to the Figure 1: Normal GRUB
kernel, it acts as an intermediate layer or mediator between
the software and hardware of a computer. The kernel, being
one of the most fundamental components of the operating
system, handles all system processes such as I/O request List configuration General settings Apperance settings

management, task scheduling, memory management and Ubuntu


menuentry / script : linux
Removed items
Windows UEFI bootmgfw.efi

overall system control. The first process that is executed by Advanced options for Ubuntu
submenu
menuentry/script:custom
Windows Boot UEFI loader

the kernel is the init process. Windows Boot Manager (on /dev/sda2)
menuentry / script: os-prober
menuentry/script:custom
EFI/ubuntu/MokManager.efi

Operating system: The init process’s first job is to check System setup
menuentry / script: uefi-firmware
menuentry/script:custom

the system’s properties, the file system and hardware to


make sure these are running fine. Init executes each run-level Figure 2: Grub Customizer - list configuration
one after the other and the corresponding scripts. Based on
the run-level, scripts are executed to start or stop various
processes to make sure the system is functional. Once the init
process completes all the tasks, the control is given back to
the kernel.

Installing GRUB in Ubuntu


Although GRUB is automatically incorporated into a normal
Linux installation, there are times when we need to install it in
various storage devices as per the user’s convenience during
multiple booting. If users want to install Linux in a specific
partition, they are supposed to specify the drive on which to
embed the boot loader information.
Entire drive: While installing Ubuntu using the ‘Erase
disk and install Ubuntu’ option, the user chooses where Figure 3: Grub Customizer - general settings
Ubuntu and GRUB will be installed, automatically.
Something else: In this method, users are given the extra
option to install the boot loader into any device that they
choose. This is needed when there is an OS already installed
in the system or is going to be installed. The disk where BIOS
is set up to boot or the partition where Linux is installed is
generally selected in this case.
Making a GRUB bootable CD-ROM: When booting
becomes difficult due to some compatibility issues,
a GRUB bootable CD-ROM can be created. So there
is no need to create a floppy or hard disk image file.
Booting from a CD-ROM involves using a special image
file, cdboot.img, which is included in the core.img file.
Also, grub.cfg, the main configuration file, should be
incorporated with the other image files. Figure 4: Grub Customizer - appearance settings

84 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Insight For U & Me

Boot Repair _

Repair the boot of the computer

Recommended repair
(repairs most frequent problems)

Create a Boot Info summary


(to get help by email or forum)

Advanced options

About Quit Figure 8: Boot-Repair - GRUB options

Figure 5: Boot-Repair - Recommended repair

Figure 9: Boot-Repair - other options

Figure 6: Boot-Repair - advanced options # mkdir –p bootable/boot/grub


-p – By default grub looks in /boot/grub directory file, to
change this –p command is used

To include the GRUB configuration file grub.cfg inside


the GRUB directory, create the bootable image, as follows:

# grub-mkrescue –o bootable.iso bootable

This creates a bootable image named bootable.iso, which


can be burned into a CD-ROM or creates a bootable USB
device using the image.

Figure 7: Boot-Repair - GRUB location Customising GRUB with Grub-customizer


After installing GRUB, some users may not like the default
You can use the grub-mkrescue program to create an ISO GRUB, and may want to edit it. There are several ways to
image file, as follows: do so. One of the methods is to manually edit it, which is
a bit risky since any mistake would possibly result in an
# grub-mkrescue -o grub.iso unbootable system. Or, you could install Grub-customizer
-o – To set the output file which is open source software that enables you to edit
GRUB in an easy manner and reduces the risks inherent with
You can create a directory that is to be included in the ISO manually editing the configuration file. Grub-customizer was
image file, as follows: developed by Daniel Richter, and it provides a graphical user
interface to configure the GRUB2/BURG settings and menu
# mkdir bootable entries. It is published under the terms of GNU GPL v3.
Grub-customizer essentially does the following:
A directory for GRUB in the top directory bootable can be ƒ Edits the menu entries (renames, reorders,
created as follows: adds or removes)

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 85


For U & Me Insight
Table 1: A comparison of boot loaders
ƒ Edits the contents of the menu entries or creates new ones
ƒ Changes menu visibility and timeout Boot Loader Can be in-
stalled via
Can boot Supported file system
from CD
License

ƒ Changes the default boot entry ROM

Linux or Yes EXT2/3/4, GPLv3


ƒ Changes the GRUB resolution or background image GNU GRUB
Bootable disk NTFS,FAT16/32
ƒ Disables recovery entries
LILO Linux Yes No specific file system BSD License
ƒ Provides support for GRUB2/BURG
ƒ Changes the kernel parameters, etc BURG Linux Yes - GNU GPLv3

To install Grub-customizer in Ubuntu and Mint, launch a rEFind Linux Yes EXT2/4, NTFS GPLv3/BSD License
terminal and type the following commands:
SYSLINUX Linux Yes - GNU GPLv2 or Later

sudo add-apt-repository ppa:danielrichter2007/grub-customizer


sudo apt-get update Using Boot-repair
sudo apt-get install grub-customizer Boot-repair offers the following options.
Recommended repair: Launch Boot-repair either from the
Working with Boot-repair Applications menu or from the terminal. The Recommended
Ever been stuck with an unbootable GRUB? Or been unable repair option allows you to solve the frequently occurring
to access your multiple operating systems from the GRUB boot up issues. It reinstalls and repairs your default GRUB.
menu? That’s when Boot-repair comes in handy. It is free Advanced options: This option lets you delve more into
software that is licensed under GNU GPL, and provides a GRUB and allows you to customise it the way you like.
graphical interface to repair frequent boot problems. Boot- Advanced options include:
repair allows the repair or recovery of your boot loader, gains ƒ Re-installing GRUB and restoring MBR
access to the unbootable operating system and fixes issues ƒ Backing up partition tables and boot sectors
when you install Windows after installing Linux. Boot-repair ƒ Repairing file system and Windows boot files
can be used either from a live CD or a Boot-repair disk, which ƒ Changing the GRUB location
automatically runs it at start-up. You can also install Boot- ƒ Purging GRUB
repair in your system if you have a bootable Linux. ƒ Creating a boot information summary
Table 1 above gives a comparison of the different
Installing Boot-repair boot loaders.
There are two ways of installing Boot-repair.
By using the Boot-repair disk: The Boot-repair ISO image
References
can be downloaded and burnt to a CD, or you can make
[1] http://en.wikipedia.org/wiki/GNU_GRUB
it bootable on a live-USB using any of the USB bootable
[2] http://en.wikipedia.org/wiki/Comparison_of_boot_loaders
software available. [3] https://wiki.archlinux.org/index.php/Boot_loaders
Installing in Ubuntu: Open a terminal and type the [4] https://www.gnu.org/software/grub/manual/grub.html
following commands:
By: Vishnu N.K.
sudo add-apt-repository ppa:yannubuntu/boot-repair The author, who is an open source enthusiast, is in the midst
sudo apt-get update of his B. Tech degree in computer science at Amrita Vishwa
Vidyapeetham. You can contact him at mails2vichu@gmail.com
sudo apt-get install -y boot-repair && boot-repair

ELECTRONICS

• Exclusive: ELCINA’s first greenfield EMC project soon to get land possession
• India is emerging as an attractive destination for international companies to invest
• Electronics manufacturing industry in Pune needs a boost
• In 5 years, India may rank 2nd in manufacturing competitiveness, worldwide
INDUSTRY IS AT A
• Highlights of state electronics policies
• National Policy on Electronics will Boost R & D in Electronic Manufacturing
• Insecure market threatens solar module makers

Log on to www.electronicsb2b.com and be in touch with the Electronics B2B Fraternity 24x7

86 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try For U & Me

How to Run Android


Without Google
You can liberate your Android device from the clutches of Google and take charge of it completely!
Remember the old adage that nothing comes for free? Well, although Android is mostly free, there is
plenty of Google’s bits and the manufacturer’s own software locked up in it. You can almost free yourself
and your device from Google by carefully reading this article and implementing its suggestions.

I
f you are dependent on Google apps or must also be easy to imagine that each Android
if you need to regularly pull email and smartphone is costing at least one polar bear its icy
social networking status updates, then home. If not, then try to guilt-trip your friends with
this article is not for you. However, if that idea... just for the fun of it.
you would like to benefit from improved Apart from general Google apps settings,
battery life, the satisfaction of causing there are other settings within each Google app
less global warming and maybe some that help to keep tabs on you. For example, the
more peace of mind, well then... read on. Chrome browser will automatically transmit
The Android Open Source Project each Web address that you visit to Google
(AOSP) is indeed open source, as it is servers, of course, for malware and fraud
built on the GNU/Linux kernel and the prevention. Well, I built my own browser. You
code is freely available online. However, can, however, use the Firefox for Android app, as
an Android device has plenty of Google bits it is totally free and open source software (FOSS).
that are closed source and deeply integrated
into the OS. If an Android device maker Change the app status
wants to remove the proprietary Google The app settings screen has several
bits, that firm can no longer be part of the columns - Downloaded, Running,
Android Alliance. (Amazon and Yandex All and Disabled. Many Google apps
have already done this but they added their as well as manufacturer bloatware
own closed source bits.) For an ordinary hide in the All column. Take some time
phone user though, it will be quite a greasy pole. to study or research these apps. If you know
But, let us give it a try anyway. what you are doing, you can stop and disable many of them
permanently. Be sure not to disable important system apps or
Change Google settings anything you might need.
Most people breeze through the initial Google settings
without waiting to read what is shown on the screen. Why? Root to remove unwanted apps
What could go wrong? Essentially, Google says that it will Most device manufacturers install online shopping and social
save your Wi-Fi network password on its servers and prevent networking apps as system apps. They start automatically,
your phone from ever going to sleep again, ever — whether run all the time and waste your battery charge and data
you are using the phone or not! connection. In India, they are less afraid of lawsuits and
Microsoft has made it impossible to turn off data cause data usage whether you are on Wi-Fi or mobile data.
collection in Windows 10, but it does not have to worry Such system apps cannot be disabled. You can get rid of them
about devices (PCs mostly) being connected to mobile only by rooting the device. But since rooting voids warranty,
data. Android devices have a multitude of connectivity remember that it is an option only for the brave.
options. Fear of class action suits has prevented Google from If the device cannot be rooted, then use a task killer or a
performing data collection with mobile data. So, Google startup manager app to disable unwanted apps regularly, on-
provides options to turn these settings off. Well then, turn demand or at startup.
them off. Google has redefined Airplane Mode. It turns the
mobile data (telco) off but Wi-Fi can be turned on. Use alternative apps and app stores
If you believe in man-made climate change, then it Some Google apps may be decidedly rogue, such as Google

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 87


For U & Me Let’s Try

Apps Apps

Running All Disabled All Disabled

Advanced Task Killer Free Amazon Shopping


1.11MB 29.95MB Disabled

Android Keyboard (AOSP) C26Essentials


Google services 2.59MB 3.06MB Disabled
Keep Wi-Fi on during sleep
Android Live Wallpapers Chrome
These service put Google to work for Always 124KB 43.32MB Disabled
your, and you can turn them on or off at Android System Drive
any time. Data will be used in Only when pluged in 0.00B 20.68MB Disabled
accordance with Google’s Privacy
Policy. Never Android System WebView Gmail
1.75MB 12.50MB Disabled
CANCEL
Back up your phone’s apps, Android Work Assistant Google Account Manager
app data, settings, and Wi-Fi 44.00KB 3.99MB Disabled
Passwords using your Google
Angry Birds Google Backup Transport
Account so you can easily restore
42.49MB 784KB Disabled
later.Learn more
Atci_service Google Calendar Sync
148KB 4.01MB Disabled

Atomic Bomber Google Contacts Sync


148KB
Figure 1: In a smartphone, ‘off’ does not mean really ‘off’
Figure 3: Stop and disable unwanted apps
Google services Google services
Use Google’s location service
minor difference is that their pipes
These service put Google to work to help apps determine location.
AVAILABLE INSTALLED UPDATES (4)
are not connected to Google data
for you and you can turn them This means sending anonymous
on or off at any time. Data will be location dat to Google, even Games
centres.
used in accordance with Goolge’s when no apps are running.
Privacy Policy. Learn more Bomber 1.0

Side-load apps
Single player arcade game GPLv3

BroserQuest 1.0.1
This phone may also receive Help improve location services Role-playing game You will have to enable the
MPL2

and install updates from Google. by letting apps and service scan
By continuing, you agree to the for Wi-Fi networks even when Bushido Blocks
Block-matching game
Unknown sources option in the
1.4
GPLv3
Wi-Fi is off.
Google terms of Service and
Privacy Policy.
settings for side-loading apps.
Candy Memory 1.4
Match-two memory game
Whenever I have to configure a
GPLv2

Use Google’s location service Help improve your Android


to help apps determine location experience by automatically Card Game Scores
Track game score
1.1 new device, I install all the apps
GPLv3+
sending diagnostic and usage
This means sending anonymous
location data to Google, even data to Google. this information Catan Dice Game 1.12
(mostly games) that I need from
when no apps are running. won’t be used to identify you and
lends a hand to teams working
Resource dice game
Google Play before disabling
GPLv3

Learn more
on things like battery life, app Chess 8.4.3
it and all its associated apps.
Chess program and FICS client MIT

performance. and new Google


Help improve location services
features. Learn more I then use Total Commander
Figure 4: The F-Droid app is a (a file manager app) to copy
FOSS alternative to Google Play APK installer files of the apps
Figure 2: To enable data collection, Google apps have almost to an SD card backup store.
complete control on your device If ever I need to install those apps again, I can get them
from the backup.
Photos app, which lets Google Settings upload your photos There are many sites that provide free downloads of
even after the app is uninstalled. You do not need the Gmail APK files. Installing apps from these sources is fraught
app to get email. Use Gmail’s Web interface to enable POP with danger. AndroidPolice.com maintains a site at www.
or IMAP access in the settings. Then, configure Gmail POP/ apkmirror.com where the latest APK builds are uploaded
IMAP access in the stock email app and use it to send/ for testing. XDA Forums is a site famous for its Android
receive email. Other providers such as Microsoft (Outlook) developer community. Many APK files that are not available
and Yahoo also provide free POP/IMAP access. on app stores can be downloaded from here. Always read the
In the February 2016 edition of OSFY, Magimai Prakash reviews before installing any APK.
has provided an introduction to F-Droid. This app is a good Racoon is a Google Play desktop client. It is a Java
alternative to Google Play. F-Droid provides downloads of software that you can run on Linux, Windows or Mac
FOSS apps. It has plenty of apps for every purpose. computers. It does not break Google’s ToC. You have to log in
App stores from Amazon, Yandex and Opera are to a valid Google account to download the APK files, which
alternatives to Google Play. Unfortunately, the Amazon store can then be manually installed on your phone. There is no
app is even more resource-heavy than Google Play. These need to have Google Play on the device.
store apps have their own data collection methods. The only In my opinion, F-Droid is the best alternative to all the rest.

88 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Let’s Try For U & Me
site offers RSS feeds, use the RSS feed instead of its app.
Browse anonymously.
ƒ Turn off services such as mobile data, Wi-Fi and GPS
when they are not required.
ƒ Turn off your phone when you are travelling because
it will waste more energy trying to connect to
transient cell towers.
ƒ Enable the Ethernet networking option in Android tablets.
Use a USB LAN adapter with your OTG cable so that
you can use a wired connection. Yes, use a LAN cable.
Disable Wi-Fi in your router and enable Airplane Mode in
the tablet. This option is not available on Android phones
as, of course, they do not have Ethernet drivers.
Finally, a security tip: Do not let your phone connect to any
free Wi-Fi access point. Wi-Fi routers get infected with viruses
Figure 5: Ethernet networking in Android and are now a favourite hacking method. Botnets work because
of compromised routers. You could instead rely on a mobile
Some more battery-saving tips data plan and use it only when necessary.
Apart from disabling unwanted apps and services, there are
other things you can do to save on battery life. By: V. Subhash
ƒ Optimise device and app settings.
The author is a writer, programmer and FOSS fan
ƒ Minimise the number of apps you need to have. One RSS (www.vsubhash.com).
feed reader app can eliminate several dedicated apps. If a

Benefit from
PREFERENTIAL MARKET ACCESS POLICY
ELECTRONICS MANUFACTURING
CLUSTER POLICY
MODIFIED SPECIAL INCENTIVE PACKAGE
SCHEME (M-SIPS)
How?
Get Latest Updates, Policy Guidelines,
Analysis, Industry Views

Log on to
www.electronicsb2b.com

Subscribe to Newsletter

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 89


Admin Overview

An Introduction to Riak S2
Riak S2, also known as Riak CS, provides powerful storage for large volumes of data.
The important features of Riak are high availability, fault tolerance, security, as well as
easy accessibility and installation.

C
loud storage is storage of data in digital form. Its easy access, cost benefits, and robustness in order to utilise the
structure consists of multiple servers that are located data to gain business advantages. RiaK CS is one such solution.
at different locations. This physical structure of We can access the Riak S2 admin interface via HTTP.
storage is governed by the hosting company that provides the RiaK CS is compatible with any client or library that
cloud services. Organisations have to access their data stored implements the S3 API such as s3cmd, s3curl, boto, erlcloud,
in the cloud, through the Internet, independent of where they the AWS Java SDK, the AWS Ruby SDK, and Fog.
are located.
Features of Riak S2
An overview of Riak S2 Riak S2 or Riak CS provides the flexibility to install and
Riak is a NoSQL database which is designed to deliver the configure the entire set-up on premise and, hence, it is secured
maximum amount of data by distributing it across multiple based on an organisation’s defined policy. Figure 3 shows a
users. It is also known as a distributed database because of few important features of Riak CS.
its capability to deliver across distributed networks. It offers
various features such as fault tolerance, high availability, Easy
accessibility Large object
indenpendent support and
scalability and simplicity. It is available in open source, of locations multi-part
upload

enterprise and cloud storage versions. Low cost


Data
backup
Masterless
Robust stats
Riak S2, also known as Riak Cloud Storage (Riak CS), is Benefits
for monitoring
and metrics RIAK
with built-in
replication
of Cloud
built on top of Riak. It is an object storage system, which stores storage
S2OPEN
SOURCE

objects in Riak and provides compatibility for an S3 interface. Easy


recovery of
data
Easy
deployment

In this era of ubiquitous social media and multiple sources Multi-tenancy


and per-user
S3-compatible
api and
reporting
of data, we now have to manage unstructured data along with
Reliability authentication

structured data. Thus, to manage such a huge volume of data,


we need to have scalability, high availability, fault tolerance, Figure 1: Benefits of cloud storage Figure 2: Riak S2, open source

90 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Overview Admin

Large associated with it.


Object
Oprn source c. Meta data is used for retrieval
storage software Easy to Setup 1 Client upload file
to Riak CS API
S3 Reporting S3 Reporting S3 Reporting S3 Reporting
purposes.
S3 Reporting
API API API API API

d. Data is replicated across nodes.


API API API API API

Riak CS
Node
Riak CS
Node
Riak CS
Node
Riak CS
Node
Riak CS
Node 2 files into 1 MB chunks e. Nodes automatically take
Riak CS breaks large

Control over stor-


age design and
Cost effective
Chunk Chunk
over the responsibilities of nodes
configuration with Chunk 3 Riak CS streams
Chunk
chunks to Riak nodes
Security Chunk Chunk that go offline.
Riak
f. Hence, data remains available
Node

Support for HA-FT (High even in failure.


Riak Riak
availablity and
Multipart Upload
Fault-Tolerant)
Node Node
4 Riak Nodes replicate
and store the chunks
3. Any node is capable of
Riak
responding to client requests.
Riak
Node Node
4. There’s no master node
Riak S2 API is
Multi-tenancy and
architecture here, and each node
compatible with the Figure 6: Architecture (source: http://docs.basho.com/
Amazon S3 API
per-usedr reporitng has the same responsibilities.
riakcs/latest/tutorials/fast-track/What-is-Riak-CS/)
5. Three replicas per object
Storage to power
Use cases of Riak S2 (data) are created, by default.
Monitoring and applications and The use cases of Riak S2 are illustrated 6. For failed or non-communicative
metrics services
in Figure 4. nodes, other nodes automatically take
over the responsibility to make data
Figure 3: Features of Riak S2 How Riak S2 works available and, hence, high availability
1. Riak S2 is built on the Riak NoSQL and fault tolerance is achieved.
On demand storage
database.
To store different
types of data or big
data such as text
capacity for internal
business units 2. Try to upload a database to Riak S2. Installation of Riak S2
via Private Cloud
files, images, videos,
other objects deployment Model This is what happens: Riak S2 or Riak CS is not supported on
a. Riak S2 breaks the object into Microsoft Windows operating systems.
Backup and smaller chunks or blocks so However, it can be installed on operating
Disaster manage- Building public or
ment scenarios for
maintaining High
private clouds, or
as relible storage
they can be streamed, stored and systems such as CentOS, Ubuntu, FreeBSD,
for applications and
Availability
platforms replicated in the underlying Riak Fedora, SmartOS, Solaris and OS X. It can
cluster. be installed on a single node for environment
Figure 4: Use cases of Riak S2 b. Each block has meta data development purposes.

References
[1] https://github.com/basho/riak_cs
[2] http://docs.basho.com/riakcs/latest/tutorials/fast-track/
Stanchion — Riak— The [3] http://docs.basho.com/riakcs/latest/
distributed
An application
used to man- database on [4] http://basho.com/products/
age globally
unique entities
top of which
Riak CS is [5] http://basho.com/products/riak-s2/
built
[6] http://searchcloudstorage.techtarget.com/definition/cloud-storage
[7] http://docs.basho.com/riakcs/latest/cookbooks/installing/Installing-Riak-CS/

Riak S2 or Riak CS itself By: Priyanka Agashe and Mitesh Soni


Priyanka is a senior software engineer. She loves to read books and her professional
interests include testing, Oracle apps, electronics and robotics.
Mitesh is a technical specialist. He loves to write about new technologies. Blog:
http://www.discovertechno.com/
Figure 5: Installation of Riak S2

Read more stories on TOPCOMPONENTS STORIES


• The latest in power conv
erters
trollers
ELECTRONICS

• The latest in microcon


Components in • Growth of Indian electroni
• India continues to impo
c components industry
rt significant amount of
INDUSTRY IS AT A
www.electronicsb2b.com
components every year
FETs and relays
• The latest in IGBTs, MOS
ds available in India
• Top 12 connector bran

Log on to www.electronicsb2b.com and be in touch with the Electronics B2B Fraternity 24x7

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 91


For U & Me Overview

An Overview of Open Source Databases


The ever increasing volumes of data, which are a by-product of modern life, require efficient
methods for their management and retrieval. Open source databases have today come into their
own, and stand out as solutions for every data management need in the enterprise. Here is an
overview of five of the most widely used and highly ranked open source databases.

T
he past few years in the data management industry MySQL
have been revolutionary for open source databases. From the most trending, let’s move to one that has been the
According to reports from DB-Engines, an online most widely used for years now — MySQL. Named after
initiative to collect and present information on its co-founder Michael Widenius’ daughter, My, it
database management systems (DBMS), the is a dominant force in the world of RDBMSs.
three fastest growing databases of 2014 MySQL is a popular choice for Web
were all open source, and the trend applications, especially among PHP
continued through 2015 as well. developers. Even among the strong
In this article, I will try and competition from other RDBMS
provide an overview of some of the databases, including the ones forked
most common open source DBMS from it, MySQL made big gains
software. Obviously, the list is so in early 2016 to top the RDBMS
huge that it would be impossible to popularity chart after a few years.
mention all of them, but I will try MySQL is available in two
and list those with the best features. editions:
I have picked five of the most widely 1. The open source
used and highest ranked databases for MySQL community edition
discussion in this article. 2. The MySQL enterprise edition
So let’s get to work and start talking about While the basic engine remains the same in
them in no particular order of ranking. both editions, the enterprise edition scores on support
and more frequent updates.
MongoDB Wide adoption of MySQL means that it does provide a
MongoDB has topped the charts for two consecutive years variety of features that excite people, though there are strong
(2013-14) and has been the runner-up in 2015. competitors. Given below is the list of a few such features
MongoDB is among the several databases that arrived and the rest is left for readers to explore:
in the mid-2000s under the NoSQL category. It broke the 1. Like most other database systems, MySQL is a relational
traditional method of storing data in tables and rows by DB system.
storing it in JSON like structures with dynamic schemas, 2. Like any other successful database system, MySQL does
which was later termed as BSON by MongoDB. provide scalability, flexibility, high performance and high
The creators of MongoDB say that the name was derived availability.
from the word ‘humongous’ to support large and massive 3. MySQL claims to offer the most powerful transactional
amounts of data. DB software went commercial in 2007 and database engine to provide robust transaction support.
then open source in 2009. 4. Strong data protection with some exceptional security
Written in C++, MongoDB breaks out from traditional features ensures absolute data protection.
RDBMS ideology and hosts a vast set of features. Here is the 5. It has comprehensive support for every application
list of some of them: development need.
1. Stores records in the form of documents, and corresponds And the list goes on and on.
to data types in most programming languages
2. Embedded documents and arrays reduce the need for PostgreSQL
expensive joins Originating from the Ingres project at Berkeley University in
3. Dynamic schema supports fluent polymorphism the 1980s, PostgreSQL underwent a series of improvements,
4. Full index support on all fields initiated by several of its founders to finally be named
5. Replication and high availability across LANs and WANs PostgreSQL in 1996. Its first release came out in January
6. Automatic scaling via auto-sharding and replica sets 1997. Its popularity has increased in the last few years and,
The list of features is huge but I will stop here, so readers according to DB-Engines, it is among the highest ranked
can explore the software for themselves. in recent times. PostgreSQL is an object-oriented RDBMS

92 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


For U & Me Overview
which emphasises extensibility and standard compliance. It MariaDB. It has been forked from MySQL as an alternative
implements a majority of the SQL2011 standards and comes to provide a drop-in replacement for MySQL, with better
with various enterprise class features. and a more enhanced set of features compared to the
The standard set of features that are expected in any equivalent MySQL release. It was started by the founder
industry DBMS are available in PostgreSQL as well. Here is of MySQL, Michael Widenius, after Oracle acquired Sun
the list of a few of its features that are gaining popularity: Microsystems (which had bought MySQL in 2008) in late
1. Runs on all major operating systems 2009, early 2010.
2. Fully ACID compliant for transaction reliability MariaDB was always meant to maintain high
3. Multi-version concurrency control to provide immunity to compatibility with MySQL, and it ensures drop-in
dirty reads and serialisability replacement with libraries as well as binary equivalence.
4. Support for SQL data types Some of the features of MariaDB are listed below.
5. Streaming replication to continuously ship and apply 1. Open and free: It is developed by the community under
Write Ahead Log records to the standby server... GPL and hence is available at no cost.
…and many more. 2. Support: MySQL itself has well established
documentation support, which can also be used for
Cassandra MariaDB. Apart from that, there is a large enough
Next in the list is Cassandra, ranked the second runner-up by MariaDB-specific knowledge base/online community.
DB-Engines for 2015. It was initially developed by Facebook 3. Speed: MariaDB is regarded as one of the fastest
engineers to power the social media giant’s inbox search and databases available, even faster than MySQL.
was then made open source in 2008. Cassandra is a distributed 4. Functionality: It provides support for all MySQL
DBMS specially designed to handle large amounts of data features, along with additional new and enhanced
spread across a vast cluster of machines. It offers support for features developed by the community.
data spanning across multiple data centres with asynchronous 5. Ease of use: As per user feedback, MariaDB is easy to
masterless replication. use, especially with features like flexible syntax.
Technically, Cassandra also broke away from the My list of DBMSs is merely a point for discussions
traditional RDBMS ‘table and columns’ model, with data to take off on the more than 200 such databases. It’s not
being stored on a single machine to scale across thousands of important which one is chosen and which is not; rather,
servers. Cassandra also places a high value on performance my attempt has been to give the reader an overview of
and scalability. A study done by the University of Toronto in how much the open source world has evolved in database
2012 found that among NoSQL databases, Cassandra was the management. Also, industry adoption of open source
clear winner in terms of scalability, and achieved the highest databases has been increasing, day by day, moving away
throughput across the maximum number of nodes. from traditional proprietary databases. Hence the mantra for
Even though its creators at Facebook have abandoned success is: “Go open, go big.”
Cassandra, it is still powering leading Web infrastructure
companies like Twitter, Netflix and Apple. A study done by
an Australian company, Solid IT, reveals that Cassandra is the References
second most popular NoSQL database after MongoDB and the [1] http://db-engines.com/en/ranking
third fastest growing database, over all. [2] https://www.mongodb.org/
Here is a small list of features that Cassandra offers. [3] https://en.wikipedia.org/wiki/MongoDB
[4] http://searchdatamanagement.techtarget.com/definition/
1. Decentralisation: Every node in a cluster has the
MongoDB
same role; hence, there’s no master, and so no single [5] https://en.wikipedia.org/wiki/MySQL
point of failure. [6] https://www.mysql.com/why-mysql/topreasons.html
2. Multiple data centre replication: It is designed [7] https://wiki.postgresql.org/wiki/Main_Page
[8] http://www.postgresql.org/
as a distributed system to support nodes across
[9] https://en.wikipedia.org/wiki/PostgreSQL
multiple data centres. [10] https://en.wikipedia.org/wiki/Apache_Cassandra
3. Cassandra Query Language (CQL): It has introduced a [11] http://www.wired.com/2014/08/datastax/
new query language called CQL, an SQL-like alternative. [12] https://en.wikipedia.org/wiki/MariaDB
[13] http://www.zdnet.com/article/open-source-mariadb-a-
4. MapReduce Support: Cassandra provides integration for
mysql-fork-challenges-oracle/
Hadoop with MapReduce support.
Again, I have picked only a few key features from
Cassandra’s vast list, leaving the rest for readers to explore. By: Ashish Bunkar
The author is a Linux professional with more than five years’
MariaDB experience in the open source field. He likes to share his interests in
SAN and virtualisation technologies.
Another RDBMS software that is gaining popularity is

94 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Overview For U & Me

Five Cool Custom Android ROMs


You Can Have Fun With
So you are sick and tired of the stock ROM provided by your Android device manufacturer? Don’t
despair—there’s hope for you, yet. Root your device and install a custom ROM. If you don’t know which
ROM to opt for, this article will help you clear your doubts.

I
f you have often heard the terms, stock ROM and custom
ROM and wondered what they meant, this article will explain screen, background services and even adding new software
the terms and their importance in the Android world. applications. Even the kernel is updated with the latest
Before we go further, ROM stands for ‘read-only patches to bring in performance, security and stability. So how
memory’ which refers to the internal storage of the phone or does one customise the default OS? As you already know,
tablet. The stock or the custom ROMs are the OS images that Android is an open source operating system and its source
reside in the ROM of your tablet, phone, etc. code is available for download, which means developers can
Stock ROMs are otherwise known as factory/ modify the code and the kernel in any way they would like to.
manufacturer OS images, which come with your phone
or tablet when you buy it. These ROMs have a few Why custom ROMs?
enhancements added over the default version of the OS The answer is simple - the stock ROM doesn’t give you much
by the device manufacturers. Some of the top smartphone room to customise your phone and tablet, whereas custom
manufacturers usually customise the default version of the OS ROMs are designed to provide flexibility along with more
by adding new features to enhance the user experience. efficient performance, features and stability.
A custom ROM is the customised version of the default ƒ Custom ROMs are not only faster but are more efficient
operating system, which is heavily modified with fun tweaks, and use less memory.
feature additions, performance enhancements and also the ƒ They have more features and better UIs, when compared
modification of the UI by changing the lock screen, home with the stock ROMs.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 95


For U & Me Overview
ƒ They have a good turnaround in getting bug fixes which, Doc:_porting_intro
in turn, increases the frequency in which you can get
updated versions. Paranoid Android
ƒ They are supported by the open source community, Paranoid Android or AOSPA has been one of the best ROMs
where community members can submit features and in the Android community for a long time. It has a lot of
fixes, which then get added to these ROMs. features and also a large user base of more than 200,000
If you have not yet explored the complete feature box-set (Source: Wikipedia).
of Android, shown below are the ‘Top 5’ custom Android Some of its features include:
ROMs that you could try. There are more custom ROMs ƒ Hover
out there with some unique features, but most of them don’t ƒ Immersive mode
support a wide range of devices. ƒ Pie
ƒ Peek
‘Top 5’ custom ROMs available in the Android It also supports dynamic system bar and a theme engine,
community
which is powered by Cyanogen Mod. If you are looking for a
CyanogenMod sleek custom ROM with more intuitive features, this would be
Paranoid Android the right choice.
PAC-ROM
http://www.paranoidandroid.co/
SlimROM
OmniROM
PAC-ROM
CyanogenMod
CyanogenMod is one of the best custom ROMs available
in the open source Android community. A very large
community of developers contribute to it, and a large
number of users have it on their phones and tablets.
With frequent updates like Nightly and Stable builds,
it’s no wonder that this is considered one of the most
reliable custom ROMs by current users. Currently, those
Figure 2: Pac-Rom logo
at CyanogenMod are working hard to bring out the
latest version of the Android operating system called PAC-ROM includes the best features among all the above
Marshmallow 6.0 to their supported phones and tablets. ROMs, and provides them in a single package. Since this is a
CyanogeMod’s OS will be named CM13.0. mix of everything, this ROM ought to be everyone’s choice
— if they are looking out for all the crazy features, along
with the stability.
http://www.pac-rom.com/

SlimROMs
SlimROMs, as the name suggests, is a completely stripped
Figure 1: Cyanogen Mod logo down version of Android. It has many features and offers up
to three variants that can be deployed in your device, based
Based on the Android Open Source Project, on your needs. There are lots of other features, too, which
CyanogenMod is designed to offer better performance are listed below:
and reliability than the Android based ROMs released
by vendors and carriers such as Google, T-Mobile, HTC,
etc. CyanogenMod also offers a variety of features
and enhancements that are not currently found in
stock versions of the Android OS (Source: http://www.
cyanogenmod.org/about).
CyanogenMod supports most Android phones and Figure 3: SlimROMs logo
tablets available in the market. To find out if your device is
supported, follow https://wiki.cyanogenmod.org/w/Devices ƒ ‘The Real Dark Slim’ is a theme which makes the UI
If you don’t find your device in the list of supported black, in addition to several other features
devices but you are keen on trying out CyanogenMod, then ƒ SlimROMs has completely different ‘recent’ and
you can follow its porting guide on how to port its OS to ‘favourite’ apps windows, and this includes the option to
your phone or tablet at https://wiki.cyanogenmod.org/w/ switch to either side. So it doesn’t matter whether you are

96 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Overview For U & Me
right-handed or left-handed! on the device as being void when you use a custom ROM.
ƒ It also supports various privacy features in addition to But again, the chances are very slim of any of these
events related to Shake and notification reminders issues arising nowadays. Our recommendation is to make
http://www.slimroms.net/ sure you check which OS image has lots of positive
feedback and go for it!
OmniROM With so many customisable options, custom ROMs will
OmniROM thrives on flexibility, and suit everyone’s needs and for a geek who is looking to explore
allows developers to work on the project new features, this is the way to go.
together to improve its features. It
supports a number of devices, and also Tip: You can try Googling the model number of your device
allows users to add features and the bugs along with “+ ROM” to learn more about it on the Internet.
discovered to their roadmap, which is Figure 4: OmniROM logo
publicly accessible.
To install OmniROM, go to http://docs.omnirom.org/
Installing_Omni_on_your_device References
https://omnirom.org/ [1] http://www.cyanogenmod.org/
[2] https://wiki.cyanogenmod.org/w/Main_Page
[3] https://www.android.com/
Challenges faced with custom ROMs [4] http://developer.android.com/index.html
Custom ROMs have a tendency to have bugs, too, since they
don’t go through the same amount of rigorous testing that By: Gobind Vijaykumar
vendors do before they ship the factory ROM with the device. The author works in the OS engineering group of Dell India R&D
Another downside is that you have to always go for a clean Centre, Bengaluru, and has over seven years of experience in
format/wipe before flashing the customised OS image on to server operating systems and storage solutions. His areas of
interest include security and storage technologies.
your device. And some vendors consider their warranty offered

Statement about ownership and other particulars about


OPEN SOURCE FOR YOU
FORM IV (See Rule 8)
1. Place of publication : New Delhi
2. Periodicity of its publication : Monthly
3. Printer’s Name : Ramesh Chopra
Nationality : Indian
Address : OPEN SOURCE FOR YOU
D-87/1, Okhla Industrial Area,
Phase-1, New Delhi 110020
4. Publisher’s Name : Same as (3) above
Nationality
and address
5. Names and addresses of : EFY Enterprises Pvt Ltd
individuals who own the D-87/1, Okhla Industrial Area,
newspaper & partners or Phase-1, New Delhi 110020
shareholders holding more
than 1% of the total capital

I, Ramesh Chopra, hereby declare that the particulars given above are true to the best of
my knowledge and belief.
Date: 29-2-2016
Ramesh Chopra
Publisher

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 97


For U & Me Interview

“Our gOal
is tO give
the wOrld
niche, cOst-
effective
technOlOgy
sOlutiOns”
ESDS, one of the world’s leading data centre
services and cloud solutions providers, has a
US patent for its method of resource allotment
and requirements. In other words, this is
a patent for auto scalability. Piyush
Somani, founder, MD and CEO of
ESDS Software Solutions Pvt Ltd,
talks about his company and the
benefits of having a patent with the
Open Source For You team.

Q How would you describe ESDS as a


company?
A. ESDS is a leading managed
cloud and data centre service
provider in India with subsidiaries
in the US and UK. It has been in
business for more than a decade
and has assisted thousands of
organisations in transforming
their presence online. ESDS takes
pride in hosting more than two
million websites and supporting
a customer base of over 35,000
from every country connected
to the Internet. The business of a
significant number of these corporate
entities is dependent on their Internet
presence. ESDS has an Uptime Institute
certified Tier 3 DC at Mumbai and a Tier 3 DC
at Nashik, along with multiple leased facilities
in the US, UK and other cities across India. The
company’s people believe in innovation, combined
with great customer service.

Piyush Somani, founder, MD and CEO of


ESDS Software Solutions Pvt Ltd

98 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Interview For U & Me
Q How have you differentiated yourselves from players like Q How does having such a patent impact the growth of a
Rackspace, or other Indian players? business?
A. Data centre players in India fall under different categories. A. The patented technology does help as a key enabler for
There are standalone data centre players that provide only co- an organisation’s growth. We are accustomed to always
location services, which means that they sell power, cooling and following the ‘big brothers’ in technology, but when you
space only. Then there are data centres owned by the telecom have a patent from the United States for a unique product,
players that are leveraging the advantage of bandwidth it helps local enterprises to develop faith in products that
connectivity (so the DC business naturally complements what are made in India. It also becomes a clear differentiator
they do), and then there are managed service providers. By regarding an organisation’s strength in technology as
and large, ESDS falls into the third category of data centres in compared to other providers, and it shows our commitment
India providing managed services. to innovate unique solutions to help businesses, which in
The differentiation that ESDS provides is the expertise turn helps in the nation’s growth.
in offering customised solutions as against standard product
offerings. ESDS believes in innovation and has its own R&D Q Where is your main data centre located?
team that is CMMI 3 certified. The team has developed quite A. Our Uptime Institute certified Tier 3 data centre is located
a few great data centre related products that are used by at Mahape in Navi Mumbai and our head office, along with
major enterprises in India today. Our biggest differentiator is our Tier 3 data centre, is situated in the heart of Nashik.
a unique auto-scaling cloud solution called eNlight, which is
combined with pay-per-consume billing. eNlight has a patent Q What would you say is the key factor driving this group?
from the USPTO (United States Patent and Trademark Office) A. A passion for continuous innovation and the trust of each
for auto scaling of resources in real-time. Compared to single employee in our vision of creating lifetime customer
Rackspace, we offer vertical real-time scaling and a pay-per- relationships (by providing cloud enabled solutions and
consume billing mode, which are clear differentiators. Our managed data centre services) are the two key factors
eNlight 360 provides users the power to manage various diverse driving the group.
hypervisors from a single control panel.
Q Then the challenge is more about proving to be a better
Q Does ESDS provide anything like a platform-as-a-service option as compared to the competitors, rather than having
or are you at the purely data centre level? to sell the concept?
A. Currently, we are a big time IaaS provider while we do A. We do not believe in or focus on proving ourselves
provide certain software on the SaaS model. We have recently as a better option. We believe in continuous innovation
launched an e-commerce platform that we provide as a rather than competing. Competition is about getting better,
service but, going forward, we will be focusing on providing centered across something similar, while our goal is to
various types of PaaS. give the world niche, cost-effective technology solutions.
We come up with solutions that fill the gap, and serve
Q ESDS has a US patent for the methods it uses for resource to eliminate the pain areas of our technology, thereby
allotment and requirements or, in other words, for auto eliminating the need to sell any concept that might or might
scalability. Can you tell us more about what this is all about? not take off.
A. A virtual machine (VM) is a typical instance of IaaS,
which consists of the CPU, RAM, the network, the disk and Q And how much is the business and revenue from
operating system with optional preinstalled applications. international clients?
The patent is for the method and system for detecting, in A. Revenue from Indian clients has crossed US$ 6 million,
real-time, the resource requirements of a system in a virtual while revenue from our international clients is more than
environment and automatically scaling resource parameters US$ 12 million.
according to these requirements. The virtual machine controller
constantly measures resource utilisation in the servers and Q Has the recent rule from the government of India to have
virtual machines associated with it. If a resource requirement more data centres located within India, particularly those
is detected with any virtual machine, the automatic resource related to the government, benefited you in any way?
scaling system detects the type of resource to be scaled, and A. Currently, the government is working on empanelling
scales the selected resource. Further, the resource may be Indian data centres for government institutions to host
scaled up or scaled down, based on the requirements. The their applications and data. Once this is done, government
scaled resource could be the CPU, RAM, disk or any such organisations and PSUs would no longer be required to host
resource. This system helps to save space and power without their data at the state data centres, and this will definitely
compromising security, performance and accessibility. benefit private data centre players.

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 99


For U & Me Interview

Q Would it be correct to assume that most of the customers customers can do a PoC (proof of concept) of their proprietary
are going to be high-end premium customers rather than systems on open platforms in our data centre.
small players?
A. We have a mix of hosting customers. The business of the Q Where do you see revenue coming from -- customers
majority of our international customers is high-volume with seeking an open source stack or those with a preference for
the per-customer revenue generation being low, whereas in the proprietary stack.
India, we are seeing mostly high-end premium customers A. We are seeing organisations taking the initiative of moving
getting on board and opting for our services. to the open source stack, and the numbers are increasing each
day. Customers are embracing open source technologies for
Q Has any startup grown with you? non-critical loads for sure, while trying out PoC for their
A. Yes, there are many startups critical systems. Currently, the
that are hosted with us and some of We believe that, very revenue would be around 50-50
them have grown 20 times in the but, going forward, the trend is
last three years. Due to our terms of soon, proprietary systems definitely skewing towards open
non-disclosure, I would be unable to will be a thing from source stacks.
identify them publicly. the past and those who
Q You have mentioned that you
Q Do you have any special do not embrace open are seeing growth coming from
programmes for startups? systems will follow suit. the telecom sector, existing
A. Our unique and innovative customers shifting to the cloud
pay-per-consume billing model itself is a special offering and from e-commerce. As I understand, these are the top
for startups. While cloud providers bill the customers for the three sources. Will this trend to continue or do you see any
allocation, eNlight cloud bills the customers for actual usage new sectors emerging?
of resources. This is something similar to the electricity billing A. Telecom players are always the trend setters in this industry.
model that we are accustomed to at our homes. We have also Fortunately, e-commerce companies were mostly startups,
assisted startups to have compute as a utility on a prepaid so they could directly move to the cloud, while the existing
billing model such as mobile SIM cards. This billing model customers who have invested heavily in their infrastructure
helps the clients save more than 30 per cent compared to the are already starting to move their workloads to the cloud,
pay-per-use model. as their infrastructure is approaching the end of its life.
Everyone is moving from the capex to the opex model, and
Q There is a lot of talk about the IoT. What is your personal many organisations have already realised the pain of having
take on this? Do you see a huge amount of business coming an in-house data centre. With the Digital India initiative, the
to cloud companies also because of IoT? government will contribute to the business at large, and with
A. The IoT is the future and it is not very far away. We will see our vertical auto scaling technology, industries in oil and gas, or
another data explosion from the point we are at today. Every aviation, or those that use legacy systems, will be able to take
instrument comes with multiple sensors attached to it today, be advantage of the cloud, thereby contributing to growth.
it a small watch, a pen or a mobile — all the way up to aircrafts
and rockets. Companies are coming up with digital products, Q So what is the biggest challenge for your business?
but where is all this data going to get stored? These companies A. The biggest challenge in the data centre industry is the
are not coming up with their own data centres. With the world huge capital required to build and maintain the data centre.
moving towards a ‘machines controlling machines’ era, the The investments needed to boost this industry in India are
business of the cloud is going to grow multifold in the next still not happening at the rate they should be. There is also the
three to five years and IoT is going to be a clear driver for this. challenge of skilled manpower, since it is a different kind of
IoT feeds into another revenue stream that goes by the name of industry with varied technology and the skills needed are not
Big Data analytics. Finally, the data has to be transformed into readily available in abundance. Besides, the cost of power and
information that is meaningful and can be used. bandwidth is still very high compared to developed nations,
thereby increasing the operating expenses.
Q What does open source mean to ESDS?
A. ESDS is a strong believer in open source technologies. Q. Lastly, what are the top three or five technology trends
We believe that, very soon, proprietary systems will be a that you would want a typical CIO or CTO to keep in mind
thing from the past and those who do not embrace open when it comes to the cloud, or even migrating to it?
systems will follow suit. Our products are completely built A. The hybrid cloud, the Internet of Things, Dockers and
on open source platforms. We also have a very unique way more initiatives around Devops, interoperability and security
of demonstrating a cross-platform concept, under which should be what CIOs or CTOs keep their eyes on.

100 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


For U & Me Interview

“A CIO
shOuld
hAve
dIgItIsAtIOn
On hIs
AgendA”
Digital India is seen as a great opportunity by
most organisations, but how it will eventually
shape up is something that we will all have
to wait and watch. However, companies
like Cisco are fully geared up to get on the
bandwagon with their digital strategy and
technology solutions, and are viewing it
as an initiative that will change the entire
technology landscape. In a conversation
with Open Source For You, Sanjay Kaul,
MD, service provider business, Cisco
India and SAARC, shared the company’s
initiatives with respect to the Digital
India movement. Excerpts:

Sanjay Kaul, MD, service provider


business, Cisco India and SAARC

Q How does Cisco plan to participate in the I think something great that happened is that the
Digital India movement? government has now carved out a vision, which is
We are already into it. We have been involved in architecting phenomenal, and we totally believe in it. We are now moving
the strategies for key stakeholders. For us, the journey from planning to execution. We are working on a number
started three years ago, with our concept of the ‘Internet of of fronts. We are working with service providers, states,
Everything’. We are working with service providers, setting municipalities, manufacturing hubs, and we see opportunities
up their connectivity layer, which is the foundation for Digital and projects all over the place. So, digitisation is happening
India. IP is what we have been doing for many decades, and and it is happening at a very fast pace.
I think that is what we are promoting — an end-to-end IP
architecture. This is the first layer towards making Digital Q How do you see the Internet of Things (IoT) space
India happen, because on top of that connectivity layer, which evolving with the Digital India concept? Do you see Cisco as
is end-to-end IP, one can put an evolved service platform. a key player in the IoT revolution?
That is where you can define how to extract value out of the If you look back, the Internet of Everything or IoE was a
connectivity layer. So for us the journey has already begun — slogan coined by John Chaves, who is our chairman. We
we are in the midst of it. always believed that digitisation has to happen, and I think

102 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


Interview For U & Me
it is happening around the globe. India needs to mature there will be millions of jobs created but these will be of
on many fronts. First is the connectivity front. And I think a different nature. It will not be about moving physical
with the government’s great initiative of connecting the goods or sitting at a retail shop attending to customers, but
panchayats, this will be of great help. We are trying to bring different activities.
in optical fibre up to the village level. Operators have already
started working on it. There are places in rural India where Q The Internet of Things is IT, electronics and telecom
there is no electricity, but operators’ mobiles work there. coming together. So when you approach an enterprise,
We have also seen a good positive movement on the who do you think is the right person to talk to,
spectrum side. Digitisation will happen over the mobile. generally? Earlier, the CIOs were approached because
So we are working on all these fronts. We are into it and they knew everything about IT. But with the IoT, there
we are very optimistic about the opportunities open to us. is a lot more than IT…
The reason is simple — we have core competence in IP Some of the best-in-class companies have appointed
connectivity. Over the years, we have built assets in the Chief Digital Officers or CDOs. In case a firm doesn’t
company that are very relevant. All this gives rise to a lot of have a different person for this role, the CIO takes on this
data. When the data is happening additional responsibility – and will be
in abundance, a couple of things called the CIO as well as the CDO. So
are very, very important, including
I think something I think it will vary from organisation to
how you collect this data, how you great that happened is organisation. The telcos will probably
analyse it, et al. Data is perishable that the government have this function divided between
and has value only for certain the CTO and the CIO, because both
hours. If it is not collected and
has now carved out parts have to digitise. So we will see
analysed in time, no value can a vision, which is different developments. But the best
be extracted from it. So, this is phenomenal, and we practice is to create a new role in the
another capability we are building form of the Chief Digital Officer, who
– it’s called data analytics.
totally believe in it. oversees digitisation across all functions.
Then we have the cloud, which Because it is not only about digitising
is a technology that will enable operators to reduce their opex. your network to IT — you are also digitising your finance
We can reduce their capex costs by up to 20 per cent. In many department, HR and logistics.
cases, we have helped them save about 65 per cent on their
opex, which is what is needed in India. So we practically Q Who is the real target audience when it comes to the
operate in all the domains of the Internet of Everything, which consumption of IoT?
is very relevant to the digitisation business. I think, typically, it’s CXOs. CIOs and CTOs can also
be the target, because they understand what their CDO is
Q So, apart from government bodies, are you also working proposing to the board.
with private enterprises to encourage IoT and digitisation?
Oh yes! All smart CIOs or CEOs have digitisation on their Q And what is the importance of systems integrators or
agenda. If they don’t, I can assure you that they will not SIs in the entire scheme of things?
be around in 10 years’ time. Because someone out there Systems integrators are just one piece in the jigsaw
is going to beat them, and we have seen examples of this. puzzle. When you go out and digitise industries, cities,
Look at what has happened to retail. We have seen examples manufacturing plants or mines, you have to form a team,
across the globe where operations have been completely or maybe even a consortium, which will include systems
digitised, leading to 50 per cent more productivity. I think integrators as well.

Read more stories on


TOPSECURITY STORIES
• CCTV camera market
is expected to double by
devices
2015
ELECTRONICS

• The latest in biometric to a bright future


security and surveillance in • CCTV Camera manufac
Ana lytic s Syst ems
turers can look forw
Turn
ard
ing a CCTV into a proactive
tool
INDUSTRY IS AT A
• Video
with new technologies
www.electronicsb2b.com urity cam eras evol ving
• Sec
eras
• The latest in dome cam nt security cameras
-proof and vandal-resista
• The latest in weather

Log on to www.electronicsb2b.com and be in touch with the Electronics B2B Fraternity 24x7

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 103


TIPS
& TRICKS

Creating a partition larger than 3TB in Linux (parted) unit TB


Many of us do not know that we cannot create a To create a 3TB partition size, enter:
Linux partition larger than 2TB using the fdisk command. (parted) mkpart primary 0 0
Using fdisk is fine for desktop and laptop users, but not
when it comes to servers and especially a storage device, Or:
where you need a large partition. For example, you cannot
create a 3TB or 4TB partition size (RAID based) using the (parted) mkpart primary 0.00TB 3.00TB
fdisk command while building a NAS solution.
To solve this problem, you can use the GNU parted To print the current partitions, enter:
command with GPT. It supports Intel EFI/GPT partition tables. (parted) print
Here is an example of how we can use parted. We can use
the following command to find the exact size of the HDD: The output will be:

# fdisk -l /dev/sdb Model: ATA ST33000651AS (scsi)


Disk /dev/sdb: 3.00TB
The output will show the exact disk size and the Sector size (logical/physical): 512B/512B
partition details. In our case, the disk size is 3000.6GB. Partition Table: gpt
So we will be using parted to create a partition instead Number Start End Size File system Name Flags
of fdisk, as follows: 1 0.00TB 3.00TB 3.00TB ext4 primary

# parted /dev/sdb Quit and save the changes, enter:


(parted) quit
GNU Parted 2.3
Using /dev/sdb
Welcome to GNU Parted! Type ‘help’ to view a list of Note: You may need to update /etc/fstab.
commands.
(parted)
Once you are done with the partitioning of the disk, you
Now create a new GPT disklabel: need to format and create a file system to make the disk
useable. Use the mkfs.ext3 or mkfs.ext4 commands to format
(parted) mklabel gpt the file system, by entering the following command:

The output will be as shown below: # mkfs.ext3 /dev/sdb1

Warning: The existing disk label on /dev/sdb will be Or:


destroyed and all data on this disk will be lost. Do you
want to continue? # mkfs.ext4 /dev/sdb1
Yes/No? yes
(parted) Once the formatting is complete, you have to mount the
Next, set the default unit to TB, enter: partition, by using the following commands:

104 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com


# mkdir /data following set of commands to change the mac address.
# mount /dev/sdb1 /data
First, disable the card as follows:
You can check if the partition has mounted or not by
issuing the command below: #ifconfig wlan0 down

# df -H Then use the command given below:

Filesystem Size Used Avail Use% Mounted on #ifconfig wlan0 hw ether 12:12:12:12:12:12
/dev/sdc1 16G 819M 14G 6% /
tmpfs 1.6G 0 1.6G 0% /lib/init/ Now check the mac address by running the following
rw command:
udev 1.6G 123k 1.6G 1% /dev
tmpfs 1.6G 0 1.6G 0% /dev/shm #ifconfig wlan0 up
/dev/sdb1 3.0T 211M 2.9T 1% /data
You should get the output with the new mac address. If the
The output will show that you have 3TB space network manager stops you from doing this, just stop it before
available for use in your system. applying the above commands by using the following command:

– Amar Gehlot, amar281494@gmail.com # service network manager stop

Here, wlan0 is the name of my wireless card.


Base64 encoding/decoding in shell
scripting – Himanshu Gupta, himanshugpt05@gmail.com
We all know the importance of base64 encoding. Almost all
programming languages support base64 encoding/decoding.
Let us see how we can do this from the command line. GNU/ Changing the host name in Kali Linux
Linux provides the ‘base64’ utility, which will do the needful. Sometimes, we need to change the host name of the
Let us explore more about it with a simple example. system that we are using. Here are a few steps to do this when
First encode text to base64, as follows: using Kali Linux.

[bash]$ echo “OpenSource for You is awesome GNU/Linux Start your system, open the terminal and run the following
magazine.” | base64 command:
T3BlblNvdXJjZSBmb3IgWW91IGlzIGF3ZXNvbWUgR05
VL0xpbnV4IG1hZ2F6aW5lLgo= #cd /etc/

Now let us decode the same text, as shown below: #find hostname

[bash]$ echo “T3BlblNvdXJjZSBmb3IgWW91IGlzIGF3ZXNvbWUgR05 #leafpad hostname


VL0xpbnV4IG1hZ2F6aW5lLgo=” | base64 -d
Change the name and save the file. Now reboot your
OpenSource for You is awesome GNU/Linux magazine. system to make the changes effective.

It’s really simple, right? – Bhargab Choudhury, bhargabchoudhury24@gmail.com


– Narendra Kangralkar,
narendrakangralkar@gmail.com

Share Your Linux Recipes!


Changing the mac address of a wireless The joy of using Linux is in finding ways to get around
or Ethernet card problems—take them head on, defeat them! We invite you
to share your tips and tricks with us for publication in OSFY
We can change the mac address (the hardware address
so that they can reach a wider audience. Your tips could be
of the network device) of any card by just running a few related to administration, programming, troubleshooting or
commands in Linux. There are also some tools available general tweaking. Submit them at www.opensourceforu.com.
The sender of each published tip will get a T-shirt.
for this, but if you do not have them, you can apply the

www.OpenSourceForU.com | OPEN SOURCE FOR YOU | APRIL 2016 | 105


OSFYOSFY
DVD DVD

DVD OF THE MONTH


Back up and manage all your valuable data.
deepin 15.1
This is an operating system with applications that allow you to
do many things—browse Web pages, enjoy movies, initiate social
interactions through networks or do your daily office work. The
bundled DVD is a live edition for the 64-bit architecture.

Linux Lite 2.8


This is an easy-to-use, simple, fast and free Linux operating
system for daily use. You can find the ISO image in the folder
other_isos on the root of the DVD. The bundled ISO image is for
64-bit computers.

FreeNAS 9.3
FreeNAS is an operating system that can be installed on virtually
any hardware platform, in order to share data over a network.
FreeNAS is the simplest way to create a centralised and easily
accessible place for your data. Use FreeNAS with ZFS to protect,
store and back up all your data.

Clonezilla 2.4.5
This is free and open source software for disk imaging and cloning.
It helps you in systems deployment, bare metal backup and
recovery.

Rockstor 3.8.12
Build and manage your own Linux and BTRFS powered advanced
NAS and cloud storage with ease. Rockstor is a CentOS based
Linux distribution designed for network attached storage. You can
find the ISO image in the folder other_isos on the root of the DVD.
The bundled ISO image is for 64-bit computers.

What is a live DVD?


A live CD/DVD or live disk contains a bootable operating
system, the core program of any computer, which is
designed to run all your programs and manage all your
hardware and software.
Live CDs/DVDs have the ability to run a complete,
modern OS on a computer even without secondary storage,
such as a hard disk drive. The CD/DVD directly runs the OS
and other applications from the DVD drive itself. Thus, a live
disk allows you to try the OS before you install it, without
erasing or installing anything on your current system. Such
disks are used to demonstrate features or try out a release.
They are also used to test hardware functionality, before
actual installation. To run a live DVD, you need to boot your
computer using the disk in the ROM drive. To know how
to set a boot device in BIOS, please refer to the hardware
documentation for your system.

106 | APRIL 2016 | OPEN SOURCE FOR YOU | www.OpenSourceForU.com

Das könnte Ihnen auch gefallen