TRUST,SAFETY, AND

RELIABILITY

CAUSES OF COMPUTER
FAILURE
Hardware errors
Software errors
The computer solves the wrong
problem
Misuse
Communication failure
Malice

Seatwork : Therac 25 case

Suppose a new graduate in say, economics, who had
completed several courses in programming with good
grades, was offered a job programming for a
company that made x-ray machines. Suppose also
that the first project for that person was to write a
program that controlled the dosage for a new x-ray
machine. The new programmer would have sole
responsibility for developing and testing the program.
A. Would it be morally permissible for the new graduate to
take that job ? Explain
B. Would it be morally permissible for the company to offer
the new graduate that job ? Explain

CAUSES OF SOFTWARE
FAILURE
a. Failure of the developers to follow
set rigorous engineering principles.
b. Unrealistic schedule
c. Lack of domain knowledge
d. Mistakes in defining user
requirements and turning them into
lines of codes.

Quiz # 1
1. Suppose you were able to identify
the particular persons who made the
decision to ignore the bug in the
codes of Thera-25. Would you hold
the person or persons responsible for
the accident. Cite a Moral Theory for
your position .

Why is there a need for software

quality ?
accurate, thorough and timely processing of
business transactions
eg. Airline online-ticket reservation, electronic- funds
transfer system of banks.

Reduce waste and costs,

Eliminate human error
Improve quality of products
Eliminate product defects before they affect the
product quality.
shorten the production time to manufacture
products.
Can create a safe operating conditions for people /
employees

Do Computer / IT professionals have

any Moral responsibility ?
Michael Bayles (2003 )
Professionals have a special obligation to
their clients to be worthy of a clients trust,
and this leads to obligations of honesty,
candor ,competence, diligence, loyalty
and discretion.

Do Computer / IT professionals have

any Moral responsibility ?
Gotterbarn (2001)
Because software engineers and their
teams are responsible for developing
safety-critical systems, they have
significant opportunities to :
Do good or cause harm
Enable others to do good or cause harm
Influence others to do good or cause harm

Safety- Critical Software

Refers to computer systems that can
have a direct life-threatening
impact. ( Kevin Bowyer, 2001)

Aircraft and air traffic control system

Mass transportation systems
Nuclear reactors,
Missile systems
Medical treatment systems.

Safety- Critical Software

Software applications used in the
design of physical systems and
structures whose failures can also have
an impact that is life threatening.
Eg. Software used in the
- design of bridges and buildings
- selection of water disposal sites
- development of analytical models for medical
treatment

Examples of Safety Critical

Failures

Conflicts of Professional Responsibility :

Employee Loyalty and Whistle-Blowing
Do employees have an obligation
of Loyalty to Employers?
Many ethicist believe that while
loyalty may not be an obligation that
is absolute, nonetheless, employees
have a prima facie obligation of
loyalty in employment contexts.

Conflicts of Professional Responsibility :

Employee Loyalty and Whistle-Blowing
Do computer/IT professionals necessarily
have an obligation to be loyal to their
employers ?
What if CIT professionals obligation of
loyalty come into conflict with his
obligation to society as a whole? Does it
still follow that he or she has an absolute
obligation of loyalty to his employer ?

Whistleblowing

The disclosure by a person, usually

an employee in a government
agency or private enterprise, to the
public or to those in authority, of
mismanagement, corruption,
illegality, or some other wrongdoing
that threaten the public interest.

Being Morally Permitted to

Blow the Whistle
De Georges three conditions
1. The product will do serious and
considerable harm to the public
2. The engineers have reported the serious
threat to their immediate supervisor.
3. The engineers have exhausted the
internal procedures and possibilities
within the company, including going to
the board of directors, having received no
support from their immediate supervisor.

Being Morally Required to

Blow the Whistle
De Georges two conditions
1. The engineers have accessible,
documented evidence that would
convince a reasonable, impartial, observer
that ones view of the situation is correct.
2. The engineers have good reasons to
believe that by going public the necessary
changes will be brought about.

Case 1 : Whistle- Blowing

and BART
In the 1970s an incident at Bay Area Rapid
Transit occurred where a certain transport system
is being developed. Three engineers working on
the project were concerned about the safety of
the transit systems Automatic Train control
System. They decided to take their concerns to
their immediate managers but were not satisfied
with the answers they received. So, the three
engineers next took their concerns to the
organizations board of directors. Once again,
however, they were dissatisfied with the lack of
support from the board. At that point the
engineers decided to blow the whistle. Shortly
afterwards, they were fired.

Case 2 : Whistle-blowing
and SDI

In the early 1980s, a U.S military proposal

called the Strategic Defense Initiative was
introduced and debated. SDI was a National
Missile Defense system that would provide a
defense shield against incoming ballistic
missiles. The SDI proposal, which was vigorously
supported by the Presidents administration,
soon became very controversial. While SDIs
supporters argued that the missile system was
essential for the countrys national defense,
crictics argued that the system was unreliable.
One Critic, David Parnas, decided to go public
with his concern about SDI.