Experiments, Tests, and Data

Massachusetts Institute of Technology, Subject 2.017

 Purpose of Experiments and Tests

• Prove or Support a Hypothesis

– The Earth’s diameter is 6500km.
– Multiple propellers on a single shaft can reduce cavitation (Turbinia).
– The archaea prokaryotes in the ocean fix carbon and consume other organisms,
and the balance has profound impact on ocean uptake of CO2. (Ed Delong, Ann
Pearson, etc.)
– Outriggers provide better roll stability than does a single hull in random beam
seas, when wavelength is much larger than the beam.

• Prove a Capability, Support Design

– Manned flight to the upper atmosphere can be achieved bi-weekly with a

specialized aircraft (X-Prize).
– Characteristic of lift force as a function of elevator aspect ratio and inflow angle.
– Delay calculation in pulsed 20kHz acoustic signals is possible with the TattleTale
Model 8, and the performance obtained is XX.

Massachusetts Institute of Technology, Subject 2.017

 • Does the work

stand up to

scrutiny?

– Use of controls

– Calibration
– Data quality
– Data processing

– Documentation and
record-keeping!

Massachusetts Institute of Technology, Subject 2.017

 Controls

• Did you really measure what you thought?

• Rat Maze: Is the maze acoustically navigable?

(R. Feynman)
• Mass Spectroscopy: When you put in a sample
of known composition, are the other bins clean?
• When measuring electrical resistance, touch the
probes together. Check a precision resistor too.
• Resonance in load measurement rigs?

• When measuring hull resistance, does zero

speed give zero force?

Massachusetts Institute of Technology, Subject 2.017

 Calibration

• More time can be spent on calibration than the rest of the experiment!
• Sensors should be calibrated and re-checked using independent

references, such as:

– Manufacturer’s specifications
– Another sensor with very well-known calibration ÅÆ
– A tape measure, protractor, calipers, weights & balance, stopwatch, etc..
• Calibration range should include the expected range in the experiment.
• Some statistics of the calibration:
– Precision of fit (r-value or V)
– Linearity (if applicable)
• Understand special properties of the sensor, e.g., drift, PWM

Massachusetts Institute of Technology, Subject 2.017

 Data and Sensor Quality

• Signal-to-Noise Ratio

(SNR): compares V to the

signal you want

• Repeatability/Precision: If

we run the same test again,

how close is the answer?

• Accuracy: Take the average

of a large number of tests –

is it the right value?

Massachusetts Institute of Technology, Subject 2.017

 Time and Frequency Domain

• Fourier series/transforms establish an exact

correspondence between these domains, e.g.,

T
Xm = V 0
cos( 2Sm t / T ) z(t) dt * 2 / T, m = 0,1,2,…
T
Ym = V0 sin( 2S m t / T ) z(t) dt * 2 / T

z(t) = X0 / 2 + 6 Xm cos( 2S m t / T ) + 6 Ym sin( 2S m t / T )

Massachusetts Institute of Technology, Subject 2.017

 Time Resolution in

Sampled Systems

• The Sampling Theorom shows that the highest

frequency that can be detected by sampling at frequency
Zs = 2S't is the Nyquist rate: ZN = Zs / 2.
• Higher frequencies than this are “aliased” to the range
below the Nyquist rate, through “frequency folding.” This
includes sensor noise!
• The required rate for “visual” analysis of the signal, and
phase and magnitude calculation is much higher, say ten
samples per cycle.
Massachusetts Institute of Technology, Subject 2.017
 Sample Statistics

• Sample mean m:

• Sample standard dev. V:

V = sqrt [ ( (x1-m)2 + (x2-m)2 + … + (xn-m)2 ) / (n-1) ]

• Error budgets for multiplication and addition

(VA is standard deviation of A):

(A + VA)(B + VB) ~ AB + AVB + BVA

Example: (1.0 + V0.2)(3.0 + V0.3) ~ 3.0 + V0.9

(A + VA) + (B + VB) = A + B + V(A+B)

Example: (1.0 + s0.2) + (3.0 + s0.3) = 4.0 + V0.5

Massachusetts Institute of Technology, Subject 2.017

 Gaussian (Normal) Distribution

Probability Density Function f(x) ~ Histogram

f(x) = exp [ - (x-m)2 / 2V2 ] / sqrt(2S) / V

This is the most common

distribution encountered

in sensors and systems.

+/- 1V covers 68.3%

+/- 2V covers 95.4%

+/- 3V covers 99.7%

Area under f(x) is 1!

Massachusetts Institute of Technology, Subject 2.017

 Filtering of Signals

x Filter xf

Use good judgement!

filtering brings out trends, reduces noise

filtering obscures dynamic response

Causal filtering: xf(t) depends only on past

measurements – appropriate for real-time

implementation

Example: xf(t) = (1-H)xf(t-1) + Hx(t-1)

Acausal filtering: xf(t) depends on all
measurements – appropriate for post-processing
Example: xf(t) = [ x(t+1) + x(t) + x(t-1) ] / 3
Massachusetts Institute of Technology, Subject 2.017
 A first-order filter transfer
function in the freq. domain:
xf(jZ) / x(jZ) = O / (jw + O)
At low Z, this is approximately
1 (OO)

At high Z, this goes to 0

magnitude, with 90 degrees
phase lag (O/jZ = -jOZ)

Time domain equivalent:

dxf/dt = O (x – xf)

In discrete time, try

xf(k) = (1-O't) xf(k-1) +

O't x(k-1)

Massachusetts Institute of Technology, Subject 2.017

 • BUT linear filters will not handle outliers very well!

• First defense against outliers: find out their origin

and eliminate them at the beginning!
• Detection: Exceeding a known, fixed bound, or an
impossible deviation from previous values. Example:
vehicle speed >> the possible value given thrust level
and prior tests.
• Second defense: set data to NaN (or equivalent), so
it won’t be used in calculations.
• Third defense: try to fill in.

Example:
ifȱabs(x(k)ȱ– x(kȬ1))ȱ>ȱMX,

x(k)ȱ=ȱx(kȬ1)ȱ;

end;

Æ Limited usefulness!
Massachusetts Institute of Technology, Subject 2.017
 Presentations: Written

and Spoken

or

People will pay more attention to

you if you communicate well!

Massachusetts Institute of Technology, Subject 2.017

 Sources and Ethics

• Somebody has almost certainly thought about

what you are doing, and parts of it have almost
certainly been solved.
• For specific items, you must give an original
source and cite it properly.
• Refereed publications vs. flashy Internet

postings.

• Plagiarism:
Consider it ILLEGAL.
If there is any question about whether a phrase
(or even a particular word) should be cited,
protect yourself! … and the associated noise is
“systematically coupled to the
underlying process” [13]. …
Massachusetts Institute of Technology, Subject 2.017
 Linearity
Introduction:
• Bring reader from general to specific
• State hypothesis or objective
• Indicate why work is important
• Review prior work that applies
• Start at the beginning • etc
and go to the end!
Approach:
Antithesis: Michael • How the experiment or test was designed
Ondaatji (The • Details of the apparatus or system
• Accuracy and precision issues
English Patient) • etc
• Flowchart or detailed
Results:
outline may help • Major Result A, with figures and description
• Major Result B
• Omit needless • etc
words*.
Discussion:
• Do results support hypothesis?
* Strunk, Jr., W. and E.B. White, 1972. • Impact of findings
The elements of style. Allyn and • Future work
Bacon: Boston. • etc

Massachusetts Institute of Technology, Subject 2.017

 Pointers on Speaking

The audience is here to see YOU, not

just your materials.

Smile and engage them!

Write out your talk so it is

clean from start to end.

Don’t lose anyone!

cumulative

Practice your talk so you

distribution of expertise
are confident up there.
level in the audience

Get feedback on your talk,

“layman” “expert”
because it will help.

average YOU average YOU

student (student) professional (professional)
Prepare for questions.
90% of the talk accessible to
90% of the audience

Massachusetts Institute of Technology, Subject 2.017

 A GOOD FIGURE > 1000 WORDS

A bad figure is worth a few bad words

Massachusetts Institute of Technology, Subject 2.017

 Wind speed and direction as a
function of time. Top two plots
are combined into the bottom
plot: one independent
variable, three dependent
variables.

Vehicle trajectory: one

hidden independent
variable; five dependent
variables
Massachusetts Institute of Technology, Subject 2.017
 Figure from Principles of Naval Architecture, E. Lewis, ed.,
SNAME: New York, 1988. Original reference: Vossers, G., and
W.A. Swaan 1960. Some seakeeping tests with a Victory model.
Int. Shipbuilding Progress.

Image removed for copyright reasons.

Figure and its caption from above-mentioned source.

Shows two independent

and one dependent

variable. Style shows the

effects of varying phase

and period.

Caption injured, and y-axis label missing; gives

three independent variables (length ratio, Froude
number, and heading to waves) and one
dependent variable (added power coefficient).
Massachusetts Institute of Technology, Subject 2.017