DESIGNS TO BALANCE THE EFFECT OF ORDER OF PRESENTATION AND FIRST-ORDER CARRY-OVER EFFECTS IN HALL TESTS

HALLIDAY J. MACFIE and NICHOLAS BRATCHELL

Institute of Food Research Reading Laboratory Shinjield, Berks, RG2 9AT United Kingdom
KEITH GREENHOFF

Mobile Sensory Testing Services 54-62 Station Road East Oxted, Surrey, RH8 OPG United Kingdom
LLOYD V. VALLIS

29 Austen Road Glastonbury Somerset, BA6 9BD United Kingdom

Received for Publication August 7, 1989

ABSTRACT
The problem of balancing out the effect of order o presentation and the carryf over effect o a preceding sample over a series o presentations o the same set o f f f f samples is addressed. A series of designs developed by Williams (1949) are used. The method o calculation is given. Tables containing about 50 consumers o f f each design for presenting from 4 through to I6 samples are given.

INTRODUCTION
In sensory trials and hall tests several effects can influence the assessments of samples or treatments. In particular it is well recognized that different assessors give different assessments. But more insidious effects may be produced by the position in order of presentation of samples and by carry-over from one sample to the next. Failure to take account of these effects can lead to biased results, and consequently to misinterpretation of the results of analysis of variance, preference mapping or principal components analysis.

Journal of Sensory Studies 4(1989) 129-148. All Rights Reserved. @Copyright 1989 by Food & Nutrition Press, Inc., Tnunbull, Connecticut.

129

130

H.J. MACFIE, N . BRATCHELL, K. GREENHOFF AND L.V. VALLIS

Suitable experimental design can minimize or remove these extraneous effects. In sensory trials it is now common practice to balance out the order of presentation of treatments so that, within a session, each treatment occurs an equal number of times in each position (Side1 and Stone 1 8 ) In the context of 95. hall tests, where a consumer samples each of the treatments once and where the number of consumers that will be recruited is not known in advance, it may be thought that simple randomization of treatment allocation to position would be sufficient. However, our experience in carrying out hall tests for preference mapping exercises (Jones et al. 1989;Nute et al. 1989) indicates that quite severe imbalance can occur. Furthermore the possibility that a particular treatment might influence the assessment of the next treatment (first-order carry-over effects) is more likely using inexperienced assessors but is not adequately addressed by simple randomization. The problem of designing an appropriate trial can be approached as follows. It is desired to estimate or eliminate the effects of consumer, position in order of presentation, treatment and first-order carry-over. This implies that each consumer receives each treatment once, but over the whole experiment (design) each treatment occurs equally frequently in each position and each treatment is preceded equally frequently by each other treatment. Inherent in this is the condition of balance which allows effects to be estimated independently of each other in a standard analysis of variance. These designs are also balanced for effects of subsequent treatments (anticipation). The problem is neatly solved using a class of designs derived by Williams (1949).This paper summarizes the method used to generate the designs and presents designs for 4 to 1 treatments using about 50 consumers. In each 6 design, each assessor receives all treatments and over the whole design each treatment occurs equally frequently at each position and each treatment is preceded equally frequently by each other treatment. The implication for general analysis (for instance preference mapping) of experiments using these designs is that one treatment does not occur mainly in one position in order of presentation or is not preceded always by the same treatment. Hence the results are not biased by positional and carry-over effects. Formal analysis of variance for estimating and testing treatment, consumer, position and first-order carry-over effects is more complex and will be more fully dealt with in a subsequent paper.

GENERATION OF DESIGNS
If there are t treatments, then there are t(t - 1) ordered pairs of treatments. Since there are (t - 1) adjacent pairs of treatments for each consumer this implies that there must be some multiple o f t consumers to achieve balance. And, clearly, some multiple o f t consumers is required to fulfill the condition of equal frequency at each position. However, to achieve balance, the two cases t even and t

HALL TESTS

131

odd must be treated separately. With t even balance is achieved with any multiple of t ; with t odd any multiple of 2t is required. Both types of design can be generated from Latin squares. These are square tables in which each treatment occurs once in each column and once in each row. For example for t = 4, the Latin square is
1 2 3 4

2 3 4 1

3 4 1 2

4 1 2 3

This table was generated by the intuitively obvious system of cyclic permutation of the rows. Although this method ensures that each consumer receives each treatment once and overall each treatment occurs once in each position it does not produce a design balance for carry-over effects: note, for instance, the triple occurrence of the pair 1-2 and nonoccurrence of the pair 2- 1. The following strategy ensures designs balanced for consumer, positional and first-order carry-over effects. The t treatments are denoted 0, 1, ..., t-1. The method is to select a first row of a Latin square and a simple algorithm allows the generation of successive rows.

t Even
In general the first row of the Latin square is given by the sequence
0

t-1

t-2

....

t/2

Williams (1949) gives alternative methods for particular cases of t .

t Odd
This requires the generation of a pair of Latin squares. In general the first row of each is given by the sequences

square I , row I

t-1

t-2

.... (t-1)/2

(t+1)/2

Note that these two rows are simply the reverse order of each other. Other methods exist for particular cases o f t (Williams 1949).

132

H.J. MACFIE, N. BRATCHELL, K. GREENHOFF AND L.V. VALLIS

Generation of Subsequent Rows

Successive rows of the Latin squares are generated by adding 1 to each element of the preceding row. Any treatment number k greater than ( t - 1) is replaced by k - t , e.g., t = 6 , k=6 gives k=O, and t = 6 , k=7 gives k = l . For example, t
=

0 1 2 3 4 5
And.
=5

1 2 3 4 5 0

5 0 1 2 3 4

2 3 4 5 0 1

4 5 0 1 2 3

3 4 5 0 1 2

0 1 2 3 4

1 2 3 4 0

4 0 1 2 3

2 3 4 0 1

3 4 0 1 2

3 4 0 1 2

2 3 4 0 1

4 0 1 2 3

1 2 3 4 0

0 1 2 3 4

The rows of the second square are simply the reverse order of the corresponding rows in the first square.

Randomization
Each row of the Latin square gives the order of presentation of treatments to each consumer. And in a single block o f t ( t even) or 2t (t odd) rows each treatment occurs equally frequently in each position (once, t even, or twice, f odd) and each treatment is preceded equally frequently by each other treatment (again, once or twice depending on t ) . Prior to application the treatments should be randomly allocated to the symbols 0, 1, ..., f-1 and the rows of the block should be randomly permuted. This does not alter the properties of the design but addresses the potential bias due to systematic generation of the Latin squares; note the order 0, 1, 2, ..., t-1 for the first elements of the Latin squares above. For example, from the 6 X 6 Latin square above, random allocation of treatments and permutation of rows may give the design

HALL TESTS

133

5 2 4 0 3 1

4 1 2 5 0 3

0 4 5 3 1 2

2 3 1 4 5 0

3 5 0 1 2 4

1 0 3 2 4 5

The first consumer receives the treatments in the order 5,4,0,2,3,1, and so on. The same strategy is employed when t is odd after placing the two Latin squares consecutively and randomizing over the 2t rows. Replication of this basic design is achieved by further random allocation of treatments and permutation of the rows of the initial Latin squares. Randomization is important as it reduces the potential bias. Consider the design for six treatments above. The ordered pair 1-3 occurs as the first pair in the last row. Repeated use of this block ignores the possibility of placing the other ordered pairs 1-2, 1-2 and 1 4 in this position.

TABLES OF DESIGNS
Tables 1-13 present balanced designs for t =4 to t = 16 treatments with about 50 replicates of each treatment (i.e. consumers). The rows denote the sequence of presentation; the treatments are labeled 1,2, ...,t. Each consecutive block of t (t even) or 2t (t odd) consumers is a balanced design. It should be noted that in applying these designs further randomization is necessary. It can be accomplished by randomizing the order of use of blocks o f t or 2t consumers. Further replicates of the basic balanced design can be generated by randomly allocating treatments to symbols for a block of t or 2t consumers and randomly permuting the rows of the block.

134

H.J. MACFIE, N. BRATCHELL, K. GREENHOFF AND L.V. VALLIS

TABLE 1 . 4 TREATMENTS: 40 REPLICATES IN 10 BALANCED BLOCKS OF 4 CONSUMERS

Samples

2 3 1 4 2 1 2 4 3 1 2 4 3 1 3 4
3

3 2 4 1 3 3 4 2 1 2 1 3 4 4 2 2 3 1 1 4 3 2 3 1 4 2 4 2 1 3 1 4 3 2 2 1 4 3
3 2

4 1 2 3 4 2 3 1 4 3 4 1 2 3 4 1 2 3 1 2 4 4 3 2 1 1 4 3 2 2 1 4 3 4 3 2 1 1 4 3 2
1

:I
3 4 5 6 7 8
9

4 3 2 1 4 1 3 2 4 3 2 1 2 1

10
11

12

::
1
16

'

4 2 3 4 1 2 4 1 3 3 1 2 4 3 2 1 4 3 2 1 4

17
18

I9 20

4 2 1 3 1 2

21 22 23 24
25 26 27 28
29 30

'i
2 3 4 1 4 3 2 1 3 4 5 6
4

31 32 3 3 3 3

Note: Blocks of the design should be applied in a random order. Improved randomization and further replicates can be generated by random allocation of treatments to symbols followed by random ordering of rows for each block of the design.

HALL TESTS

135

TABLE 2. 5 TREATMENTS: 50 REPLICATES IN 5 BALANCED BLOCKS OF 10 CONSUMERS

samo1es
1

2 5 1 2 4 3 3 4 5 5 1 2 5 2 1 4 3 3 5 2 1 4 3 4 2 1 5 2 1 4 3 5 5 2 1 4 3 1 4 5 2 3 1 5 4 3 2 4 3 5 2 1 4 3 2 1 5 1 5 2 4 3

4 4 5 1 3 2 4 5 1 4 2 3 3 4 2 5 1 5 4 1 3 2 1 2 3 5 4 4 3 5 2 1 3 1 5 2 4 2 3 1 4 5 3 2 1 5 4 2 1 3 5 4 2 1 5 4 3 3 2 4 1 5

5 3 4 5 2 1 5 1 2 3 4 1 5 4 3 2 4 2 3 5 1 5 3 1 4 2 5 2 1 4 3 4 5 3 1 2 4 5 2 3 1 5 4 3 2 1 5 4 1 3 2

I
1

2 3 4 5
6

7
8

1 2 3 5 4 2 3 4 ' 1 4 1 3 2 5 1 3 4 2 5 2 5 4 3 1 3 5 2 1 4 1 4 2 3 5 5 2 3 1 4 1 2 3 4 5 6 7 8 9 0

2 3 4 1 5 1 2 3 5 2 3 5 1 4 2 1 5 4 3 4 1 5 2 3 1 4 3 5 2 2 3 4 5 1 3 1 4 5 2

9
10

11

12 13 14
15
16

17
1u
19 20

21 22 23
24

25
26

27 28
29 30

31 32 33 34 35 36

37
38 39
40

4 4 4 4 4 4 4 4 4 5

136

H.J. MACFIE, N. BRATCHELL, K. GREENHOFF AND L.V. VALLIS

TABLE 3. 6 TREATMENTS: 48 REPLICATES IN 8 BALANCED BLOCKS OF 6 CONSUMERS

Samples

1 2 3 4 5 6
7
8

6 4 2

5 6
2 3 1
6 5

2 6

5 3
4 6

1
2 3 5 6
4

1
5 I
2

1
3
4

1
5

1
2
2

3 3 1
4

5 5 4

6
4 6 3 2

1
2 6
4

9 10

1
3

2
1

3 5
6 4

11
12
13 14

6
2

2
4

3
6

1
3 5

5 3
4

1
5 5 4 3 2

1
3

1
2 3
4

2 6
4

15 16
17 18

5
5
3

6 5 2 4

1
4 6 2
6

2 5
6
3

1
5

3
3
4 1 2 5 6

1
4

1 6
2 6 3

19 20 21 22 23 24

1
5 2 3 6 4

1
4

2
5

6 4 1

5 2
3
2

6 3

1
4 5 4 2 3 6

2
6
5 1
4

1
3
4 5

25 26 27 28 29
30

1
6 2 3 4 5

5
3
6

1
4

1
4

3
6

5 3 6
3
4

1
6
2 4

5 1
2

31 32

5
2

33
34

3 5

3
6
1
4

1 5 6
4 2

1
4 2

2 5
6

6 1 2
3 5 3
5

1
4

3
5
6

35
36

1
4

37 38
39
40

2
4

2
4

5 6

1
3 5
6 4

1
3 5

1
3 6
5 2 4

2
4 3

1
5 3
4 6 5 3

2
1
4

41 42 43 44 45 46

2
6

1
6

2
4 5

1
3 4 2
6

1
5 3

1 2
3 4

1 3 2
6 4 5

41 48

1
2

6 1

Note: Blocks of the design should be be generated by random allocation of applied in a random order. Improved treatments to symbols followed by randomization and further replicates can random ordering of rows for each block of the design.

HALL TESTS

137

TABLE 4. 7 TREATMENTS: 56 REPLICATES IN 4 BALANCED BLOCKS OF 14 CONSUMERS

Samples
1 2 3 4 5
6 7
8
9

1 6 4 3 7 1 5 2 5 3 7 4 ~2 1 6

2
5 6 4 2 7 1 3 6 2 1 3 7 5 4

4 1 5 6 3 2 7 4 4 7 5 2 1 6 3 3 5 5 2

5
3 2 7 5 6 4 1 1 6 3 5 4 2 1 1 2 6 4 1
3

10 11 12 13 14

4 3 2 1 5 6 1 1 4 2 6
3

7 5 1 1 4 2 6 4
3 1

1 1 5 4 3 2 6 3 1 6 1 5 4 2 4 3 1 5 2 6 7
5

2 1 1 6 4 3 5 2 5 4 1
6

3 7 2 4 7
3

15 6 5 6 1 6 1 1 7 3 1 1 18 1 9 5 3 2 1 20 21 4 2 22 2 4 2 3 1 24 4 3 2 5 1 26 5 6 27 6 1 28 3 5 29
30 31 32 33
34

6 4 7 1 3
6 2
5

5 7
5

1 5 6
6

1 2 2 3 5 4 6 5 4 7 3 1 2 7 2 1 6 5 4 3 5

7 1 1 6 7 4 1 3 5 2 6 6 1 4 2 3 5 7 1 3

1 6 4 3 2
3 1 2 5 4 6 7 2 4 5 1 7 3 6

4 6 3 1 2 1 2 3 5 6 1 4 1 3 6 2 7 4 1 5 3 2 4 6
1

3 1 5 2 4 1 4 6 1 1 2 3 5 4 3 7 5 2 6 1 4 5 1
3

35
36

37
38
39

40 41 42 4 44 4 46 41
48 49

1 1 3 2 6 5 4 5 1 6 3 1 2 4 3 4 5 5 2

5 2 6 4 1 1 3 1 5 3 4 6 7 2

1 6

1
2 3
5 4

1
6 3

2
3

7
5 6

7 6 1 4
2 5 3

50

1 2
3

51
52

1 3 5 4 1 6

4 1 1 6 7 5 2
3

4 5 4 2 3
6

7 2 1
5

1
5

2
6 3

6 7
5

4 2 1 6 3

5 54 55
56

4 3 1 2

7 1

4 2 3 1 1 5
6

1
6

7 4 2 5

138

H.J. MACFIE, N. BRATCHELL, K. GREENHOFF AND L.V. VALLIS

TABLE 5. 8 TREATMENTS: 48 REPLICATES IN 6 BALANCED BLOCKS OF 8 CONSUMERS

Samples
1 2

3 8 1 5 2 4 3 7 6 5 1 2 4 7

4 7 4 2 1 3 8 6 5 8 4 5 2 7 4 3 6 1 1 2 6 8
3

5 1 6 8 7 5 2 4 3 3 5 4 1 6 8 7 2 7 8 4 2 5 3 1 6 8 2 1 4 3 7 5 6 5 4 7 2 3 1 8 6 3 7
2 5

6 2 7 3 8 6 5 1 4 4 2 1 6

8 3 2 6 5 1 4 8 7 2 6 7 3 4 1 5 8 8 1 3 7 4 6 2 5 2 4 3 6 7 5 1 8 3 5 4 8 7 2 6 1 7 8 4 2 3 6 5 1

I
5 8 4 3 7 6 2 1 9 1 7 6 2 5 4 4 3 2 8 7 2 2 5 6 5 2 3 1 1 4
8

4 5 7 6 2 1 3 8 7
8

6 3 1 4 8 7 5 2 6 3 8 5 2 7 1 4 4 3 8 5 7 1 6 2 7 5 6 1 8 2 4 3 8 2 1 5 6 4 3 7 2 4 7 3

10
11

3 8

1 13 I 15 16
17

1
6

3
8 6 5 4 7 6 2 8 3 1 3 7 4 5 6 8 2 1 2 1 6 4 8 7 5 3
5

8
5 3 7 3 6 1 4 8 2 5 7 5 1 8 3 2 4 6 7 6 8 2 3 1 5 7 4 4 1
8

1 19 20 21 22 23 24 25 26 27

5 1 6 4 8 3
6 8 5 2 1 3 7 4 4 7 3 1 5 6 2 8 6 3 5 1 8 7 4 2

5 7 4 4 6 7 8 5 1 3 2 7 3 5 6 4 8 1 2 8 6
1

28
29 30
31

32
33 34

1 3 2 7 4 6 8 5 1 6 8 7 2 3 4 5
1

35 36 37 38 39 40

41
42 43 44 45 46 47
48

5
6

2
3

8 4 2 7 3

6 1 4 8 7

4 7 3 2 5

6
8

1 4

7 2 5 3 6

5
1 6 8

Note: Blocks of the design should be applied in a random order. Improved randomization and further replicates can be generated by random allocation of treatments to symbols followed by random ordering of rows for each block of the design.

HALL TESTS

139

TABLE 6. 9 TREATMENTS: 54 REPLICATES IN 3 BALANCED BLOCKS OF 18 CONSUMERS

Samples
1

2 2 4 5 3 8 9 6 1 1 2 4 6 9 8 3 1 1 5 8 6 9 1 2 5 1 3 4 7 3 8 6 5 2 4 1 2 9 1
6

3 4 3 1 6 1 8 9 5 2 1 2 3
6

4 9 8 3 1 2 1 5 4 6 8 7 1 2 4 5 3 9 6 3 7 2 6 4 1 8 9 5 6 8 7 1 4 9 2 5 4 3

6 1 2 1 4 9 6 3 8 5 4 3 9 8 7 6 5 2 1 9 8 4 7 5 6 3 2 1 1 7 6 5 2 3 9 4 5 8 7 4 1 3
6

8 6 9 4 8 1 5 7 2 3 7 5 2 4 3 1 6 8 9 2 3 5 8 1 7 9 4 6 5 6 1 4 9 8 3
2

9 5 1 8 2 6 3 4 9 7 3 6 8
'1

I
I 2 3
4

5 6 7
8

9
10 11

12
13
14

15
16

17 1s
19

8 7 6 5 4 2 1 3 9 9 8 5 1 2 7 4 6 3 7 1 3 5 9 4 6 8 2 8 9
1 3

7 5 9 1 3 4 2 6 8 6 9 7
5

3 6 2 9 5 7 8 1 4 5 1 4
3

9 4 8 5 ? 6 5 8 4 3 2 1 7 9 3 2 9 8 6 5 1 7

1 8 2 3 4 1 4 1 2 8 9 5 6 3 9 4 2 3 7 1 6 8

6 2 9 7 8 5 2 6 9 7 3 4 1 8 2 5 4 9 8 6 7 3 1

5 9 1 4 2 4 9 1 3 6 8 2 5 7 4 1 5 2 3 1 8 9 6 5 1
? 6

20
21

22 23 24 25 26 27 28
29

30 31 32 33 34 35 36
37 38 39 40
41

7 1 4

I:
I 8
5

4
3 6

2 9
8

42 43 4 4 4 5 4 46 7 47 1 48 9
49

9 1 7 6 2

4 2 3 9 7 3 8 2 8 6 1 9 5 3 4 7 5

8
2

5 1 1 9 3 2 4 7
8

7 5 4 3 6 1 1 4 5 8
6

9 1 5 4
1 3

6
2 3 9 8 7 4 1 5 6

3
8 2 5 9 1
6

8 2 6 5 1 8 3 1 4 9 2
6

8 5 9 2 8 9 3 4 5 7 2 6 1

4 1 4

3
1

6
1 9 3 2 1 5 8

50 51 5 53
54

2 4

5
2 8 4 7 9

6 2
8

9
6

5 3

1 5

3 1 9
2

4 2 9 8 3 9 5 2 6 7 1 8 3 4

Note: Blocks of the design should be applied in a random order. Improved randomization and funher replicates can be generated by random allocation of treatments to symbols followed by random ordering of rows for each block of the design.

H.J. MACFIE, N. BRATCHELL, K. GREENHOFF AND L.V. VALLIS

TABLE 7. 10 TREATMENTS: 50 REPLICATES IN 5 BALANCED BLOCKS O F 10 CONSUMERS

Samples 1

3 9

5
3 8 1 2 10 1
9 5 6

10

1 2

2
8 9 5 7 1
3

3
4

1 3 5
6

5
10 3 8 4
6
2

5
6
7 8

2 10 4
9

4 9 2 1 3 8
7

6
3
4

9
5 10 1 8
2

10 4 5 7
6

10 1 9
7

5
7 1
8

8 4 6 10

6 2
8

4 9
2 10

1
3 5 7
2

9 10

4
10

7 8
8

1 7
7

6 0 5
4

4 1 9 8 4 5
2 10 6

3 8 9 1 6 5 10 8
2

4 5 3

3 6 3
2 7

9 9
4

11 12 13
14

10 6
2

15
16

5 3
9

1 5 9
6
3

3 4 2
9 5

7 9

3
1 6 5 8 2
10

2 10 3 6
7

17 18 19 20 21 22 23
24

4 7
8 1
7

10
4

8 1 10
6

1 9
4

7
3

5
8

4 7 3 1
9

5 8 6 1 10 7 3
2

0
4 8 1 9

1 7
8

10
6 5 3
2

9
4

7 8

6 5
4 6 5 2

6
3

10
7

25 26 27 28 29
30

6 1 8 5 10 9 3 4
1 7 8 6 2

9 3
4 5 6 2 1

9 4 7 10 2 1 8
5

3 5 8 6 2 1 4 1 10
9

9 8 1 5 6 2 3 4
7

2 4 3
7

10
9

10
5
2

1 8 5 6
7

1 1 4 10 9 3 8 5 6
2

0 1 2 8

9 10 3

5
6 9 3 4 7
4

1 8
1
7 0 9
3
8

4
7 6 2 1
8

31 32 33 34 35

2 4
10 1

6 3 9
5

9
5 2

3
6 9

36 37
38

5
4

39
40

9 3 10
4

8
7

1 10 1 4
2

7 1 8
10

4 10 6

8 6
3

10 1 7
8

9
5 2 4 8 7

0 1
7

6 3
9

8
3

8 9 4
2

4 6

7 1 9

4 2 10 5 9 1

5 9
2 3 6

9
3

5 6
8

10 4 1
2

4 6 Z 5 3 1 4 9
10

5 7 5
6

6 1 10

3
10 5 3 1 4 2 6 7 9 8

41 42 43
44 45 46

8
9

1
4

1 7
8

2
9

7
3

3
10 7 8 9 6 5 1 2 4

47
48 49

6 10 7 1
2

2 5
3

5
3

7
9

1 8
6

9 6 0 3
4

6 10 7 8 4
5

1 4 2 5 10
8

10 7
8

50

3 5

10
6

7 10

5
2

1 3

6 9

8 7

2 3 4 1

HALL TESTS

141

TABLE 8.
11 TREATMENTS: 44 REPLICATES IN 2 BALANCED BLOCKS OF 22 CONSUMERS
%moles
1 2
3

10

11

1
1 2 3
4

7 10 8 11
3 4

11
2

11 3 7 8 5 1 2 9 10 6
4

8 9 10
3 2

5 8 7
4

5 6
7 8 9 10

8 10 7

6 9
1

5 1 6
4

11 12 13 14 15 16
17

5 2

9
6

5 4 6 10
3 1

5 1
3

7 5
2

ie
19 20 21 22 23 24 25 26 27 28 29
30

7 11 9 2 8
2

8 9 4 7 2 10 11
3

10 6 11 8 1 9
3

11 4 6 5 7 1 1 6 9 8 11 2 5 4 10 3 7
8

11 6 9 10 2

3 1 2 10 9
8

11
4

5 7

1 6 9 2 1
3

0 1 5 6 4 9 10 8 3 2 1 1 8 3 11 6 1
7

1 5 1 9 6 0 8 7 1 3 4 0 2
3

7
5

1 2
3

9 4 6 1 5 2
3

11
4 7 8 5 2 9 10 7 4
3

1
3 7 11 4
9 2 5

11
6 9 1 2 11 7 10 6 5
3

10 9
8

1 8 0 7
3

11
8

0
8

7 1 9 4
3

8 3
6

10
5 6
2

1 10

8
4

1 6 8 1 5

0 2 4 9

1 5

4 5 8 9 1 1 7 6
1 5

5 6 1
2

9 7 4 1
9

7
3

1
8 7
6

0
2 8 9
3

10 11 8
6

9 0 1
5

5 8
4

1 1

0 5
2

1
5

7 4
3

4
5

31 32 33 34
35 36
37
38

9 1 11 1 2
5

4 1 9 9 7 4
6

6 1 7
6

1 1 1

9
3

2 6 1 1 0
5

1 1 1

3 1 2 9 6 1 7 0 5 1 8
2

0 1 7 9 4 1 2 8
3

6
3 5

1 1

5 6 7
8

2 7 4 1 1 9 0 8
4

7 1 1 6 2 1 6 11 4 2 10 9 5 3 7 1 0 9 8 1 4 8
5

9
6 3

1
8 7 0 2 1

4 1 5 8 6 0 1 7
3

6
9
4

0
3

39
40 41 42 43

8 10 4
3

44

9 7 6

1 1

3 8 5 2 1 0 1

4 0 7 2
8

1 2
3
6

1 1 5

4 7 8 9

7 1 3 0 6
9

1
1

2 1
5

1 9 8 7
5

10 3 1 1 1 5
4

2 5 7 8 9
11

9
6

0 1
4

4 8 2 9 7 6 5 1 1
3

2
3
4

1
1 1 0 1 6 7 9 2 5
8

0 3

8 2 7

4 0

Note: Blocks of the design should be applied in a random order. Improved randomization and further replicates can be generated by random allocation of treatments to symbols followed by random ordering of rows for each block of the design.

H.J. MACFIE, N. BRATCHELL, K. GREENHOFF AND L.V. VALLIS

TABLE 9. 12 TREATMENTS: 48 REPLICATES IN 4 BALANCED BLOCKS OF 12 CONSUMERS

Samples
I

1 6

10

11

12

1 2 3 4 5 6 7 8

1 3 4
8

7 0
8

9
10 11

9 1 2 7 5 6 1 1 1 2
10
2

9 4 1 5 3

12
13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

12
9 12 3 11 8 1 2 6
4

5 2 7 8 3 4 1 0 6 2 9 1 1 1

1
1

3 2 1 4 1 9 2
6
8

2 1
6

3 7
8

1 1 7

8 5 9 2 1 1
4 3

0 9

5 7 6
3 1 2 2 1
8

7 1 1 4 1 5 1 2 1 0 9
7 4 1 9 5 1
6

1 5 0

0 6 1 2 0
6

1 4 2 7 4 6
5

4 6 1 1 6 1 2 1 0 3 1 9 1 8 1 2 7

1 4 2
5
7

1 1

1 0 2
3

1 2 1 7 8 2 1 0 0 5 1 2 1 2 6 8 9 1 1 1 1 4 1 1
5 7

1
3

1 2 0 5
4

2 9
6
8

9
8

3
6

4
3

5 1 10

5 1 1 1 8 1 2 9 3 1 2 1 0 4 1 2 3 6 1 2 4 6 1 2 7 9 1 10 5 8 7 1 1 1 4 1 1 1 1 2 1 2
8

3 0
6

6
5

1 1

2
7

1 3 0
8

1 1

2 7 8 4 1 2 1 9
5

5 7 4 2 9 1
3 8

0
6

2 4 9 6
7

2 1 9
8

0 1 5 1 2 8 1 3 1 1 1 9 1
1

1
9 1 2
1

1 4 9 7 2
8

2 3 0
8

1 1

0 2
3

2 1 7 5 9 1
4 6

4 9 2 1 1
5 8

1 1

1 1
8

2
3

10 7
6

10 7 5
4 6

4
8

1 3 2 5
8

2 3
10
3

2
5 10
8

6 1 7
3 8

1
6

28
29

30
31

9
11
2

32 33 34 35 36

10 5

6 5 3

! 4

10 9
7

12
9

2 0 9 4 7 8 1 5 1 2 1 3 1 7
3

5 2 1 0 2 3 8 1

7 1 3 5 4 1 2 9 2 8
6

2 7
0

2 4
6

2 5
9 1 2 3
8

1 1

1 0 5 4 3 7 2 2
6

0 1
2

2 1 6 1 3 4 1 8 1 9 1 5

3 0 9 2 1 7 5 1 2
5

1 1

6 1 0 1 9 4 7 2
3
6

2 9 11 7
5
6

1 1

7 6 1 0 4
6

4 1 12
8

7 12 2 11 1
6

3 9
10 11 12 9 5
4

37

5
8 7

38
39

4
8

4
6

40
41 42 43 44 45 46 47 48

10

2 11
6

1
3 1

2 3 12 1 0 11 4 9 6 1

0 1 1 2 5 2
8

1 1 9

1 8 7 9 3 1 4 6 2 1 1 2 8 1 5 1 0 1 1

2
6

5 1 9 0
4

1 7

1 9 5 4 8 1 0 3 6 7 1 2 1 2

4 1 10 2 1 1 1 2 9 1 7 1 6 3 1 5
8

2 12
6

1
8

1 1

7 2 4 0 9 1
3
6

0 1 1 7 5
8

1 7 9 3
8

5 1
8

2 2
9

1 1

2 5 2
4

9 4
11 10
7 3

7
3

1
6

8 2

Note: Blocks of the design should be applied in a random order. Improved randomization and further replicates can be generated by random allocation of treatments to symbols followed by random ordering of rows for each block of the design.

HALL TESTS

143

TABLE 10. 13 TREATMENTS: 52 REPLICATES IN 2 BALANCED BLOCKS OF 26 CONSUMERS

Samoles

4
2

5 9 12 6 1
3

9 7 9 10 2 13
6

10
a 1 1 9 1
3

11
1 0

12
3

13

I
I
2
3 4

11
5

13
5

5
6

7
8

9
10 11

12
13 14

9 6 1 7 13 11
7 3 1

15
16

17
I8
19 20

11 9 10 6 8 9 10 8 3

9 8 1 12 13 2 6
7

4 8 2

6 11 7
1 4
8 9 2 2 0 1 3 3

10 6 4 1 12 8 11 7 9 3 6 1 13 8
5

9
13 1 0 3 5

11 4
5

7 10
8

1 1 1

3 3 4 6 1 8 7 1 2
5

6
1 4
2

11
1 3 4 8 12
4

12
6 11 1 7 3

3 10 12 4
6

10 1
13 3 2

21
22
23 24 25

IE 1
12
110
8 1 0

12

11 4
9 5
1

11

26 2 7 1 1
2

4 11 12 7 10 9 3 2

1 2 5 9 1 0 7 1 3 8 12 1 2 4 9 13 11 12 10 11 1 2 6 6 8
3

9 12 5 1 0 2 1 0 1 1 8 4 7 2 6 4 3 11 1 0 5 1 3 6 8 9 5 1 12 3 4 7 9 1 3 1 1 2
4

12 6 9 1 7 8 3 2
5

11 5 7 1 1 5
2

13 11 10 5 2 2 1 0 1 3

9 12 7 6 3 8 1 4 1 8 6
9

I 7 3 5 11 12 2 4 5 2 7 1 1 0 12 9 11 0 9 7 6 3 1 3 8 8 4 1 3 2 12 5 1 3 2 4 1 1 6 9 1 3 3 8 3 5 4 2 11 9 12 11 5 12 1 0 6 7 6 11 9 3 1 0 1 1 7 1 0 4 8 1 3 2 1 3 4 0 8 1 3 1 2 2 5 1 7 9 6
6 3 1 1 0 8 1 3 9
5

29

7 3 1 1 3 1 2 1 1 1 1 1 6 30 9 2 8 1 3 7 4 1 1 6 31 32 4 8 6 9 3 1 1 1 0 7 1 33 34 2 1 3 9 5 5 1 2 1 3 1 0 35 36 8 9 4 2 3 7 1 3 5 2 1 2 3 8 1 2 1 0 5 3 1 39 6 4 1 8 40 6 1 4 7 41 8 4 9 6 42 5 1 3 1 2 2 1 4 3 1 2 5 1 0 1 3 44 4 6 8 1 4 5 1 0 1 2 3 5 1 46 7 1 1 1 3 47 1 7 6 1 1 48 3 1 0 1 1 1 2 49 2 9 1 3 8 50 2 4 1 9 8 5 1 1 1 3 7 1 0 5 2 1 3 2 5 9 1
7

0 5 3 4 1 1 2 8 2 6 9 3 7 8 2 0

6 1 2 7 1 3 4 1 0 5 6 8 3 2 7 1 5 1 2 4 3 9 1 3 1 1 0 8 1 1 2 9 1 1 1 1 9 1 1 3 9 3 3 2 1 1 9 7 2 1 1 3 7 6 1 0 4 4 3 8 7 5 1 5 4 1 2 3 6 5 1 1 2 6 2 8 1 0

9
4

1 8 1 2 9 1 3 6 1 0 1 1 5
3

1 1 1 1

0 2 5
4

7 2 3
2

1 1 1 1

6
8 7 4 9 3 2 5 1
7

1 1 1 1 1

7 2
3

1
1

7 8 9 1 2
2

1 1

3 7 2 1 1 3 0 1 4
8

1 4 1
6

1 1 1

1 1

0 3 6 1 5 4

1 1

3 1 8 9 6 0 2 4
3

3 9
5

8 0 3 2 7 1 5
6

2 1 3 1 3 2 1 8 1 1 0 7 6 5 1 2 3 1
6

2 8 3 1 5 0 9 7 6 3 1 4 2
5

0 1
6

1 1

4 0
8

2
4 2

9
2

2 2 1 7 3 6

1 1

4 3 0 8 1

3 5 9 7 1 2 1

1 1

3 8 1 3 9 7

144

H.J. MACFIE, N. BRATCHELL, K. GREENHOFF AND L.V. VALLIS

TABLE 11. 14 TREATMENTS: 56 REPLICATES IN 4 BALANCED BLOCKS OF 14 CONSUMERS

Samples

4 4 2 9 3 2 3 8 0 4 1 1 1 1

5
1 9 0 1 4
8

7 9 1 5 6 1 2 4 4 3 2 7 0 3 8 1 3 4 9 5 1 2 0 3 1 6 8 2 4 8 9 4 2 3 5 1 2 1 1 4 0 3 6 4 2 1 0 6 3 5 3 8 2 4 7 9 1 1 1 1

9 1

10 7 8 3 2 5 9 0 1 4 6 4 2 3 1 9 3 4 8 7 2 6 1 1 0 5 4 2 3 1 7 0 5 4 3 1 9 3 2 8 6 4 2 2 1 1 1 8 7 5 1 9 2 0 1 6 3 3 4 4 1 1 1 1

11
6 3 8 0 1 2 9 3 5 4
2

12
5 1 2 8 0 1 3 9 2 1
3

13

14

I
1 2 3 4 5 6 7 8 9 10 11 12
13

1 1

3 1 2 1 4 4 1 3 1 1

1 1

1
1

3
9 3 1 2 8 1 2 0 6 4 5 8 1 3 4 0

14

1 1 1 4 12 4 5 1 0 4 2 1 5 9 1 1 8 1 3 6 1 9 2 8 10 3 10 5 2 1 1 1 4 1 1 8 1 4 8 1 3 2 5 0 3 6 7 1 4 13 2 1 4 1 1 12 3 9 1 6 9 4 5

1 1

1 1

5
1 1 1 7 6 2 0 5 2 3 8 4 9 1 1 3 6 4 7 6 3 1 7 8
3

3 2 6 4 5 3 2 7 5 2 1 1 8 3 1 4 6 4 9 0 3 2 7

1 1 1

2 4 1 2 8 6 7 3 3 0 9 4 5 1
3 4 8 1
6

1 1

1 1 1 1

1 1 1

8 2 4 4 7 1 5 6 2 3 1 3 0 9 6 2 0 5 9 4 3 4 2 8 1 7 3 1

6
7 3 9 4 1 2 0 3 8 5 4 2 1

1 1 1

3 0 2
5

1
1 1 1

1 1 1

1 1 1 1

1 1 1

6 3 1 4 7 1 4
8

1 1

0 5 4 7 3 4 6 1 8 9 2
2

7 1 4 3 4 9 3 2 7 1 5 4 6 2 1 0 8 3 6 7

4 6 4 7 6 2 0 4 3 9 3 5 4 8 1 1 2 3 11 13
6

1 1

9 2 1 1 6 2 3 9 4 1 0 3
4

1 1

1 3
4 9 3 4 1 6 7 2 8 2
0

15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
30
31

1 1

1 1 1

1 1

1 1

2
1 1

5 2 9 1 7 4
6 1 3 1

1 1 1

1 1

1
1 1

1 1 1

0 2 1 3 5 2 9 1 4 4

1 1

1 1

1 1

2 1 1 6 1 4 3 8 2 9 3 5 4 0

1 1

1 1 1

1 1 1

1 1 1

1
1

1 1
1

1
1

9
8 2 4 1 0 7 4 1 3 2 5 6 3 1 5 4
3

2
1 1 4 1 3 1 4 9 3 8 7
6

2 8 5 0

3 5 1 2 4 2 14 6 8 5 3 10 13 11 9 1 1
4 3 8 2 1 5
0

32 33 34 35 36 37 38
39

40 41 42 4 3 4 4 4 5 46 4 1 48 49 5 0 51 52 53 54 55 56

1 2 1 0 3 14 9 6 2 8 1 3 2 1 0 13 9 1 1 4 1 0 1 3 1 2 11 4 1 5 6 4 5 6 1 4 3 1 2 1 8 7 2 1 1 1 8 1 3 9 1 1 1 0 1 3 9 1 4 2 3 1 2 5 6 8 4 1 7 1 8 7 5 9 4 3 2 1 0 4 6 5 6 3 4 8 2 1 1 9 4 0 2

1 1

1 5 0 2 6
3 8

2 3 1 1 2
8 6

1
9 11 14
10

5
2 4
1

1 1 1 1

2 5 4 9 0 1 4 2
3 4 2 1 5 6

1 1

5 8 9 14 1 6
12

1 1

1 4 9 3 2 4 9 4 1 8 0 7 1 3 4 2 6 2

1 1 2 1 1 1

9 1 3 4 5 0 3 9

1 1 1 1

2 8 5
3

1 1

2 0

5
8 1 1 2 5 6 4 3 9 1 1 0 7

12 4 0 1 9 4 2 1 4 1

12 10 1 14 9 8

13 2 4 1 3 11 6 3 4 2 2 1
8

1 1

2
1 1 1 9 6 4 2 1 3 5 4
8

1 1

1 1 1

1 0
7

1 1 1

1 1 1

1 1 1 1

7 4 1
8

1 1 0 2

1 1

6
1 3 4 5 1

1 1 1 1 1 6

3 1 1 1 2 1 3

1 1

9 0 3
2

8
7 4

5
3

1 8 2 0 3 9 1 4 2 1

2
1 1 1 3 4 1

1 6 1 3 1 2 3 8 5

5 4 3 7 1 0 9

1 1 1

9 2 1 3 1 6 4

HALL TESTS

145

TABLE. 12 15 TREATMENTS: 60 REPLICATES IN 2 BALANCED BLOCKS OF 30 CONSUMERS

Samples
1

3
1 2 1 14 11

4 10

5
5
1

6 13 15 14
5

9
3

l
8 3

2 9 12 11 6 15
1

5
4 14 1

2 3
4 5
6

2 6 9
8

6 10 12
3 1

13
4

5
4 0 3 1

11 2
8

14

12
6

15 4 3 9
2

11 5 13 12
8 1
1

15 1
8
1

1
9
3

14
8

6 3

7
8 9
10 11

13
15 9
5

10 15
3 9 8 4
5

1
2

15 1 12 1
14 5

1
9 4 2
8
1

6 13
8

11 4
8

12 13 I4
15

3 4
1

3
6

16 17
18 19 20
21

15 14 6 11
8

2 1 9 2
3

13
8

9 12
1 0

10 5
3

1 14 12 11 10 3 13
1

11 10 12 9 5
14

2 10 14 15 4 9
3

10 11 9 12 14 6 3
2

11 10 9 14

15
5 7 6 4 13 12 1
8

2 13 5 12 4
8

6 1
4

2 15
12

12 4 5 10

13 11
5 2

10
3

4
14 3

1 2 14
8

13 9
6 8 3
5

11
1

10 11 6
1

12 10 9
15

13
5 10
2

9 6

11 15
1

1
13 4 5 2 12 13 6

5 13
4

12

22 23 24 25 26 27 28 29 30
31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
48

1 4
3

14 9 15 12
8

4 9 14 11 12
1

9 5 15
2
1

1 11 13
1

4 1 13 9 14 15 3
10

11 14 12

12 15
10
4

11 15 2 5
8

1 11
4

2 5 1 6
1

15 14

14
1 1

1
8

15
3

13
8

4
3
1

9 13 6
15 12 10 14 2 3
8 1

a
5

3 14 11 2
8

12 1 10 2 13
9

10 2

9 15

11
8

6
1

5
3
1

5 6
3

1
6 2 15 13

4 2 9
10

13
8

13 10

10
6
8

5 11 6 2
13 3

1 15 2 13

6 11 3 2
1

14 6 12 13 10 9

6
4

10 4 15

7
8

13
12 9 4

1
1
8

14 5 11
6

13 14 1 15 4 12 12
10

10 1 2 12
5

10 11 15
8

13 6 9 3 10 14 1 13 2 5 6 15
1

1 2 15 10 12 11 6
4

15 10 1 11 2 13
5

9 11
4

1 3

14 4 11
3 3

9 1 14 4
1

12
3 8

12 9
8 14

10
15 12 8 4 13
2

9 13
4 1 15 10
1

15 1
2 9 12 5
1

9 2 10 6
8

5
1

12
8

15 11 6 10 5
3

9 14
3

11
4 1

5
2 11
14

10
14
7

1 11
8

12
14

1
3

2 4
6 3

5
8

5
4 3

14
8 3

11 13

14 11 5 6
3

4 12 15 14
1

1 11
3

13 1
4 9
1

11 15
2

10

12 6
9

10 9
6 14
4

49 50 51 52 53 54 55
56 57 58 59

11 12 2 6 15 5 1 6
2

14 4 10
3

12 2
8 14

1
9 9
8

5 11
8

13 10 4
9

1
8

13 9 14
8 4

3 10

15 11 5 12 6
2

2 3 10 15 6
1

2 12 15
4 5

15 6 13 10 11 5 9
3

6 2 15 9 11 12 13
3

1 6 12 4 10

13 14 9 10 2
3

11 2
1

13 15 1 6
5 10

15 9

1
14 15 13 12
5
8

12
3
8 5

9 15
4
8

14 1 12 9 6 4 11 13
B

13
1

1 13 15 3 1
8

4 1 5

13
6

11 1 2
8

14 2

10
13 6

10 5 15
3

15
12 2 12 11 5
14

14
5
14

5 1
1

10
4
14 8 1

15
5

11 9
8 1

9 4 1
1 3

11 10
1

1
4
I4
6

13 12 10
8

14
8

6 13 12

3 15
4

2 12 11 9

6
11
4

13 3

13

10
1

1
11 14 5 12 4

60

11 1 12

3 14
1

9 1 6

12 15 2 14
8

2 14
15
4

9 10
7

13

9
3

11 14

4 1

12 1
2

2 5 9 1 11 13 12 15 6
3

13 3
14

15
11
1

10 11
4

2
3

1
6
1

1
6 5 12 15
4

1 10 15
4

2
3 8 4

3
8 1

2 13 12 6
5

9 14 12 10 11 13

12 1
8

4
5
14

1
6 2 5
8

14
8

2
10

11

13 9 6

10
5

15 9

1 9 10 11 5 13 15

15
2

3 10
8

13

Note: Blocks of the design should be applied in a random order. Improved randomization and further replicates can be generated by random allocation of treatments to symbols followed by random ordering of rows for each block of the design.

146

H.J. MACFIE, N. BRATCHELL, K. GREENHOFF AND L.V. VALLIS

TABLE 13. 16 TREATMENTS: 48 REPLICATES IN 3 BALANCED BLOCKS OF 16 CONSUMERS

Samples

6
4

I0

I1

12

13

14

15

16 9

2
3
4

5 6 7
6
9 10 11
12 13

15 3 4 2 10 8 5 1 9 14
6

13

1
6

11
8 12 4 6

1
15 7 2
3

10 I4 13 4 16

5
11
7 8

10 9 6 16 8 3 15 13 14 11 12

5
8

16
4

a
14 11 1 3 9 10 4 12 5
7

13
7 15 10 12 16 3 9 4 14

7 3
5 6

12 10

15
9

1
11 5 15 14 2
8

1
13 2 12 6 11 10 7 3 14 5
8

11 13 10 2 16
9 1

14 15

16
17

12 7 16 13 11 14 13 5 15
9 6 16

16 I4
5

10
9 9

2 9 12 15 3 2
9

5 2

1
4
7

11 2 1
6

12
15
8

15 16 13
6

14 1 11 15 14 12 2 5 16 6 4 13
6

2 12 10 4 9 11 14
8

3 13 9 6 7 16 4
6

6 2 9 15 11 7 4 8 16 1 14 13 5 10
3

2 13 5 3 16 1 7 14 4
6

3 12
7

14 15 16
6

13 9 14
8

11 16 14 10
7

7
1

5
2 4 14 12 3 11 15
10

12

e
2
16

12 5 9
1

I5
11 8 9 12 10
8 15 11 3 6 16 13 2
1

6 5 15 2
10

10
8

1
4 11
16 10

12 11 16 2 13 10
9 6
7

1 3 4 11 2 13
5 6 12 16
8

2 6 3 1 13 12 4 15
8

11 9 i3 4
5

16
8

9 5
16 1

10 3 14 15 7 8 1
10 5 11

13
6

7 1

16 19
20

21
22 23 24 25 26 27

12
3 8

1 8 4 13 15 3 11

7
6

13 12 6 2

14
8

1
4 7 10 5 15 9 11 8 14
3

7
6

2
7

26 29
30

1
4 10 11 12 2 11
13 6

14 5 12 2
16

31 32
33
34 35
36

10
8

10 1 16 5 4 3 13 15 11 12 13 1 5
16

4 14 3 8 2 5 11 13 10 7 15 16
9

15 2 16 5 4
7

4 10

3 14

7
15 3

1
16 13 15 5 4
9

9 12
3

12 9 11
3 6 10

1
14 12 16
8

3
6 2

16 14 16 7
8

6 12 1 16 3 15
6

10 9 7
3

14
8

1 13

1 15 16 13 5 7

11 2 5 13 9

15 14 12 4 3 1
8

10

5 12 4
1 9

5 5
8

14 1 10
8 3

11 2 7 12 6 8 11 14 2 5 4 15 10
9 6 8

10 9 4 13 11 7

1 4 8 2 13
10

1
15 3 14 2 3 11 12
9

14 5 11 15
6

14 15 9 12 3 13
6

3 5 13 11 7 12 2
6

14 1 16
9

16 2 4 9 5 13
1

8 10 4 15

37

38
39 40 41 42

16
8 4

6 10 12 16 13 14 15
1

2 12 6
14

10 1
1

2
9

43 44
45 46 47 46

5 14 3 9 15

11 4

11 3 10 15
8

13 12 4 5 9
6

1
2 13 8 5
9

4 13 9 14 12
6

6
9

15
12 3 4 8 14 5 10 1 13 2 9 11 16 6
7

2 7
14
1

10 4
6

7
15 16 10 5 11 9
1

4 2 3
1

1
15 14 4 11
7

3 10
15

11 12 4
1

15 I1 3 16 1
7

16 12 15
7

1
3 5

9 7 4

2 1 11

10 14

5 6 2 10

5 14 13

9 3 6 13 4 10 15
8

7
9

10 7

11 15 5
7 3 16

10
11 8

16 11 5 12

2 12 6 3 10 13 16
9

2 16 14 11 4 12
6

1
6 8 5

16

1
7

14 13 2

14 13 2
9

11 10 3 2 12 15
8

13 4 12
6

5 15 13

12 13 3

15
4 16

8 12 1

16 1 4 14 6

3 14 8 2

Note: Blocks of the design should be applied in a random order. Improved randomization and further replicates can be generated by random allocation of treatments to symbols followed by random ordering of rows for each block of the design.

HALL TESTS

147

EXAMPLES
There are a variety of ways in which these designs can be used. Several examples are listed below.

Case 1
Suppose a trial is devised to test seven competing products. From Table 4 we need at least one group of 14 consumers to produce a balanced design. Consumer 1 receives the samples in the order specified in row 1 of Table 4, consumer 2 receives the samples specified in the second row of the table, and so on. Subsequent groups of 14 consumers are allocated treatments after further randomization of treatments and rows of Table 4.

Case 2 In this example products are formulated in accordance with a (3 X 2) factorial design: factor 1 has 3 levels, factor 2 has 2 levels, giving 6 treatment combinations. The appropriate design is chosen from Table 3 as above. Case 3
Consider products formulated in the following strategy. There are five factors each at 2 levels, giving a 25 factorial design with 32 treatment combinations. This can be reduced to 16 or 8 samples using a half- or quarter-replicate design. An appropriate order design is chosen as above.

ANALYSIS
Frequently the data recorded from consumers are hedonic and intended for preference mapping. These designs ensure that over the whole experiment each treatment is assessed by all consumers, and over the whole design (experiment) occurs in each position in order of presentation and is preceded by all other treatments. The results for any treatment is not biased by, for instance, being most frequently in one position or frequently preceded by one particular treatment with a strong carry-over effect. It is often a useful practice to perform an analysis of variance to examine the relative effects of consumer, order of assessment and the treament and firstorder carry-over effects (termed direct and residual effects by Williams 1949). Two carry-over effects can be defined. The average carry-over effect of a treatment is its effect on the subsequent treatment averaged over all (subsequent) treatments. The specific carry-over effect is the effect a treatment has on each of

148

H.J. MACFIE, N. BRATCHELL, K. GREENHOFF AND L.V. VALLIS

the other individual treatments. Williams (1949) presents an analysis of the average carry-over effects. Such analysis is complex and will be discussed more fully in a subsequent paper and compared with an analysis of the specific carryover effects.

CONCLUSION
This paper presented a strategy for designing hall trials in which each consumer receives each of several treatments so that the number of treatments, the positions of treatments in order of presentation and preceding treatments are balanced over the whole trial. Such designs ensure that treatment effects or means are not biased by the treatments being frequently presented in a particular position or preceded by other treatments. The balanced nature of the designs allows analysis of variance to be applied to test the relative effects of the consumers, positions and preceding treatments.

REFERENCES
JONES, P.N., MACFIE, H.J.H and BEILKEN, S.L. 1989. Use of Preference Mapping to relate consumer preference to the sensory properties of a processed meat product (Tinned Cat Food) J. Sci. Food Agric. 47, 113-123. NUTE, G.R., MACFIE, H.J.H. and GREENHOFF, K. 1989. Practical application of Preference Mapping. In Food Acceptability, (D.M .H. Thomson, ed.) Elsevier Applied Science, London. SIDEL, J.L. and STONE, H. 1985. Sensory Evaluation Practices, Academic Press, New York. WILLIAMS, E.J. 1949. Experimental designs for the estimation of residual effects of treatments. Austral. J. Sci. Res. Ser. A. 2, 149-168.