Angela Schwering, Ulf Krumnack, Kai-Uwe Khnberger & Helmar Gust (eds.

Investigating Experimentally Problem Solving Strategies in Geometric Proportional Analogies

PICS
Publications of the Institute of Cognitive Science Volume 30-2010

ISSN: Series title:

1610-5389 PICS Publications of the Institute of Cognitive Science 30-2010 Osnabrck, Germany September 2010 Kai-Uwe Khnberger Peter Knig Sven Walter Thorsten Hinrichs

Volume: Place of publication: Date: Editors:

Cover design:

Institute of Cognitive Science

Investigating Experimentally Problem Solving Strategies in Geometric Proportional Analogies

Editors: Angela Schwering, Ulf Krumnack, Kai-Uwe Kuhnberger, Helmar Gust Institute of Cognitive Science, University of Osnabruck June 2008

Contents
1 Introduction 2 Background 2.1 Psychological and computational aspects of analogies 2.1.1 What is an analogy? . . . . . . . . . . . . . . 2.1.2 Stages in analogical reasoning . . . . . . . . . 2.1.3 Types of analogies . . . . . . . . . . . . . . . 2.1.4 Geometric proportional analogies: our focus . 2.2 Gestalt psychology . . . . . . . . . . . . . . . . . . . 2.2.1 Gestalt theory . . . . . . . . . . . . . . . . . . 2.2.2 Gestalt principles . . . . . . . . . . . . . . . . 2.2.3 Application of Gestalt principles . . . . . . . . 3 Experimental part 3.1 Motivation . . . . . . . . . . . . . . . . . . . . . . 3.2 Technical framework . . . . . . . . . . . . . . . . 3.2.1 General introduction to the Analogy Lab . 3.2.2 Lab elements and basic functions . . . . . 3.2.3 Experimental paradigms and programming 3.2.4 Measure to prevent multiple participation 3.2.5 External programs . . . . . . . . . . . . . 3.3 Pretests . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 First pretest . . . . . . . . . . . . . . . . . 3.3.2 Second pretest . . . . . . . . . . . . . . . . 3.3.3 Mini tests: HIT and Technologietag . . . . 3.4 Final experiment . . . . . . . . . . . . . . . . . . 3.4.1 Participants . . . . . . . . . . . . . . . . . 3.4.2 Procedure . . . . . . . . . . . . . . . . . . 3.4.3 Materials . . . . . . . . . . . . . . . . . . 3.4.4 Results . . . . . . . . . . . . . . . . . . . . 3.4.5 Comment classication . . . . . . . . . . . 1 4 5 5 5 6 12 17 19 19 20 25 27 27 28 28 30 32 40 42 47 47 52 88 97 97 98 99 101 133

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . . . . . . . . . issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

4 Closing discussion Bibliography A First pretest A.1 List of analogies . . . . . . . A.2 Analogy combinations . . . A.3 Participant information . . . A.4 Test environment . . . . . . A.5 Results by analogy . . . . . A.6 Feedback . . . . . . . . . . . A.7 Suggestions for improvement

134 138 139 139 142 142 145 146 153 155 157 157 160 160 161 163 165 168 168 169 176 178

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

B Second pretest B.1 List of analogies . . . . . . . . . . . . . B.2 Stimuli combinations . . . . . . . . . . B.3 Example randomization . . . . . . . . B.4 Randomization of Check/Radio options B.5 Participant information . . . . . . . . . C Mini tests: HIT and Technologietag D Final test D.1 Demographics . . . . D.2 Stimuli . . . . . . . . D.3 Solution classication D.4 Solution analysis . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

Foreword The work presented in this technical report was developed by the study project Psychological and Computational Aspects of Analogy. The Study project was divided in an experimental and a formal part. In the experimental part, dierent solutions for ambiguous geometric proportional analogies were investigated. The web-based experiment reached 238 participants and gave us valuable results for analyzing dierent perceptions and solution strategies of participants. In the formal part, the study project developed a computational approach to formalize dierent perceptions of geometric gures and solve these ambiguous analogies. This part was inspired by the work from Mehdi Dastani and his colleagues, who developed a language of perception for proportional analogies. This technical report covers only the experimental results of the study project. It gives some details about the background from analogy research, describes the motivation from Gestalt Psychology and explains the experimental design. Two events in Osnabrueck - the Hochschulinformationstag and the Technologietag were used to carry out the experiment with a small number of participants. It was followed by a comprehensive web-based experiment. In the end of this report, we discuss our ndings with respect to possible explanations for dierent perceptions and dierent solution strategies. The members of the study project were Clemens Bauer, Judith Degen, Irena Dorceva, Maxim Haddad, Martin Schmidt, Rolf Stollinski, Kae Sugawara, Martesa Tantra and Radomir Zugic. It was held by Angela Schwering, Ulf Krumnack, Kai-Uwe Khnberger and Helmar Gust at the Institute u of Cognitive Science, University of Osnabrck in the summer term 07 and u winter term 07/08.

Chapter 1 Introduction
Clemens Bauer, Judith Degen, Irena Dorceva, Maxim Haddad, Martin Schmidt, Rolf Stollinski, Kae Sugwara, Martesa Tandra & Radomir Zugic Institute of Cognitive Science, University of Osnabrck u

Analogy-making in the scope of human perception underlies a high-level cognitive process that transfers information from a particular entity (the source) to another (the target) by drawing a comparison in order to show a similarity in some respect. In general, it can be inferred that if two entities share some similarities, they probably also share many others as well. Analogies frequently occur in everyday life and thus play a central role in human cognition such as in reasoning and learning processes, or even in education. Furthermore, humans often use analogies to explain and understand new phenomena by resorting to already known facts from a familiar domain based on past experience. While a multitude of analogy types are often built across dierent domains, (geometric) proportional analogies have the general form (A:B)::(C:D) (read A is to B as C is to D) whereby A and B (the source domain) as well as C and D (the target domain) comprise elements from the same domain. In the context of geometric gures, the main task for humans amounts to nding a suitable element for D such that the same structural relations between the source elements A and B and the target elements C and D hold. However, the resulting solutions often dier signicantly among humans due to the possibility of perceiving dierent conceptualizations of the same gures within a given geometric analogy. It can therefore be claimed that analogy-making is a highly sophisticated cognitive process. 4

Chapter 2 Background
Clemens Bauer, Irena Dorceva & Martesa Tandra Institute of Cognitive Science, University of Osnabrck u

2.1
2.1.1

Psychological and computational aspects of analogies

What is an analogy?

Analogies are partial similarities between dierent situations that hold further inferences (Bechtel and Graham [1999]). In other words, an analogy is a type of similarity in which the same arrangement of relations holds across dierent objects. Analogies thus get the core of correspondences across dierent situations. Analogy-making is crucial for human cognition Kokinov and French [2003]. Many cognitive processes involve analogy-making: perceiving a square in a cubist painting as a human face, solving a mathematical problem in a way similar to another problem previously solved, understanding metaphors, communicating feelings or emotions, learning a foreign language, understanding poetry etc. (Gentner [2001]). All these cases require an abstract mapping to be established between two cases or domains based on their common structure or relations. This may require re-representation of one (or both) of the domains in terms of the other one. The rst domain is called the source, and the second is referred to as the target. Analogy-making is a very basic cognitive ability which probably starts with the simple ability of 5

babies to repeat a behaviour that leads to a satisfactory outcome, e.g. crying to get milk, then it slowly progresses to childrens being able to recognize an analogy between a word or utterance and the corresponding real object and ultimately culminates in the adult ability to make complex analogies between various situations. This seems to suggest that analogy-making serves as the basis for numerous other kinds of human thinking and explains the importance of developing computational models of analogy-making (Kokinov and French [2003]). Analogy-making, or analogical reasoning, involves at least the following sub-processes: Retrieval of a source for the analogy Mapping source onto the target Transfer, i.e., making inferences Evaluation of unmapped elements from the source to the target for applicability Learning and consolidation based on the experience, which includes generalizing from specic cases and possibly developing general mental schemas.

2.1.2

Stages in analogical reasoning

Five dierent stages in analogical reasoning are commonly distinguished: the retrieval, the mapping, the transfer, the evaluation, and the learning and consolidation stage. Retrieval This has been extensively studied experimentally and it is now clear that supercial similarity plays the major role in analogical retrieval, i.e. the retrieval of a source for an analogy is easier if its objects, properties and general theme are similar to those of the target. Structural similarity, the familiarity of the domain from which the analogy is drawn, the richness of its representations and the presence of generalized schemas also facilitate retrieval (Kokinov and French [2003]). The process of analogical retrieval starts with the comparison of two domains that are usually considered dissimilar. In most problems, our knowledge of the source domain is greater or more accurate than that of the target and the goal of the analogy is to say something about the target from our knowledge of the source. The reasoning involved 6

generally starts with a question about the target and the retrieval of a base representation of it. Thus, the rst step in forming an analogy is the retrieval of an analogue (Kokinov and French [2003]). While analogical reasoning is used everywhere in human cognition, it has been shown that the retrieval of suitable analogues is complicated. This can be exemplied in the experiment by Gick and Hoyloak (Gick and Holyoak [1983]) using Dunckers radiation problem:
Suppose you are a doctor faced with a patient who has an inoperable stomach tumor. You have at your disposal rays that can destroy human tissue when directed with sucient intensity. How can you use these rays to destroy the tumor without destroying the surrounding healthy tissue? Before participants were given this problem, they read various stories. For some participants, one of the stories involved a general who had to decide how to attack a fortress. The fortress was surrounded by mines, which would be set o if crossed by a large force. The general solved this problem by approaching the fortress with several small forces from multiple directions that would converge on the fortress all at once. Using this problem as an analogue to the problem they were supposed to solve, participants would know that they could use smaller doses of radiation from multiple directions, which would arrive at the tumor at the same time, thus allowing for the same large dose of radiation to be administered without having to destroy tissue around a single location on the body.

The result of the experiment showed that participants who were not given the fortress story prior to reading the radiation problem solved it about 10% of the time. Participants who were given the fortress problem solved the radiation problem about 30% of the time. Although this was a signicant increase, it also meant that 70% of the time participants were not able to retrieve the relevant analogue. This shows that retrieval is a rather dicult step and given that supercial similarity facilitates retrieval, as mentioned above, one also has to take into account that people still have problems retrieving suitable analogues. This may be due to the fact that people tens to retrieve items from memory based merely on surface similarity (i.e., based on objects and attributes), rather than relational similarity (i.e., based on common relational structures) (Gentner et al. [1993]). Since analogies are all about relational similarity, this is a problem. The tendency to retrieve surface-similar analogues can also be an explanation for why experts are better at analogical reasoning in their domains of expertise than beginners (Novick [1988]). This last fact actually hints at the reasons for accepting

domain-general theories of analogical reasoning. While expert reasoning may appear to be using domain-specic processes, what may actually be happening is that experts simply have more complex relational knowledge within that domain. More complex representations yield dierent types of results in analogical reasoning. The next step after retrieving an analogue is to form an analogy between the retrieved information and the target. This is the mapping step. Mapping This is unquestionably the core of analogy-making and, therefore, all computer models of analogy-making include mapping mechanisms, i.e., means of discovering which elements of the source correspond to which elements of the target (Kokinov and French [2003]). In a typical case of analogical mapping the most familiar situation is the source domain, which is then matched to a less familiar situation the target domain. The familiar situation suggests ways of viewing the newer situation as well as further inferences about it. Analogical mapping requires aligning the two situations, that is, nding the correspondences between the two representations and projecting inferences from the source domain to the target domain. This match is then evaluated as are its implications for the solution. The problem with this is that one situation can be mapped onto a second situation in many dierent ways. The psychological processes that go on in analogical mapping were described by Gentners Structure-Mapping Theory (Gentner [1983]). According to this theory, the comparison process involves nding an alignment between the source and target representations that reveals a common relational structure. On the basis of this alignment, further inferences are projected from source to target. Gentner [1983] says that people prefer to nd an alignment that is structurally consistent, that is, there should be a one-to-one correspondence between elements in the source and elements in the target, and the arguments of corresponding predicates must also correspond, thus having a parallel connectivity. For example, in the analogy below, Peter in (a) could be put in correspondence with Peter in (b) (local entity match) or with Anne in (b) (relational role match). People appear to consider both possibilities during processing, but have to settle on one or the other by the end of the process. (a) The dog rescued Peter. (b) Peter rescued Anne. 8

Another important theory for the mapping process was given by Cheng and Holyoak [1985]. Their pragmatic mapping theory focused on the use of analogy in problem-solving and held that analogical mapping processes are oriented towards attaining goals (such as solutions to problems). According to the pragmatic mapping theory, it is goal relevance that determines what is selected in an analogy. Holyoak and Thagard [1989] later combined this pragmatic focus with structural factors in their multi-constraint approach to analogy. Analogical inference projection is a crucial part of the mapping process. Once an alignment is achieved, further inferences can be made by projecting information from the source domain onto the target domain. For example, in the above analogy, suppose we knew more about event (a), such as: (a) The dog rescued Peter because he knows him. The dog has black hair. (b) Peter rescued Anne. In this case, the likely inference in (b) is that Peter rescued Anne because he knows Anne. This ability to request new inferences are central to the role that analogies play in reasoning. Importantly, analogical inference is rather selective. For example, we are unlikely to make the inference (in the above example) that Peter has black hair. This illustrates the selection problem in theories of analogical inference. If people projected everything known about the source onto the target, analogies would be useless in reasoning; it would even make things more complex (Holyoak and Thagard [1989]). However, this is not normally done and that is why the characterization of the selection process is a central aim of theories of analogy. For this at least three factors have been proposed. Holyoak and Thagard [1989] proposed goal relevance: the inferences projected are those that t with the reasoners current goals in problem-solving. A second factor, relational connectivity or systematicity, was proposed in the theory underlying the Structure-Mapping Engine (Gentner [1983]). Here, there is a higher preference for projecting matching systems of relations connected by higher-order relations such as cause, rather than projecting local matches. In many cases, goal relevance and systematicity will make the same predictions because problem-solving goals often involve a focus on causal systems (Gentner [1983]). A third factor in selecting inferences, as proposed by Keane (Gentner [1983]), is adaptability. In this case the ease with which a possible inference can be modied to t the target is preferred. There is evidence for all three of these criteria. Spellman and Holyoak [1996] showed that when two possible mappings are available for a given analogy, people would select the 9

mapping whose inferences are relevant to their goals. Evidence for systematicity comes from the nding that when people read analogous passages and make inferences from one to the other, they are more likely to transfer a fact from the source to the target when it is causally connected to other matching predicates (Markman [1997]). Finally, Keane [1996] found evidence that the degree of adaptability predicts which inferences are made from an analogy. Experimental work has demonstrated that nding this type of structural isomorphism between the source and target domains is crucial for mapping (Gentner [1983]). Object similarity also plays a role in mapping, although generally a secondary one. Another factor is the pragmatic importance of various elements in the target, because people try to nd mappings that involve the most important elements in the target. Searching for the appropriate correspondences between the source and target is a computationally complex task that can lead to combinatorial explosion if the limits are not constrained (Kokinov and French [2003]). Transfer This is the process of introducing new knowledge into the target domain based on the mapping (Kokinov and French [2003]). After the two domains are mapped onto each other, information is transferred from the source domain to the target domain. In Structure Mapping Theory, this involves producing candidate transfers or inferences. These are potential inferences about the target from the source that come from the entities that are part of the common relational structure but are present only in the source. Those entities are carried over to the target and placed in the corresponding position in the relational structure of its representation. If these inferences are good ts, they are kept. If not, they are discarded (Kokinov and French [2003]). For example, suppose a new type of public transport has been implemented in Osnabrck and you heard in the news that this is the same that u is used in New York. It is cheap, ecological and fast, but you also know that it is very dangerous at night. Transfer is when you wonder whether the analogous new public transport will also be unsafe for use at night in Osnabrck. u Transfer is present in one form or another in most models of analogymaking and is typically integrated with mapping. Transfer is considered by some authors as an extension of the mapping already established, thus adding new elements to the target in such a way that the mapping can be extended (Kokinov and French [2003]).

10

Evaluation This is the process of establishing the likelihood that the transferred knowledge will turn out to be applicable to the target domain. Once the common alignment and the candidate inferences have been discovered, the analogy is evaluated. According to Gentner [1983] evaluating an analogy involves at least three kinds of judgment: 1. structural soundness: whether the alignment and the projected inferences are structurally consistent; 2. factual correctness: whether the projected inferences are false, true or indeterminate in the target; and 3. relevance: whether the analogical inferences are relevant to the current goals. In practice, the relative importance of these factors varies quite a bit, for example, in domains where little is known or where there is disagreement about the facts. In the example above, the evaluation process would have to assign our degree of condence about whether the new public transport will also be dangerous at night in Osnabrck. Evaluation is often implicit in the u mechanisms of mapping and transfer. Learning and consolidation Abstraction All the later processes involved in analogy-making lead to an eective learning of the individuals. This translates into a consolidated knowledge that can be used to solve future analogies by retrieving prior knowledge. Thus in analogical abstraction, the common system that represents the interpretation of an analogy is extracted and stored. This kind of schematic abstraction helps to promote transfer to new examples. When people are asked to compare two analogous passages, they are better able to later retrieve and use their common structure (given a relationally similar probe) than are people who were given only one of the stories (Gick and Holyoak [1983]). Further studies have shown that actively comparing two analogous passages leads to better subsequent retrieval than reading the two passages separately. These ndings are consistent with the claim that analogical alignment promotes the common structure and makes it more available for later use (Gick and Holyoak [1983]).

11

Analogies in real-world reasoning Analogies are often used in commonsense reasoning to provide plausible inferences. It must be noted that analogymaking is not a deductive process (Bechtel and Graham [1999]). There is no guarantee that the inferences from a given analogy will be true in the target even if the analogy is carried out perfectly and all of the relevant statements are true in the source. However, the set of implicit constraints described above make analogy-making a relatively tight form of inductive reasoning (Bechtel and Graham [1999]). There is a ip side to the lack of deductive certainty in analogical reasoning. It means that analogies can suggest genuinely new hypotheses whose truth values could not be deduced from current knowledge. Analogical comparison can lead to new learning in at least four ways: analogical abstraction, inference projection, dierence detection, and re-representation. Bechtel and Graham [1999] state that in analogical abstraction, the structure common to the source and target is noticed and extracted. Sometimes the common system is stored as a separate representation: this is referred to as schema abstraction (Gick and Holyoak [1983]). In inference projection, a proposition from the source is mapped to the target. If it is retained as part of the target structure, then learning has occurred. In dierence detection, carrying out a comparison process leads people to notice certain dierences, namely those connected to the common structure. In re-representation, two non-identical predicates are aligned and decomposed (or abstracted) to nd their commonalities, resulting in a re-representation that contains a common predicate: for example, comparing chase(dog, cat) and follow(detective, suspect) might result in pursue(entity1, entity2). A further kind of knowledge change hypothesized to take place in scientic discovery is restructuring, in which the target undergoes a radical change in structure (Bechtel and Graham [1999]).

2.1.3

Types of analogies

Analogies are known to play a key role in cognition, especially in creative problem-solving across areas, whether art or science. They are also part of standard intelligence tests. For researchers in cognitive science, analogies open a new door to get closer to understanding cognition and their role in human cognition (Indurkhya [1989]). However, analogies come in a wide range of variations depending on the eld of study and psychological content. There are signicant variations in what is meant by the term analogy itself, but they can be reduced to at least three distinct usages. In the following these three basic senses of analogy are presented and the roles they play in cognition (Indurkhya [1989]). Following an overview of the types of analogy, 12

the particular type of analogy used in our study project is presented. Analogy by rendition This rst use of the term analogy concerns seeing or perceiving an object or situation (target) as another object or situation (source) (Indurkhya [1989]). The main characteristics of this mode of analogy is that the similarities between the two objects or situations do not exist prior to viewing one as another, but instead are created by the process. This can be made clear by an example from Schn [1963], where a development team, in trying to imo prove the performance of synthetic bre paintbrushes, ended up viewing the paintbrush as if it were a pump. In the example, it becomes clear that it was not that the researchers noted some similarities between the paintbrush and the pump and then imported more features from the pump to the paintbrush, but rather the act of viewing the paintbrush as a pump created the similarities similarities that were not there before. Put another way, whenever we notice that two objects or situations are similar, we can do so only with respect to their existing ontologies and descriptions, their actual information load. However, in analogy by rendition a completely new ontology is created and a new level of description for the target object can emerge (Indurkhya [1989]). This type of analogy, according to Gordon [1961], tries to make the unfamiliar familiar or the familiar unfamiliar, depending on the situation. The research question is more along the line of trying to explain how new perspectives are created, which is a deep cognitive process that involves too many aspects to be covered so easily. This mode of analogy is closely related to models and metaphors and all require elements of interpretation in their environment to become meaningful (Indurkhya [1989]). However, this is only to illustrate how complex analogies can get when abstraction or change of established environments come into play. Proportional analogy Another use of analogy is the so-called proportional analogy. This type of analogy is concerned with relations of the form A is to B as C is to D. As in gills are to sh as lungs are to man. The process referred to is usually that of generating the fourth term (D) of a proportional analogy relation given the other three (Evans [1962]). At rst sight, a proportional analogy seems to be a syntactic process: that is, it appears that only the terms A, B, C, and D and their structures are important, but no meaning is involved. This could be more evident when abstract symbols are used such as geometric gures or letters. This is why the rst computer models used such types of analogies 13

and were moderately successful (Evans [1962]). However, these analogies also have an element of interpretation. For instance, to understand the analogy gills are to sh as lungs are to man requires one to interpret the role of lungs in the context of physiology of another species, in this case that of sh, since these are dierent. Hofstadter and Dennett [1981] point to the fact that this interpretation is not limited to proportional analogy relations involving words or other meaningful symbols. They discussed many cases where all of the four terms of the analogy relation were meaningless strings of characters, yet one needed to identify the roles of the various characters in a string and then interpret these roles in the context of another string. Furthermore, when geometric gures are used in such proportional analogies, there is also a need to interpret the gures. Coming up with an adequate interpretation strongly depends on the gure or on the relations it has within or to another gure etc. Figure 2.1 shows examples of abstract geometric gures. In these three proportional relations shown, the rst term is always the same. Depending on the analogy in which it is embedded, it requires a dierent interpretation or a dierent way of seeing it. If we think back to analogy by rendition, this seeing strongly reminds us of the process involved there, so a strong separation among the types of analogies is not possible. They all have some aspects that they share. In this case however, the interpretation is even more complex because in each step of the proportional analogy, the gures A, B, C, and D contribute to the context of the interpretation of the remaining gures, while at the same time being interpreted in the context of the others (Indurkhya [1989]). In the example given in Figure 2.1, gure A can be described as two inverted triangles that are put on top of each other in an o-centre manner. In the second case, it is described as a hexagon with an equilateral triangle on each side. The symbols hexagon, triangle, inversion etc. that emerge from the conguration of the analogy provide an ontology and structure to A. One could say that high-level concepts are obtained by grouping lowlevel representations of the other gures (Indurkhya [1989]). Thus, while A is described using a vocabulary furnished by the other gures, this very act, at the same time, provides a vocabulary for describing other gures, which in turn aects the vocabulary being used to describe A, and so on; they all interact and one has to see the whole analogy as a singular interacting entity. As with the previous type of analogy, the real challenge is to model the interaction between the four terms of the analogy. Another important aspect of all proportional analogies is the symmetry property that they exhibit with respect to interchanging the second (B) and the third term (C). That is, if A is to B as C is to D is a proportional relation, then so is A is to C as 14

Figure 2.1: Three proportional analogies that each require a dierent interpretation of the star in A B is to D. Any reasonable theory or model of proportional analogies ought to have the same characteristic. Proportional analogies are interesting to studying cognition because of the underlying process of shifting perspectives and changing representations (Indurkhya [1989]). It is indeed a hallmark of intelligence to be exible and to be able to adjust mental models to solve the actual problem, and this in the best and most economic way. Therefore, by successfully modelling these types of proportional analogies, we can get some modest insight into the cognitive process of describing or seeing the same situation from dierent points of view: a process that indeed lies at the very heart of cognition (Indurkhya [1989]). Predictive analogy Predictive analogies help to explain a new domain by observing similarities with a known domain (source) and then transferring further information from the known domain to the new domain (target) (Gentner [2001]). Here the main dierence to analogy by rendition is that the process involved in predictive analogies are less often applicable than that of analogy by rendition, since it requires that there be some existing similarities between the two domains. The target must have some ontology and this ontology must be triggered by some existing similarities between the source and target (Indurkhya [1989]). In analogy by rendition, the ontology of the target is changed as a result of the process, and a new perspective (similarity) is created. The new similarities created between the source and target are from their existing ontologies. 15

It is also important to notice that in the predictive analogies, the predicted similarities are made with respect to the existing ontologies and no new ontologies are created. Predictive analogies are more powerful than analogy by rendition as they predict that there might be other similarities, which analogy by rendition cannot claim since one cannot know in advance how the regrouping might aect the rest of the target. In contrast, predictive analogies do exactly that; possible similarities that have previously been encountered in similar problems are sought and it is predicted that this will t and help resolve the new problem. This characteristic of predictive analogies, to solve the newly-encountered problem by trying to nd a similar, previously-solved problem, is what makes them so attractive for a heuristic process for problem-solving (Indurkhya [1989]). At the same time, it also makes it the most problematic one because the transfer or inference is not considered to take place in the logical sense of the word; this inference is not necessarily true but is considered to be justied. This is to say that in some psychological sense a rational person nds it more reasonable than a random inference, but this degree of reasonableness, or justication, strongly depends on its association with an inference from the analogy which increases with the amount of known similarity between the source and target, and it can vary strongly amongst individuals (Indurkhya [1989]). Predictive analogy is also known as analogical reasoning and is applied in most cognitive computational models, such as Structure Mapping Engine (Gentner [2001]). A well-known example of a predictive analogy is the Rutherford analogy: The atom is like the solar system. The Rutherford analogy is given in Figure 2.2 with the source domain solar system on the left-hand side and the target domain atom on the right (Gentner [2001]). A domain is represented as a structure (a graph) with objects (such as sun, planet-i), attributes (such as yellow(sun)), and relations (such as attracts(sun, planet-i)). Besides the rst order relations, which are dened over objects, there is a second-order relation, dened over relations: cause (attracts(sun, planet-i), revolves-around(planet-i, sun)). All these entities are represented as nodes in the graph. The arcs represent relations between these entities, in other words, their roles (such as subject or object). In contrast to proportional analogies, the representation (description) is xed. Analogical reasoning is modelled as a structure preserving mapping of objects from the source to the target. Following the principle of systematicity, mappings of larger structures are preferred. For the Rutherford example, 16

Figure 2.2: Structure mapping for the Rutherford analogy, Gentner [2001] the mapping of sun to nucleus and of planet-i to electron-i results in a large structural congruence between both domains. Since each object can carry over nodes from the source to the target to which it is connected, the causal explanation of why a planet revolves around the sun is transferred to the target domain, resulting in the inference that an electron revolves around the nucleus because it is attracted by it (Schmid et al. [2003]). Another aspect relevant to our study project is that proportional analogies can be viewed as special cases of predictive analogies. In the latter, the inferred relation becomes its cause relation, whereas in proportional analogies, the inference is typically not the complete pattern (or cause) but only a sub-structure of the source that is mapped to the target (Schmid et al. [2003]). These substructures that can be inuenced by Gestalt laws and relations within the gures of geometric proportional analogies is what motivates our study project.

2.1.4

Geometric proportional analogies: our focus

Geometric proportional analogies (GPAs) are a type of analogy formed between two collections of geometric gures (Mullally and ODonoghue [2005]). Geometric analogies then are graphical proportional analogies (Figure 2.3), where A, B and C are each identied by a geometric gure and they are also of the form A:B::C:D (A to B as C to D). The source domain (A:B) identies some transformation(s), which must then be applied to C, yielding the solution D. For example, the analogy in Figure 2.3 revolves around dividing the central polygon of part A to produce part B. This partitioning transformation must then be applied to C, which allows us to generate D. There are two key points to note about GPAs. First, the change between the terms in the source domain (i.e. between A and B) is called the transformation. Second, parts A and C are used to identify the 17

inter-domain mapping (Gentner [2001]), i.e. small square (A) maps to small circle (C) and central square (A) maps to central circle (C). The combination of the transformation and mapping will yield D.

Figure 2.3: Simple geometric proportional analogy

Figure 2.3 shows proportional geometric analogies, as used in our study project and some solutions that might be interesting for the modelling of a heuristic. As said above, the similarity between proportional analogies and predictive analogies makes them interesting for our project because the transfer or inference is not considered to be an inference in the logical sense of the word and the degree of reasonableness of the inference can vary strongly amongst individuals (Indurkhya [1989]).

Figure 2.4: Examples of geometric analogies used in our study project

18

Often dierent geometric gures exist which are all possible solutions for an analogy; however, they might not be equally plausible and depend on the perception of the elements A and B (Schwering and et al. [2007]). When the human visual sensory system observes a geometric gure, it transforms the unstructured information into a structured representation of coherent shapes and patterns. Human perception tends to follow a set of Gestalt principles for organization: stimuli are experienced as a possibly good Gestalt, i.e., as regular, simplistic, ordered and symmetrical as possible (Koka [1935a]; Khler [1929]; Wertheimer [1929], Wertheimer [1954]). Gestalt psychology a identied dierent principles according to which humans construct Gestalts (e.g. principles of proximity, similarity, symmetry and good continuation, see Gestalt chapter for details). Applying dierent Gestalt principles to the same geometric gure might result in dierent perceptions: Although certain rules have been identied as to which Gestalt principles are cognitively preferred (Wertheimer [1929]), there does not exist a xed hierarchy of Gestalt principles. The perception of geometric gures may dier among humans and depending on context.

2.2
2.2.1

Gestalt psychology
Gestalt theory

Human perception is a complicated process with many steps involved before an action is produced as an outcome of the perceptual process (Figure 2.5). There have been many theories originating from the study of human perception that try to explain the process. Gestalt theory is one of the theories which accounts for human perception. According to Gestalt theory, human perception is a parsing process, by which a stimulus pattern (unstructured information) is given a structured representation (Koka [1935a]). Gestalt principles try to account for this parsing as a process that follows certain rules. Therefore, when the human visual system perceives a geometric gure, it transforms the unstructured information into a structured representation of coherent shapes and patterns (Dastani and Scha [2003]). The transformation of the unstructured visual information into less unstructured (or structured) information follows certain principles, namely the Gestalt principles (Koffka [1935a]). Intuitively, it can be said that the role of Gestalt principles in explaining human perception is that of linking stimulus and perception via a phenomenological method, i.e, when one describes what is perceived (reception-perception).

19

Figure 2.5: Human perceptual process

2.2.2

Gestalt principles

Gestalt theory originated from the study of human perception as an outcome of concrete investigations in psychology, logic and epistemology. In 1912, Wertheimer stated for the rst time the principles of a Gestalt theory, with the main idea being that the whole is more than the sum of its parts (Wertheimer [1924]). The general idea of the Gestalt theory by Wertheimer is described further in the Gestalt principles, which are a series of laws of how human perception organizes small parts into the whole to make sense of perceived objects. Wertheimer [1924] identied several Gestalt principles which underlie human perception as follows: 1. Law of Proximity According to the Law of Proximity as dened by Wertheimer [1912], elements that are closer together will be perceived as coherent objects. Therefore, the group of circles in Figure 2.6(a) forms columns of circles, rather than rows of circles as in Figure 2.6(b). The organization into either rows or columns of circles is based on the distance between the circles. The vertical distance between the circles in Figure 2.6(a) is closer than their horizontal distance, which is why they are perceived as columns of circles. 2. Law of Similarity The Law of Similarity states that similar elements are grouped as an entity. The similarity itself depends on the relationship constructed based on color, size, etc. of the elements. An example of applying the Law of Similarity in object perception can be seen in Figure 2.7. There are two groups of elements in Figure 2.7 with the grouping of the elements based on the shared

20

(a)

(b)

Figure 2.6: Law of Proximity characteristic of the objects, their color. The circles are thus perceived as rows of white circles interspersed with rows of black circles.

Figure 2.7: Law of Similarity

3. Law of Common Fate According to the Law of Common Fate, the grouping of several objects can be based on direction, that is, if two or more objects move in the same direction, they are considered to be one unit. The application of the Law of Common Fate is illustrated in Figure 2.8. The grouping of the black circles can be ascribed to the direction of the movement of alternating columns of circles, i.e. a group of circles that move upward and another group of circles that move downward. 4. Law of Objective Sets The Law of Objective Sets states that humans have a certain tendency to perceive one aspect of an event as a gure or foreground and another as a 21

(a)

(b)

Figure 2.8: Law of Common Fate ground or background. Moreover, a certain objectively ambiguous arrangement will be perfectly denite and unequivocal when given as a part of a composition. This idea is illustrated in Figure 2.9. Depending on our perception of foreground/ background, we will see either a black prole of two human faces facing each other or a white vase.

Figure 2.9: Law of Objective Sets

5. Law of Direction The Law of Direction states that lines are preferentially regarded as if they followed the smoothest or the simplest path. When two lines intersect at a certain point, the lines are usually not assumed to deviate at the intersection point. This is illustrated in Figure 2.10, for the case of intersecting lines. There are two lines in Figure 2.10. According to the Law of Direction, people will perceive the rst line as going from A to B and the second line from C to D, rather than, for example, one going from A to C and the other from C to D.

22

Figure 2.10: Law of Direction 6. Law of Good Curve (Continuation) The Law of Good Curve implies that a pattern is continued even after it stops, following certain regularities. Moreover, according to the Law of Good Curve, stimuli that seem to be a continuation of the previous stimuli are considered to belong together. The Law of Good Curve is illustrated in Figure 2.11, where the dashed lines are perceived together as a continuous line.

Figure 2.11: Law of Good Curve

7. Law of Closure The Law of Closure states that humans tend to perceive enclosed (geometrical) objects as a whole as opposed to an open structure. Therefore, spaces are enclosed by completing a contour and disregarding gaps in a gure. As can be seen in Figure 2.12, humans tend to perceive the central gure as one white triangle and three black circles, ignoring the gaps that make the circles and the triangle incomplete. 8. Law of Symmetry The Law of Symmetry states that symmetrical alignments are seen as belonging together regardless of their distance, as illustrated in Figure 2.13.

23

Figure 2.12: Law of Closure

Figure 2.13: Law of Symmetry 9. Law of Past Experience Lastly, Gestalt theory also accounts for peoples habits and knowledge, which are said to guide human perception. This is referred to as the Law of Past Experience. It is implied in the Gestalt Law of Past Experience that every experienced sensation is stored in long-term memory; current sensations are then compared to already-stored information. An example of this case is given in Figure 2.14. Although the written word is not complete, we are able to make out that the word meant is Gestalt. According to the Law of Past Experience, this is because we have information about the word Gestalt somewhere in our long-term memory. Thus, when we get this new input, we are able to recognize that it is the word Gestalt.

Figure 2.14: Law of Past Experience However, Koka [1935b] argues that there is only one general principle that guides human perception, that is, the Law of Prgnanz, which means a simplicity or regularity. According to the Law of Prgnanz, perceptual ora 24

ganization always follows the rule of simplicity. It could mean that when there is an interaction between various Gestalt laws, which result in dierent structures of a pattern then, following the law of Prgnanz, the simplest a or the most regular of these structures is the one that is actually perceived (Metzeger [1928]).

2.2.3

Application of Gestalt principles

The fundamental formula of Gestalt theory is dened as follows that the behaviour of a system is determined by the intrinsic nature of the whole, and that Gestalt theory hope to determine the nature of such wholes (Wertheimer [1924]). Therefore, Gestalt theory is a way to investigate science and the problem in science from its core, by assuming it as a whole part, and paying attention to the relationship holds between its parts, no longer investigate a system as separation of parts (Wertheimer [1924]). In a relation with human perception, stimulus recognition is a perceptual organization, which is done by organizing a scene in the environment to form perceptually separate object to facilitate object recognition and make perception have sense. The role of Gestalt principles is guiding the scene organization. Inspired by Gestalt Principles which said were said to guide human perception, a general theory of pattern perception was developed, called Structural Information Theory (SIT). The theory aims at explaining why a pattern is perceived as having a certain structure, not a dierent one (Dastani and Scha [2003]). The idea is the same as Gestalt principles that aim at explaining why such structure is perceived as having such and such representation, not a dierent one(s). SIT is therefore working based on the assumption that human perceptual system is sensitive to certain kinds of structural regularities of a pattern (Dastani and Scha [2003]). SIT accounts for complexity measure of a pattern. According to SIT, the chosen pattern would always be the pattern with least information load (Dastani and Scha [2003]). This is, so to say, would be a formalization of Gestalt law of simplicity. On our experiment of perception of geometric analogy, we choose a set of gestalt principles, with its working denition that Gestalt laws are the apparently innate mental laws that determines human perception of objects: 1. Law of Proximity 2. Law of Symmetry 3. Law of Similarity 4. Law of Good Continuation / Direction 25

These laws are taken into consideration because of the possibility to formalize them and their importance in cases of perceiving geometric gures, other than perception of others. Therefore, our set of analogy examples that were used as stimuli were created by taking either Law of Proximity, Symmetry or Good Continuation into consideration. Our assumption is that dierent application of Gestalt law in perceiving the same analogy would result in dierent solution. Figure 2.15 and 2.16 provides illustration of how applying dierent laws to the same analogy would result in many dierent solutions.

Figure 2.15: Law of Proximity in proportional geometric analogy Figure D in Figure 2.15 is a result of applying the Law of Proximity to solve the proportional geometric analogy problem. Following the Law of Proximity, the circle and the square below are grouped together. The analogy is completed by moving the circle up toward the square, as what was done in the source domain (gure A and B). However, the Law of Proximity is not the only Gestalt law that guides the problem solving of this particular analogy. This analogy can also be solved by applying the Law of Good Continuation (gure 2.16). According to the Law of Good Continuation, the circles are grouped together, as well as the squares. The analogy is completed by moving the circle down, against the square.

Figure 2.16: Law of Good Continuation in proportional geometric analogy

26

Chapter 3 Experimental part

Clemens Bauer, Judith Degen, Irena Dorceva, Martin Schmidt, Rolf Stollinski & Kae Sugwara Institute of Cognitive Science, University of Osnabrck u

3.1

Motivation

Analogies play a central role in human cognition, for example in reasoning and learning processes. They are often used to explain and understand new phenomena by applying already known facts from a familiar domain. Analogies are used in education and science as well as in everyday life. The CoUGAR study project is part of the AI group at the Institute for Cognitive Science, Osnabrck that has been studying analogies for the past few u years and developed a formal framework for handling analogies called HDTP (Heuristic-Driven Theory Projection). The main drive of the project is to provide empirical evidence for analogical cognitive processes that are involved in the solving of these analogies. The rst underlying hypothesis is that human perception tends to follow a set of Gestalt principles for organization and this also applies to geometric proportional analogies. Accordingly stimuli are experienced as a possibly good Gestalt, i.e., as regular, simplistic, ordered and symmetrical as possible (Koka [1935a], Khler [1929], and Wertheimer [1954]). In other words, when a the human visual sensory system observes a geometric gure, it transforms the unstructured information into a structured representation of coherent 27

shapes and patterns. The second hypothesis is that the solution to geometric proportional analogies can best be studied by focusing more on recall memory rather than on recognition memory. This based on the theory that recalling an item from memory requires more information storage (i.e., memory strength) than just recognizing an item. Thus, if assessed, it can yield more insight on the cognitive process involved in solving the analogies. In order to address the former hypothesis, we will apply a set of dierent Gestalt principles to the same geometric gure in the analogy that might result in dierent perceptions: Although there have been some proposals for a hierarchy in the preference of Gestalt laws (Wertheimer [1924]), there does not exist a xed hierarchy of Gestalt principles. Thus, we restrict our set of Gestalt principles (see Gestalt section). This will hopefully yield some information about the rst-order perception and its inuence on the solution of analogies. For the latter hypothesis, our experiments are designed in a way that the participants have to construct the solutions from a given set of gures, thereby shifting the analogical reasoning process to a recall memory cognitive task instead of just a recognition memory task. This conscious recall process induced by the setup of the experiment is further analyzed by implementing a comment option whereby the participant can enumerate the steps in reasoning that produced the answer. The combination of disambiguating the Gestalt laws involved in the rst-order perception, the induced recall memory cognitive process to solve the analogy and the option to comment on the steps that produced a particular solution to an analogy is expected to yield important information that can be used in HDTP to get better and more accurate results in solving geometric proportional analogies.

3.2
3.2.1

Technical framework
General introduction to the Analogy Lab

During the course of the study project, in addition to the nal experiment, two initial pretests and two supplementary experiments (TdT1 ; HIT2 ) were carried out. Both were realized with the so-called Analogy Lab. This online lab, originally developed for the purpose of exploring analogical cognition, has been reworked and rened in the course of time for the benet and success of the study project. It was used as a programming environment to set up and conduct all of the above mentioned pretests and experiments.
1 2

TdT: Tag der Technologie HIT: Hochschulinformationstag

28

When visiting the Analogy Lab3 , guest users have the possibility of familiarizing themselves with the general ideas, basic concepts, and the terminology of analogy-making. One can also look into pending experiments dealing with analogies, especially into those that are concerned with geometric proportional analogies. Although not limited to such analogies, the main focus of the Analogy Lab has been on the presentation of the general subject matter of analogies, including the research done by the ICS Analogy Group as part of the Institute of Cognitive Science at the University of Osnabrck. u Figure 3.1 illustrates a rough sketch of the labs architecture. The study project used this framework to conduct all experiments.

Lab Content

Lab Resources

WIKI & Graphics Back End Browser

Prolog Lisp Huskell R Octave VSA

User Management

Parameter Passing

User Data

Shared Data

Figure 3.1: The overall architecture of the Analogy Lab By means of a typical web browser one can easily access all freely available global contents and resources. However, in order to gain access to vital lab areas and functions one has to be logged into the system as a registered user with certain necessary user-rights granted by the labs super-administrator. In this way, it is possible to modify the lab by adding, editing or deleting desired contents such as plain text, scripts, mathematical formulas, complex graphics, or images. To facilitate work and to increase its usability in general, the Analogy Lab provides diverse back-end capabilities, i.e. external
3

Link to the Analogy Lab:

http://mvc.ikw.uni-osnabrueck.de/labs/osw_en.php?&lab=analogy-lab&usr=#urls

29

interface-to-server-side applications which can be called on demand. SGML (Standard Generalized Markup Language), common derivates such as HTML and XML, JavaScript and even LaTeX code are likewise accepted by the system and can easily be implemented by the user. In this context, it is worth mentioning that lab structures are organized each time as (web)pages in a tree-like fashion. The path to a page can be compared to a directory path on a data medium, and, indeed, local pages are represented by directory paths on the lab server. Specifying a hitherto non-existing pathname would for example automatically create a new blank page provided that certain naming rules are observed. In doing so, one can generate as many pages as desired and simultaneously arrange them in a desired way. As these lab pages can analogously be regarded as les on a hard-disk drive, they can eortlessly be edited as one desires using typical disk operations (creation, overwriting, copying, and deletion). For the purpose of editing a page, numerous formatting possibilities are at ones disposal, among others the creation of tables and lists, textual enumerations and indentations, the integration of graphics and images, or the visualization of formulas using LaTeX code. By this means, it was possible to properly program all pretests and experiments that were conducted in the course of the study project.

3.2.2

Lab elements and basic functions

The upcoming section aims at exemplifying the basic working elements of the Analogy Lab by briey explaining its functions and purposes. Figure 3.2 displays the main window of the lab system in its guest user mode. The separate column on the left-hand side contains all globally available pages that are organized in a tree structure as explained before. A simple mouse-click on one of the listed item names is sucient to jump to the desired page whereupon all existing subpages will be shown in addition (i.e. the page Final entails the subpages Welcome, Description, Final Combo1, Feedback, and Results). The currently visible page (Welcome) is always highlighted in red letters whose particular content can be viewed within the embedded window at the centre of the screen. The arrows as well as the plus and minus symbols in the upper left corner can alternatively be used to navigate through the pages and to increase or decrease available graphics, respectively. The login elds are located in the upper right corner of the screen. This brings us to the labs administrative mode. In order to fully switch from the guest user to the administrator (admin) mode, one has to log into the lab system. Here, one should be able to recognize that several more buttons have appeared in the lower right corner of the screen (Figure 3.3). 30

Figure 3.2: The Analogy Lab as seen in its guest user mode By pressing the raw button, it is possible to hide all navigation elements to bring up only the raw content of the currently active page. Pressing the slides button enforces a mode that shows entire pages as content slides with slim navigation features only. The novel working window allows the user to edit a page as desired. The main editor window just o the middle of the screen displays the underlying code of the currently called Welcome page whose actual content is concurrently shown next to the editor window on the left-hand side. Towards the bottom of the lab window (Figure 3.3) one can nd a separate section which serves as a designated upload interface. In this way, it is possible to upload les including images or other resources to the user space or other lab areas. The Durchsuchen button opens a single browser window that allows the user to search local drives and other data media for the desired le to be uploaded to the server. As with pathnames, one equally has to comply here with certain naming rules when specifying directory paths within global lab areas or in the space of the current user. In this respect, the target path itself can be entered by using the provided eld right to the upload button, which must be pressed in order to upload a le. The list button lists the content of the current directory in a demarcated window 31

Figure 3.3: The Analogy Lab in its administrative mode that can be displayed on demand (not visible in Figure 3.3). Finally, the off button in the rightmost lower corner disables the administrator mode without logging out of the system.

3.2.3

Experimental paradigms and programming issues

The following paragraphs are intended to outline the various technical aspects with reference to the pretests and experiments that were conducted in the course of the study project. Apart from general remarks on the experimental paradigms, special emphasis is placed on diverse programming issues comprising detailed comments on selected sample code as well as on important interaction elements that were implemented for the benet of the project and not least for the success of the experiments. In sum, the purpose of this section is solely to address crucial aspects, not to provide an overall overview. Both the two pretests and the subsequent experiments had in common that they were concerned with solving geometric proportional analogies of the form A is to B as C is to D. In all cases the participants were encouraged

32

to complete D independently whereupon they were given two fundamentally distinct ways of solving analogies. With the exception of the second pretest which followed a dierent experimental paradigm, the rst way of solving analogies was to select one out of several given solutions (D1, D2, or D3) for D by clicking on the number of the solution that the participants found most plausible. A typical stimulus can be seen in Figure 3.4. Here, each object and textual element was manually implemented with the aid of the lab environment. In order to give a comprehensible example, gure A as part of the analogys source domain was basically composed by the following sample code: 1 2 3 4 ... 5 @graph[0,0,800,250,white] @rec_[38,-5|<b>A</b>] @obj_[24,30,45,45|example/circle_black] @obj_[24,116,45,45|example/square_white] @graph/[]

To begin with, the @graph command is used to start a graphic area, @obj_ signies a graphical object such as an image, and @rec_ eventually designates an empty rectangular eld that can be lled with some content such as plain text. The underscore characters in this context imply that the corresponding objects can only be dragged in admin mode. If they are left out, the pertaining objects can also be dragged in guest user mode. The @graph/[] tag in line 5 nally closes a graphic area. The square brackets in each line [...] may contain various parameters that allow for an adjustment or a more detailed specication of the associated objects. Concerning the rst line of the code, the embedded parameters yield a graphic area whose size amounts to 800 pixels in the x-direction and 250 pixels in the y-direction. This size is incidentally sucient to cover the whole analogy shown in Figure 3.4. What is more, the graphic area is assigned a white background colour whereas the numerical values 0,0 specify its starting position within a two-dimensional hidden coordinate system. The second line contains the code for the letter A at the relative position 38,-5. The HTML tags <b> and </b> are used here to make the letter appear in bold type. Both the third and the fourth line specify the two geometric objects that can be found in gure A. These objects are mere GIF format images that were previously stored in the directory example under the name of circle_black and square_white, respectively. From there they are now called, loaded and put on the screen. The values 24,30 and 24,116 designate their particular positions in relation to the mentioned coordinate system whereas the 33

Figure 3.4: One task consisted of selecting the most plausible solution for D in order to complete the given analogy numbers 45,45 represent the current image size (width/height) which can of course be changed as desired. In this context, the following issue should not remain unmentioned with respect to the preliminary tests and experiments. It was found that depending on the browser and resolution used by the participant, most analogies would be distorted due to shifting of the various objects included to set up the whole scene (again, cf. Figure 3.4). In particular, it was taken into consideration that the position of the colons (: / ::) in relation to the objects was an important factor in the perception of the symmetry within the analogy. Therefore, in order to prevent any confusion or distraction that may be caused by changes in the positioning of the colons, it was ultimately decided that a static representation of the entire analogy would be used for the nal experiment. The images were created using an external image processing application and saved in PNG format to obtain a high quality and to reduce data size. The resulting PNG images were then inserted at appropriate positions within the lab code on all relevant pages by using the HTML tag <img src=...>. By doing this, the amount of code could also be reduced which in turn facilitated work. The second way for the participants to solve geometric analogies entailed the possibility of constructing a solution for D from the available geometric objects within the so-called resources toolbar at the bottom of the screen (cf. Figure 3.5, p. 35). Essentially, this toolbar can be considered a special graphic area with certain useful characteristics. The purpose and functionality can be briey explained as follows: by clicking on the desired shape in the resources toolbar and simultaneously holding the mouse button, the participants could drag the selected object into the solution space for D. 34

Figure 3.5: Another way of completing the given analogy was to construct an own solution by hand Then, by releasing the mouse button again, they could drop their selected object into the desired position. Although it was only possible to drag and drop one object at a time, the participants were allowed to use any object from the toolbar as often as they wanted. Once they were satised with their solution for D, they clicked on the OK button to submit it and to proceed to the next page of the experiment. A typical sample code for the toolbar would look as follows: 6 7 @rec_[4,220|<b>Resources toolbar:</b>] @graph/[+example|20:20:0:270:h|45]

Analogous to the previous sample code (lines 1-5, p. 33), line 6 provides all necessary commands and parameters to generate the text Resources toolbar: in bold letters at the stated position (4,220). Line 7 is responsible for the appearance of the actual resources toolbar on the screen. As one can see (Figure 3.5), the toolbar itself contains numerous geometric shapes which rst of all have to be stored as images (GIF, PNG, JPG, etc.) in a certain local directory on the lab server. From there all images in the specied directory (here, it is again a folder called example) will be loaded and displayed in the toolbar. Directory names marked with an initial + cause the lab to display the associated toolbar, i.e., both in admin and guest user mode. Otherwise, the toolbar appears only when a user is logged in as an administrator. The remaining parameters in line 7 relate to the width and height (20:20), the modiable position (0:270), and the orientation (h = horizontal alignment; v = vertical alignment) of the toolbar. The initial scale of all object images that are inserted into the solution space is determined by the last parameter (here: 45). 35

The green trash can symbol in Figure 3.5 is also featured in the toolbar. If the participants needed to remove any of the objects from the solution space, all they had to do was to click and drag the desired object onto the trash can symbol and release it. In this way it was possible for them to subsequently modify a solution for D or to delete an unintentionally dragged object. The trash can is basically composed of a single line of code: 8 @obj_[585,183,45,45|objects/Trash]

As in the cases before, the parameters within the square brackets determine the relative position (585,183) as well as the size of the image (45,45) that represent the trash can. The higher the latter values, the larger the trash can image and, as a consequence, the larger the area that functions as a trash can. It is worth mentioning that if the source lename of any image contains the string trash, this object will function as a trash can so that the users attention is required to avoid unforeseen consequences. Going back to Figures 3.4 and 3.5, one can see several button-like elements which have not been addressed up to this point. However, they should not remain unmentioned since these kinds of buttons were used as central navigation and dedicated interaction elements in order to guide the participants through the lab pages, to let them submit their user data to the lab server, or to handle the stimuli in a pseudo randomized, xed sequence (the latter signicantly changed with the development and programming of the nal experiment). As indicated, dierent types of functional buttons were applied to set up the pretests and experiments. To start with, line 9 contains the sample code for a local link button (@loc) with the label NEXT written in bold letters. Functionally it has the purpose of skipping to the next branch (n) in the tree structure where all pages are organized (cf. Figure 3.6, p. 37). 9 10 11 @loc[n|<b>NEXT</b>] @loc[n,n,d,u,p|<b>Continue</b>] @loc[r,n,n,n,n,n|Random]

Instead of going to the next branch, local buttons can also cause the system to go to the previous branch (p), to the superordinate node (u), or to the subordinate node (d) of the tree. It is likewise possible to combine any of these commands to reach a desired page with a single button press as shown in line 10 of the sample code. A special case is illustrated by line 11 where the system randomly (r) selects one of the next ve (n,n,n,n,n) branches. In this way it is possible to access pages in an unpredetermined fashion. 36

Figure 3.6: A typical local button (NEXT) as used in the pretests and experiments The visible buttons in Figures 3.4 and 3.5 (labelled 1, 2, 3, and OK, respectively) do not only look dierent, they also pursue another functional purpose. Their related code is: 12 13 14 15 @sgs_[512,228|1|1:objects:+example|d] @sgs_[644,228|2|1:objects:+example|d] @sgs_[770,228|3|1:objects:+example|d] @sgs_[515,190|OK|1:objects:+example|d]

The @sgs command, standing for send graphic state, commonly stores the current state of a graphic in a certain parameter and then jumps to the specied target. In the cases at hand (lines 12-15), all graphics are stored in the parameter 1 and appended (+) to the le example in the objects folder. By doing so, it was possible to record both the participants choices (cf. Figure 3.4) and their individual creation for part D of the particular analogy (cf. Figure 3.5) in a desired log le. As usual, the leading numerical values such as 770,228 dene the buttons positions, the underscore characters after the @sgs command prevent the buttons from being moved in guest user mode, and, nally, the parameter d invisibly guides the participant to the subordinate node once the associated button has been pressed. As it can be quite cumbersome to only use the parameters d, n, u, or p in order to jump from the current to a required page, @sgs tags entail the useful option to directly specify the pathname of an existing lab page. Again, one method for the participants to solve the analogies that they were confronted with (cf. pp. 34 .) entailed the construction of a solution by resorting to the already discussed toolbar. After the participants had submitted their solution, they briey explained how they arrived at their particular solution in the previous step. For this purpose, a comment box was provided. Ultimately, the participants had to click on a button to submit their comment and to proceed with the experiment (cf. Figure 3.7, p. 38). 37

16 17 18

@form[comment] @box[d2|50:4] @form/[SEND|2:comment_U1-2:|objects:+RESULTS2|n]

Figure 3.7: Subjects were given the chance to comment on their created solution The sample code above is almost self-explanatory: line 16 announces a form (@form) of the type comment whereas line 17 denes the overall size of the comment box (4 lines, each time with 50 character spaces per line). Finally, line 18 contains the necessary closing tag for the form (@form/) as well as several parameters that are related to the submit button labelled with SEND. These appended parameters cause the lab system to store the user comment in the log le RESULTS2 within the folder objects at the very moment when the SEND button is pressed. Furthermore, the string comment_U12: is automatically attached to the front of the corresponding comment for better identication purposes. The last point of this subsection has special reference to the second pretest 38

and the nal experiment as they entail some technical aspects that have not been discussed yet. As stated in the introductory part (cf. pp. 28 .), the Analogy Lab occupies substantial back-end capabilities and accepts, among other things, HTML code or JavaScript. In particular, the latter two possibilities were taken advantage of for the second pretest. Besides a comment box, it was decided that labelled check boxes and radio buttons would be provided so that the participants could construct their comments according to the module principle (cf. Figures 3.8 and 3.9 below). To verify that at least one radio button or check box was selected, a short Java based script had to be written whose code is illustrated in Figure 3.10. This script simultaneously checked whether or not the comment box was empty and displayed an adequate pop-up message if that was true. For this purpose specically, there are some built-in tags for general text formatting that one may conveniently use. The opening tag <OSW:SRC> and the closing tag </OSW:SRC> have to be used for instance if one wants to embed raw HTML code and JavaScript (cf. Figure 3.10, p. 40). Such tags are equally applied to access and call external programs from within the lab (cf. pp. 42 .). These programs usually run in parallel to the Analogy Lab on a server and are going to be the topic of section 3.2.5. To specify, most of the programs presented therein were written for the purpose of the two supplementary experiments (TdT, Hit; cf. p. 28) and especially for the nal experiment.

Figure 3.8: Check boxes as used in the second pretest

39

Figure 3.9: Radio buttons as used in the second pretest

Figure 3.10: This Java script was used for the second pretest

3.2.4

Measure to prevent multiple participation

In order to prevent participants from taking part in the online test more than once, it was decided as a minimal measure to include a web browser cookie whose presence would be checked before entering the online setup stored in the Analogy Lab. When rst accessing the online test through the link on the CoUGAR homepage4 , participants were given the choice between the two language versions,
4

Link:

http://www.cogsci.uni-osnabrueck.de/~cougar/test/

40

German and English. Clicking on either opened a link to the Analogy Lab in a separate popup window (955 x 855 pixels with all toolbars removed) in the raw viewing mode (v = 2), i.e. ,only the main frame (cf. Figure 3.11).

Figure 3.11: Sample code to generate a separate pop-up window in raw viewing mode All potential participants were free to browse through the initial descriptive pages as many times as desired. Until the participant reached the Information page (cf. Figure 3.12), no cookies were set. Upon entering his or her personal information on this page and pressing the SEND button, a Java script was written that created and saved a browser cookie (e.g. under the name cougarf in the case of the nal experiment).

Figure 3.12: The information page of the nal experiment The intention was that once participants went past this Information page, they would complete the entire test and would therefore be excluded from participating a second time. The expiry date for the cookie was tentatively set for the end of the calendar year as it was seen as sucient time until the online experiment closed (cf. Figure 3.13). Within the HTML code for the website test portal, a Java script was included (cf. Figure 3.14) that checked for the presence of the cookie from the Analogy Lab. The script worked such that it established the existence of the cookie. If 41

Figure 3.13: The expiry date for the cookie was tentatively set for the end of the calendar year 2007 the cookie was already saved on the participants computer, it would not be possible to go through the test a second time using the same web browser and a popup message was displayed accordingly. If no such cookie was present, i.e., this was the rst time, the online test was accessed from the specic computer using the specic browser, the experiment continued as usual.

Figure 3.14: A Java script was written to check the presence of the cookie from the Analogy Lab

3.2.5

External programs

This nal section aims at presenting several programs all of which were written in Python (a general-purpose, high-level programming language) in order to support data analysis and to improve the usability of the conducted experiments, respectively. With the exception of the log parser that has to be executed locally, all other programs are stored on the lab server and called from within the Analogy Lab by means of specically placed tags. Special

42

emphasis will be placed on their functionality and less on the underlying code. Log parser The log parser has proven itself a powerful and versatile tool to facilitate data analysis, especially with respect to the nal experiment whose results were stored in a log le with thousands of entries. As it would have been a tedious task to parse such a log le by hand, the log parser was created before the start of the rst pretest. Ever since, the program has been improved and adapted according to the current needs. In order to function properly, one rst of all has to make sure that any log le is properly encoded in UTF-8, which then must be manually passed to the log parser. After the read-in process, the log is converted to comply with a predened, more suitable internal data structure. The parser then checks this newly structured log le for inconsistencies and lters out incomplete and redundant task entries. Records of users with incomplete data sets will no longer be incorporated in the nal results. In the case of redundant entries, the chronologically rst entry is chosen whereas all others are discarded. Next, all solutions for D that were once created by the participants in the course of the construction condition (cf. pp. 34 .) are clustered. Clusters are generated by letting the geometric objects within a given solution snap to a virtual grid whose mesh size is always determined by the smallest object within that particular solution. The object with the lowest, leftmost position is nally taken as the grids origin. All existing objects are then snapped to the nearest grid point. The signature of a separate object is dependent on its relative grid position, colour, shape, and size. The signatures of all available objects within a solution characterize the wholescene. Scenes with similar characteristics are regarded as identical and assigned to the same cluster. In this way, similarly created solutions for D are grouped together in a proper fashion. It is worth mentioning that the clustering process can be customized with respect to the mesh size and origin of the grid. A manual adaptation might be necessary if the relative position of the objects in D (measured along the vertical axis) in comparison to the C-part of the analogy plays a decisive role for the evaluation. Moreover, clusters can be manually merged by name if specied in order to facilitate the analysis. Once the clustering process is nished, the log parser generates a single SVG image per cluster comprising all associated user IDs, their solutions as well as their comments if available. As a last step, the parser splits the entire output le (among other things the participants reaction times and 43

choices as well as all cluster names are included) by generating several CSV les so that the resulting data packages can be handled by programs such as Microsoft Excel or Apples Numbers. Feedback generator The feedback generator was created in view of the nal experiment to give participants the possibility of comparing their task performance in relation to other participants. For this purpose, they saw one out of six predened sentences after the completion of the experiment. These sentences were presented on a separate lab page and suitably selected by the feedback generator according to the current participants average reaction time (fast; medium; slow) and the outcome of the clustering process (average; non-average). The corresponding tag read <OSW:feedback></OSW:feedback> and was embedded at a position within the lab code where the feedback sentence was intended to appear. First of all, the six sentences for the English version of the nal experiment were: (Accordingly, there were six German equivalents) 1. Fast & non-average: You quickly perceive the relevant relations in the analogies without getting distracted by the details. Your solutions are innovative and demonstrate a high degree of creativity. 2. Medium & non-average: The time you invest in analyzing the analogies and constructing their solutions is well balanced. The result is very original. 3. Slow & non-average: You are highly original in creating solutions to the analogies. Further, your observations are very thorough and thoughtful. 4. Fast & average: You quickly perceive the relevant relations of the analogy and dont get distracted by details. The solutions you create are highly preferred by the majority of participants. 5. Medium & average: You invest a sensible amount of time in the analysis of the analogies. The solutions you create are highly preferred by the majority of participants. 6. Slow & average: After thoughtful in-depth analysis of the analogies you extract the relevant relations and construct solutions that are very popular among the majority of participants.

44

As implied before, a single participant is classied by the feedback generator as fast, medium, or rather slow as a function of the resulting average reaction time. To this end, the time needed to complete the whole experiment is divided by the total number of tasks (note that e.g. the nal experiment comprised 30 dierent tasks in total). On the basis of the arithmetic means of all previously available data sets it is now determined whether or not a certain participant is x standard deviations away (the value for x can be dened by hand). People below this threshold of x standard deviations are considered to be fast. Otherwise, they are either regarded as average or even slow. Whether somebody can be rated as creative (non-average) or rather as normal (average) basically depends on the cluster into which a single participant can fall per task during the construction condition. More precisely, the total number of people per task thatshare the same cluster as the current participant is summed up. This result is then divided by the number of tasks and people that have performed exactly those tasks. In this context, a modiable threshold that can take a value between 0 and 1 comes into play. If this value is set for instance to 0.6, i.e., that people above these 60% will be classied as creative (non-average). People equal to or below this 60% threshold will consequently be regarded as normal (average). Stimuli randomizer Prior to the nal experiment, entire stimuli sets had consistently been pseudorandomized based on a pre-dened sequence. As this situation was rather unsatisfactory with respect to the validity of the nal data analysis, a dedicated program was written for the randomization of all available stimuli of the nal experiment in a more sophisticated manner. The surrounding tag <OSW:random> [...button code...] </OSW:random> was incorporated into the code of specic buttons to call the randomizer from within the lab. These buttons had to be pressed unavoidably by the participants if they wanted to proceed with the experiment. The stimuli randomizer keeps track of which stimuli pages exist, where on the lab server they are located as well as the fact that the nal experiment consists of altogether 30 tasks that have to be completed by each participant. Moreover, it is also fed with the information that the nal experiment is separated into three discrete blocks of tasks (see Experiments for details). Whenever a new participant requests his or her rst stimulus page by pressing a specic button, the randomizer is called whereupon an individual le is created to continuously store the progress of that participant. That is, after the tenth stimulus the randomizer forwards the current participant 45

to the tutorial of the second task and to sequentially pick out ten dierent stimuli to comply with the second block of the experiment. After the 20th stimulus, the randomizer consequently proceeds to assign another set of 10 stimuli from the rst block to the current participant in accordance with the third block. Once the 30th problem has been completed, the randomizer forwards the participant to the experiments nishing pages. It is worth mentioning that under all circumstances the randomizer ensures a uniform distribution of all stimuli depending on the count of already used stimuli. In this way, it cannot happen that a certain stimulus is chosen signicantly more often than the remaining stimuli. Progress bar The progress bar was introduced as a new feature during the development of the nal experiment. It conveys the progress of the running experiment. Simply put, the bar consists of 30 interchangeable PNG images, i.e., one image for each of the 30 tasks that the nal experiment comprised. The images are stored on the lab server and are appropriately loaded from within the Analogy Lab with the aid of specically placed tags (see below). For each participant, the program in control of the progress bar keeps careful track of the number of already completed tasks and returns the link to the correct image on the server to visualize the bar. The embedded tag on each relevant lab page to display the correct progress bar image read: <OSW:progress></OSW:progress> The background program in control of the progress bar always sourced necessary data from the stimuli randomizer. Thus, it was called by means of the same surrounding tag as the randomizer: <OSW:random> [...button code...] </OSW:random> This is the progress bar image referring to task 1 out of 30:

This is the progress bar image referring to task 30 out of 30:

46

3.3

Pretests

Having decided to make use of the Analogy Lab environment and its features to conduct the online experiments, preliminary tests were carried out to ensure usability of the platform as well as the stimuli created by the Experimental Group. Access to the all experiments was made available via a link on the CoUGAR homepage. Participation was limited to once per participant and only to one of the language versions since identical versions of the tests were created in German and English. In the end, there were two rounds of pretesting for the purpose of checking how the test could best be set up in order to make it both accessible for the participants and at the same time convenient for subsequent analysis of the data collected. In the rst pretest, the two methods of choosing and creating an own analogy solution were contrasted. Subsequently, in the second pretest, ways to collect more detailed information on the creation of solutions were evaluated. As a result of the data and feedback accumulated in the two pretests, the features and structure for the nal test could be netuned and additional modules were implemented to parse the results log, to cluster the solutions constructed by the participants as well as to provide feedback to the participants about their performance in relation to previous participants. In addition, an abbreviated version of the nal experiment was created for the purpose of collecting data in a more controlled environment and making nal adjustments for the nal online experiment. This version was run at two university events which enabled collection of fair amounts of data from the visitors.

3.3.1

First pretest

The rst online pretest was open for voluntary participation in the period from 17 to 28 August 2007. Participants A total of 83 session IDs were logged, of which 27 were removed automatically using the results log parser prior to statistical analysis because participants were either unable to perform the experiment completely or there was some sort of discrepancy in their results log or IP address. In sum, 56 valid sets of data were collected and included in the evaluation of the pretest. Participants were 56 selected acquaintances of the project members, ranging in age from 19 to 48 years. There were 30 males and 24 females with more than half being students (60.7%) in various disciplines of study. Half 47

of the participants were from Germany and others from all over the world and 62.5% of the participants opted for the German version of the test. The majority were right-handed and had never taken an IQ test before.

Procedure Participants sat in front of a computer screen and completed the pretest online. They were instructed to solve geometric analogies of the form A:B::C:D (read A is to B as C is to D). Their task was to complete D. Before the actual test began, participants completed an information form to provide information about their gender, nationality, handedness, occupation, area of studies (if applicable) and when they had last taken an IQ test. These factors, though not our primary focus in the experiment, were examined for possible preference correlations. The test consisted of two blocks, the choice (C) and the make (M) conditions. In the C condition the analogy solution was to be selected from among three alternatives (multiple choice). In the M condition participants created the solutions themselves by dragging and dropping shapes from a toolbar onto the solution space on the screen. Further, in the M condition participants were asked to comment on how they had come up with their solution (i.e., what reasoning they had applied). The purpose of this was to have an explanation (possibly already in terms of a rudimentary Gestalt interpretation) for potential novel solutions that we had not thought of beforehand. Materials Participants were randomly presented with one of four pseudo-random combinations (listed in A.4) of 16 analogies (listed in A.1). Each combination consisted of a C condition followed by an M condition, each consisting of 8 trials. Further, half of the analogies had unambiguous solutions (U); the other half comprised ambiguous analogies (A), the solutions to which had previously been classied in terms of dierent Gestalt laws. The A analogies were further divided into subgroups: one group of four analogies (A1-1, A1-4, A2-1, A2-4) consisted of variations of each other, diering in features such as shape, color, dimension, and distance of objects to one another. Similarly with a second group of three analogies (A1-2, A2-2, A2-3) and an additional, unrelated one. The motivation for varying features was to test whether a certain solution preference was robust or whether it could be easily manipulated by varying certain features. 48

The multiple choice answers for the U analogies consisted of the correct answer plus two additional incorrect answers. For the A analogies, the answers consisted of the ambiguous alternatives. The U and A groups were further subdivided into U1, U2 and A1, A2, consisting of four analogies each. This was a purely technical decision: in half of the combinations U1 occurred in the C condition, in the other half it occurred in the M condition. Thus, a pseudo-random distribution of analogies was ensured over the test. Results Due to the Analogy Labs randomization function, the dierent combinations occurred a dierent number of times. The majority of participants (23) were assigned to Combo1 (41.4%), followed by 12 each in Combo2 and Combo3 (both 21.4%) and 9 in Combo4(16.1%). Thus, A1 and U1 occurred 21 times, A2 and U2 35 times in the C condition. In general, solution preferences are to be interpreted as tendencies, and not as statistically signicant preference dierences. Relative answer frequencies for the C condition are shown in Figures 3.15 and 3.17, for the M condition in Figures 3.16 and 3.18. Comparison of the C and M conditions by analogy are provided in Appendix A.5. Unambiguous analogies There were clear answer preferences for the unambiguous analogies, as expected. (For specic solutions and distributions of the solutions to each analogy, see Appendix A.5.) It should be noted, however, that U2-1 and U2-3 showed a second solution preference of 11% in the C condition. Since this is not repeated in the M condition, this second choice is attributed to lack of attention to detail in what could be considered the easier task of choosing from pre-given solutions. In the M condition, participants created an additional interesting solution for U1-1 (25%) and U2-1 (15%). In both cases, solution 4 was created by introducing an axis (horizontal in U1-1 and vertical in U2-1) along which the gure in A is reected. From such cases, it can be seen that some of the analogies which had previously been categorized as unambiguous and for which participants were presented with pre-given solutions in the C condition needed further modication and that interesting and creative results could mainly be obtained from the task in the M condition.

49

Figure 3.15: Relative frequenciesunambiguous relative frequencies of unambiguous analogies frequencies condition in the C Choice: Choice: unambiguous relative
100 100 75 75

50

50

25

25

U1-1-CP

U1-2-CP U1-1-CP

U1-3-CP U1-2-CP

U1-4-CP U1-3-CP

U2-1-CP U1-4-CP

U2-2-CP U2-1-CP

U2-3-CP U2-2-CP

U2-4-CP U2-3-CP

U2-4-CP

Answer 1

Answer 2

Answer 3

Figure 3.16: Relative frequencies of unambiguous analogies in the M condition Make: unambiguous relative frequencies
100 75

50

25

U1-1-DP

U1-2-DP

U1-3-DP

U1-4-DP

U2-1-DP

U2-2-DP

U2-3-DP

U2-4-DP

Answer 1

Answer 2

Answer 3

Answer 4

Answer 5

Ambiguous analogies Compared to the unambiguous analogies, answers for the ambiguous analogies were more evenly distributed. Additionally, many more dierent solutions were created in the M condition. Although in general there was not enough data to determine whether seeming dierences in answer preferences are signicant, these dierences again are interpreted solely as tendencies. The rst group of ambiguous analogies (A1-1, A1-4, A2-1, A2-4) showed a general preference for color switching, especially in A1-1 and A2-4-M. The second preferred solution was generated by rotating the entire gure by 180 degrees. There was an increase in preference for this solution upon introducing an additional shape to the analogy. The third solution, which was explained in terms of movement of the black object upwards, was highly

50

Figure 3.17: Relative frequencies of ambiguous analogies in the C condition

Choice: Ambiguous relative frequencies
100

75

50

25

Choice
0 A1-1-CP A1-2-CP A1-3-CP A1-4-CP A2-1-CP A2-2-CP A2-3-CP A2-4-CP

Answer 1

Answer 2

Answer 3

Choice

dispreferred in the C condition, except in A2-1. This may be explained by the rectangular objects giving the impression of two parallel aligned lines, which in turn may facilitate perception of movement (i.e., following the Law of Good Continuation). Comparing the C and M conditions revealed that the Good Continuation solution was more preferred in the M condition than in the C condition for A1-1 and A1-4. This may be due to the fact that participants perceive or represent the problem dierently or reason about it dierently when asked to create an own solution than when asked to choose from among given alternatives. Figure 3.18: Relative frequencies of ambiguous analogiesfrequencies M condition in the Make: Ambiguous relativeAmbiguous relative Make: frequencies
60 60 45 45

30

30

15

15

A1-1-DP

A1-2-DP A1-1-DP

A1-3-DP A1-2-DP

A1-4-DP A1-3-DP

A2-1-DP A1-4-DP

A2-2-DP A2-1-DP

A2-3-DP A2-2-DP

A2-4-DP A2-3-DP

A2-4-DP

Answer 1 Answer 8

Answer 2 Answer 9

Answer 3 Answer 10

Answer 4 Answer 11

Answer 5 Answer 12

Answer 6

Answer 7

It can be clearly seen from the many plausible solutions that were created by participants in the M condition that detailed analysis of the explanations given by participants of how they came to their solutions is needed. In terms of categorizing the comments received (if at all) and the expressions used 51

for their explanations, the task of unifying the varied explanations was extremely complex and based on subjective interpretation of the oftentimes intuitive and vague statements left behind by the participants. Thus, assigning preferred Gestalts to dierent solutions without a large margin of error was dicult.

3.3.2

Second pretest

The second pretest was run from 19 October 2007 to 7 November 2007. Participants Complete data sets were obtained from 44 participants who were acquainted with members of the study project. To facilitate data analysis, demographic information obtained from the participants was done by selecting a response from a given list of options (instead of free input) for all categories. The participants ranged in age from below 20 to over 40 years of age with most participants falling into the 20-25 age group. There were 33 males and 10 females with 40 right-handed participants, one left-handed participant and one ambidextrous participant. With respect to familiarity with geometric proportional analogies, IQ test-taking history showed that most participants had never taken an IQ test before, and if they had, this had been the case in the last 1-5 years. In order to see if a mathematical/logic background would have an eect in terms of familiarity in solving geometric analogy problems, a question was included on area of study. Participants background revealed no dominant discipline. Almost half of the participants were from Germany, with the rest spread out from dierent countries in the world. Students and employees made up a majority of the participants. (For details, see Appendix B.5.) Procedure The procedure was a modied and extended version of the make condition of examples rst used in the rst pretest. In particular, since the comment analysis for the rst pretest proved to be extremely complicated due to the diversity in explanations given for the construction of the experiments, the second pretest was primarily used to test whether a more homogenous clustering of the solution comments could be achieved. Again, participants sat in front of a computer screen and completed the pretest online. They were instructed to complete geometric analogies of the form A:B::C:D by constructing the solution for D using the objects available 52

to them in the toolbar. The test consisted of two conditions, the checkboxes (Check) and the radio buttons (Radio) conditions. In the Check condition, participants were given a multiple selection option to indicate all the operations (a-j) that were applied to the C gure to derive at their solution for the analogy. In the Radio condition, the participants were asked to choose from a number of statements (a-f) describing a possible set of operations to derive at a correct solution and select one that best matched their strategy. Materials In alternating order, participants were assigned to either the Check or Radio condition. Within each condition, participants were presented with one of four pseudo-random combinations of 24 analogies (see Appendix B.3). In total, there were 42 examples (six original analogies with six modications each). For each of the six original examples, modications were made systematically resulting in example classes, each consisting of the original, three with modications to the objects in the A (and/or B) source domain (i.e. all gures in the C remain as in the original) plus an additional three with modications to the objects in the C target domain. These variations of the original with xed A (and/or B) or xed C domains were divided equally into two mixes (see Appendix B.2) to create two combinations of 24 examples each, including the six original examples in all combinations. The motivation for varying features in the source and target domain, respectively, was to determine which manipulations of features would most inuence the interpretation of the analogy in terms of preferred Gestalt principles. By keeping one part of the analogy constant throughout all examples in each of the six example classes, the aim was to nd out the most robust features and/or inuential manipulations. Following the page on which the participants were asked to create an own solution to each analogy problem using the objects in the toolbar of the Analogy Lab, the analogy was again displayed in conjunction with the solution created by the participant. Depending on the condition, the participant was asked to select either multiple options from a list of nine operations or one statement from a list of ve sentences. The operations and statements were devised from the most commonly mentioned explanations for the solutions created in the rst pretest (see Appendix B.4). In the Check condition, participants were given a list of nine operations that could possibly be performed on the object(s), separately or in some combination, to arrive at the solution. Participants were asked to select all that applied in the creation of their solution. Alternatively, in the Radio condition, participants were given a list of ve sentences that represented possible 53

sequences of operations performed to arrive at a correct solution, from which participants were asked to select one only, that which most matched their own solution strategy. In both cases, an option to add a comment was included in case none of the operations or statements seemed adequate. It should be noted that one of the Radio button statements was always an incorrect option to ensure that participants did indeed read through the statements. In sum, there were two randomized sequences created of each of the two mixes to create four combinations of 24 analogies each. Thus, all participants solved all the original examples (S1 to S6) in addition to three sets of three variations in either the A (and/or B) (A1 to A3) and the C domains (C1 to C3). Each participant was asked to create solutions to 24 analogies and comment on their solution, either by selecting one or multiple checkboxes or one radio button statement. Results The second pretest was prematurely terminated as it became clear that it would not be possible to achieve a large enough sample size to obtain significant results. Furthermore, the test was deemed too long, time-consuming, monotonous and complicated so that it could not generate sucient interest among participants to complete the whole experiment. From the feedback received it became clear that the slight variations made to each of the original examples were in fact too small for an experiment participant who instead of focusing on the details viewed the analogies displayed as somewhat repetitive and did not give each the full attention necessary to tease apart and appreciate the minute dierences. In addition, the task of reading through and selecting the appropriate checkboxes and radio button statements took much longer than expected, especially since many participants created their solutions based on intuitions that they found, as already discovered in the rst pretest, to be dicult to express in words. All of the six original analogies were solved by all participants. For the other examples, the number of participants ranged from 22 to 27. The results logged in the Analogy Lab were parsed and the clusters manually adjusted. Although an initial attempt was made to classify the solutions according to Gestalt principles based on the result of selected checkboxes and radio buttons, the combinatorial variation proved too great to draw any conclusions. The following gures show the relative frequencies of the preferred solution clusters and their visualization. Due to the limited amount of data, no signicant conclusions could be drawn and the distribution of solutions created are to be viewed as pure tendencies. (All solutions that were created by two participants or less, though included in the plots, are not displayed.) 54

S1 In the S1 series, the original example was modied to test the eect of adding a third color to counter grouping by similarity (S1A1), changing the initial shape to rectangles that created an illusion of being more closely connected and thus highlighting movement following the Law of Good Continuation (S1A2), highlighting positioning of dierent parts of a bigger object following proximity (S1A3) and respective changes to the C domain, i.e., adding a third color in the C domain (S1C1), changing the shapes in the C domain to create the illusion of connectedness (S1C2) and highlighting positioning of dierent parts of a bigger object (S1C3). In sum, it could be argued that the stability of the preference for movement can be manipulated by dierent feature changes and may not be as robust throughout the variations of the original example in this class. Original For the original example in the S1 series, three solutions were created most frequently.

::

?
40
S1Original

4 12

4 2

16 0.35 0.30 11 0.25 0.20 0.15 5 0.10 2 0.05 0.00 4_12 4_11 4_20 4_15 4_17 4_0 4_2 0_0 1_0 2_0 4_3 4_6 4_7 2 1 1 1 1 1 1 1 1

n=44

Solutions: 1. Group C into two separate objects and move black circle to top right 2. Group C into single unit (according to proximity) and rotate entire unit 180 degrees 55

3. Group C into single unit and switch colors The variations were created to gain a better understanding of the separation into dierent objects (whether this was inuenced by positioning, similarity/dissimilarity of color, shape, etc.) and if perceived movement of objects was aected by whether the shapes touched or indicated a clear direction of movement. S1A1 Variation to test inuence of additional dissimilarity of color
: :: :

?
45 40 4 11

Solutions: 1. Group C into two separate objects and replace top part corresponding to A with B 2. Group C into single unit (according to proximity), switch colors (white to grey) and rotate entire unit 180 degrees 3. Group C into single unit and switch colors (white to black and black to grey) The preference order remained unchanged despite the introduction of a third color.

56

S1A2 Variation to test inuence of connectedness of shapes

: :: :

?
40 4 11 4 2 44

Solutions: 1. Move black circle to top right 2. Group C into two separate objects and switch positions of top and bottom 3. Group C into single unit and rotate entire unit 180 degrees 4. Group C into single unit and reect along diagonal axis Rectangles replaced the circles in A and B. It was reasoned that the two rectangles seemed more connected to each other, forming a line which could be followed, thus enhancing the eect of movement of the black object. Although there was no clear preference, it is noteworthy that the simple color-switch solution was not among the top three solutions.

57

S1A3 Variation to test inuence of position

::

?
41 40 43

Solutions: 1. Group C into two separate objects and switch positions of top and bottom 2. Group C into single unit and reect entire unit along horizontal axis 3. Group C into single unit and rotate entire unit 180 degrees A straight vertical line of four circles replaced the circles in A and B. It was reasoned that a sense of two contrasting halves on top of each other would be created. The preference for the position switch and reecting the unit along a vertical axis located along the midline of C support the intended purpose of the variation.

58

S1C1 Variation to test inuence of color

: :: :

?
43 45 40

Solutions: 1. Group C into single unit and rotate entire unit 180 degrees 2. Group C into two separate objects, move black circle to top right by switching with grey circle in that position 3. Group C into single unit and reect entire unit along diagonal axis Similar to S1A1, an additional color was introduced. This time, however, it was in C. This resulted in an increased preference for the rotation solution. S1C2 Variation to test inuence of connectedness of shapes

::

?
40 42 41

59

Solutions: 1. Group C into single unit and rotate entire unit 180 degrees 2. Group C into two separate objects, move black rectangle to top right 3. Group C into single unit and reect along diagonal axis Similar to S1A2, rectangles replaced the circles, albeit in the C domain. The eect was a slight enhancement of the rotation solution, but also of the movement solution. S1C3 Variation to test inuence of similarity of shapes
: :: :

?
40 41

60

Solutions: 1. Group C into single unit and switch colors 2. Group C into single unit and rotate entire unit 180 degrees A diagonal line of circles replaced the four circles in the C domain to create a line of continuity. This resulted in increased occurrence of the colorswitching strategy. The second most often created solution involving rotation does not yield a gure dierent from that in C.

S2 In the S2 series, the original example was modied to study the eect of changing the color across the analogy of the central object (S2A1), varying the size of the center (S2A2), changing within the analogy the color of the central object (S2A3), osetting the center (S2C1), countering the color and position of the central object (S2C2) and manipulating the symmetry around the center (S2C3) in order to determine exactly which rule was applied in arriving at the most preferred solution. In sum, it should be noted that having a black and central circle had a persistent eect as the object to focus making changes on when creating solutions to the analogy. Attention seems to automatically be drawn to the object demonstrating the attribute of centrality and blackness. Original

::

?
10

61

Solution: Retain black square. Although there is only one solution to this unambiguous analogy, it should be noted that there are four dierent rules that could apply to arrive at the same solution, i.e., 1. Retain black objects 2. Remove white objects 3. Retain central objects 4. Remove outer objects Variations were created to test which of these rules applied and how each could be emphasized by manipulation of the objects. It is striking that the stark contrast created by the black square against the white circles makes the square such a focus of perception. S2A1 Variation to test inuence of color and centrality

::

?
10 21

62

Solutions: 1. Retain central objects 2. Remove black objects Given the two competing features of interest of the central black square, the modication was created to contrast centrality and color (black). The preferred solution was to retain the central object, regardless of color. S2A2 Variation to test inuence of centrality

::

?
10 31

63

Solutions: 1. Retain central objects 2. Remove grey objects Another line of shapes and an additional color was introduced to further highlight the centrality of the black square. Although the preferred solution is to attribute centrality solely to the black square, solution cluster 3 1 shows that the white circles can also be seen as creating a group with the black square if given an additional color and/or shapes. S2A3 Variation to test inuence of color and position
: :: :

?
10 11

Solutions: 1. Retain black objects and move downward 2. Retain top objects and move downward The solution for this variation required manipulation on the non-centrally located black shapes. Still, the preference was for keeping the black circles and shifting them down to the center analogous with the black square in C instead of viewing the black circles as the top objects and moving those down. 64

S2C1 Variation to test inuence of color, centrality and position

::

?
10 40

Solutions: 1. Retain most central objects 2. Remove white objects A modication of the C domain was created by extending the gure with additional objects of dierent shape and color. Again, the preference to keep only the central black square was strongest. S2C2 Variation to test inuence of color and centrality
: :: :

?
11 20

65

Solutions: 1. Retain most central objects 2. Remove white objects By modifying the object in C to have as its central object not a black shape, but a white circle, the same shape and color as in the previous analogy, the features of central position and color were pitted against each other. The tendency was to prefer to retain the most central object, rather than remove the white objects. S2C3 Variation to test inuence of color and centrality

::

?
10 20

66

Solutions: 1. Retain most central objects 2. Remove white objects Despite adding more of the same shape and color objects as those removed in the previous modications, the solution of keeping the central black object was still preferred.

S3 In the S3 series, the original example was modied to test the eect of pitting reection against rotation as a preferred operation. A triangle was used since the directionality of the vertex could reveal which of the two operations were applied. The variation involved using dierent colored triangles (S3A1), contrasting colored triangles (as a group in S3A2 and separate in S3A3), contrastingly angled triangles (as a group in S3C1 and separate in S3C3) and addition of triangles (S3C2). In sum, it could be said that the manipulations with dierent colored and angled triangles made visible the contrast between the reection and rotation solutions. It appears that there is no clear preference for one over the other, even when the arrangement of the shapes would call for it. Original
: :: :

?
23 21

67

Solutions: 1. Group C into single unit and rotate entire unit 180 degrees 2. Group C into single unit and reect along horizontal axis Since the gure remains the same even after possible reection, the variations created were an attempt to distinguish the operations applied. S3A1 Variation to test inuence of separation of objects by color
: :: :

?
22 20

Solutions: 1. Reect only left object along horizontal axis 2. Rotate only left object 180 degrees Dierent colors for the same object were used in the AB domain to highlight the change in only one of the objects. The asymmetry of the triangle reection along a horizontal axis and rotation made it easier to see which operation was used. In terms of preferred solution, however, both were used with almost equal frequency.

68

S3A2 Variation to test inuence of grouping of objects

: :: :

?
20

Solution: Group C into single unit and rotate 180 degrees /reect along vertical axis Contrasting colors were used for the triangles to bring the movement of the gure as a whole to the fore and this was also the only solution to be created. It cannot be distinguished, however, which operation was used. S3A3 Variation to test inuence of separation of objects by color
: :: :

?
20

Solution: Group C into two separate objects and rotate each 180 degrees 69

Contrasting colors were again used to further demonstrate rotation of each separate object, which was the most commonly created solution. S3C1 Variation to test inuence of color
: :: :

?
22

Solutions: Group C into single unit and rotate 180 degrees/reect along horizontal axis An additional object in a dierent color was added in the C domain. Although only one solution was highly preferred, it cannot be claried which operation was used more often. S3C2 Variation to test inuence of grouping by color
: :: :

?
35 33

70

Solutions: 1. Group C into single unit and rotate entire unit 180 degrees 2. Group C into single unit and reect along horizontal axis In contrast to S3C1, the gure in the C domain appears dierent after rotation and after reection, making clear that for this analogy, the preferred operation is the rotation of the group. S3C3 Variation to test inuence of separation by color
: :: :

?
20 25 23 21

Solutions: 1. Group C into single unit and reect along horizontal axis 2. Group C into single unit and rotate entire unit 180 degrees 71

3. Group C into two separate parts and rotate each triangle 180 degrees 4. Group C into two separate parts and rotate grey triangle 180 degrees In contrast to S3C2, the asymmetric gure comprising only two triangles decreases the rotation preference slightly with the reection solution also being applied. Furthermore, other solutions involving separate rotation of the objects is considered more than in S3C1.

S4 Similar to the S3 series, in the S4 series, reection and rotation were measured against each other by manipulation of color (S4A1 and S4A3), addition of objects (S4A2 and S4C1) and both (S4C2) and changes in the distances between objects (S4C3). Given the circles and squares, which in reection and rotation about their own centers remain unchanged in shape, the manipulations could highlight the exclusive use of one operation over the other (namely rotation) as reected in the preferred solutions. Original
: :: :

?
21 22

Solutions: 1. Group C into single unit and rotate 180 degrees 2. Group C into single unit and reect along vertical axis This example class aims at teasing apart dierences in the application of reection and rotation. 72

S4A1 Variation to test inuence of addition of color switching

: :: :

?
22 21

Solutions: 1. Group C into single unit, rotate 180 degrees and switch colors 2. Group C into single unit, reect along vertical axis and switch colors By adding the need to switch colors, preference for the reection solution was slightly increased. S4A2 Variation to test inuence of addition of shape
: :: :

?
21 20

73

Solutions: 1. Group C into single unit and reect along vertical axis 2. Group C into single unit and rotate clockwise 90 degrees The more complex gure in the AB domain in the variation was intended to increase the perception of rotation as a solution. As the solution preference indicates, this was indeed the case with reection no longer being as highly preferred. S4A3 Variation to test inuence of grouping by color and shape
: :: :

?
20

Solution: Group C into single unit and reect along horizontal axis By eliminating the possibility of rotation due to the asymmetric triangle rotation of the gure in A, the reection solution became the obvious highly preferred solution. S4C1 Variation to test inuence of addition of shapes
: :: :

?
40

74

Solution: Group C into single unit and rotate 180 degrees /reect along vertical axis By adding more objects, a direct object switching solution was suppressed. However, although there is only one preferred solution, the operations used is not clear from just the visual representation. S4C2 Variation to test inuence of separation by color and shape
: :: :

?
22 21

Solutions: 1. Group C into single unit and rotate 180 degrees 2. Group C into single unit and reect along vertical axis This variation was again an example of visualizing the rotation versus reection solutions with a dierent composition of shapes for the gure in C, with the rotation being slightly more preferred than the reection. 75

S4C3 Variation to test inuence of separation by position

: :: :

?
20 21 24

Solutions: 1. Group C into single unit and rotate 180 degrees 2. Group C into single unit and reect along vertical axis 3. Group C into two separate parts and switch positions By creating more distance between the two objects in C, it was intended to emphasize an object switch solution, which was not as obvious in S4C2. Indeed, this solution 2 4 appeared as a possible candidate by this modication.

S5 In the S5 series, the running example was modied to see the eects of changes to the analogy in order to distinguish the direction (up versus down) of the movement of objects and to which object the movement would be applied. This was done by addition of black circles (S5A1), changing of the object to rectangles which suggest a directionality in movement (S5A2 in the AB domains, S5C3 in the C domain and multiple addition in S5A3) and removing the dierence among the object shapes (S5C1 and S5C2). It is clear from the solutions that the downward movement is a prominent feature of the analogy. The salience of the black circle is again obvious in this 76

running example with participants tending to apply changes to this object rather than another. Original Running example
: :: :

?
25 20

Solutions: 1. Group C into two separate objects and move white square down/switch objects 2. Group C into two separate objects and move black circle up This class contains modications to the running example. For the original example, as had been the result previously, both the solution to move the black circle up toward the square and that to move the white square down toward the circle were fairly equally preferred. The modications were created to stress one of the directions.

77

S5A1 Variation to test inuence of movement by multiple objects

::

?
61 60

Solutions: 1. Group C into two separate objects and move white squares down 2. Group C into two separate objects and move black circles up By adding more black circles in the B domain it was intended to stress the downward movement. The result, however, indicates that additional objects did not necessarily inuence the good continuation of the downward movement or that of the black circles suciently strongly. Both solutions were still chosen fairly equally. S5A2 Variation to test inuence of downward movement by single object

::

?
21

78

Solution: Group C into two separate objects and move white square down With the similar intention to accentuate the downward movement by using rectangles that formed a continuous line of movement, S5A2 succeeded in making the downward movement of the white square more prominent. S5A3 Variation to test inuence of downward movement by multiple objects

::

?
60

Solution: Group C into two separate objects, repeat three times and move white squares down

79

As in S5A2, the rectangles accentuated the downward movement in contrast to the circles which had not done so in S5A1. One consideration is the strong eect of the black circle, which seemingly becomes a focal point of manipulation in creating a solution. S5C1 Variation to test inuence of similarity of objects
: :: :

?
23

Solution: Group C into two separate objects and move top white square down By removing considerations of the shape of the moving objects, the downward movement created by the highly focal black circle is simply applied to the top white square. S5C2 Variation to test inuence of color and similarity of objects

::

?
20 23

80

Solutions: 1. Group C into two separate objects and move top black circle down 2. Group C into two separate objects and move black circles down Viewed in conjunction with S5C1, the results for the solution preferences suggests again that the movement of the black circles is taken into consideration more than if they did not appear in the C domain. S5C3 Variation to inuence movement by shape

::

?
25 21 23

81

Solutions: 1. Group C into two separate objects and move black rectangle up 2. Group C into two separate objects and move white square down 3. Group C into two separate objects and move black rectangle down A downward-pointing rectangle was added to further enhance the eect of the downward movement for black objects. However, the solution created by moving the rectangle further downward (2 3) was less preferred than the moving of the rectangle in the opposing direction upward (2 5) and also that of moving the square down (2 1) involving continuation of the movement in a downward direction.

S6 Similar to the S4 series, the rotation versus reection distinction for gures comprising multiple objects is investigated in the variations of the original example in the S6 series. The rotation solution is made more obvious by connecting the shapes comprising a whole gure (S6A1 and S6A2) and manipulating the color and shape distribution among the multiple objects (all remaining variations). In sum, it would seem from the preferences of the solution in this class that the solution involving grouping together of all objects into a whole and rotation of that group is heavily preferred when multiple objects are present. Original
: :: :

?
40 42

82

Solutions: 1. Group C into single unit and rotate counterclockwise 90 degrees 2. *Group C into single unit and rotate clockwise 90 degrees Although there is only one preferred correct solution, it should be noted that there are three dierent rules that could apply to arrive at the same solution, i.e., 1. Group C into single unit and rotate counterclockwise 90 degrees 2. Group C into single unit and reect along diagonal axis 3. Group C into two separate parts and switch colors of the bottom two objects The S6 series investigated the eect of modications on the preference of rotation of the gure in C as a whole versus reection along diagonal axis and possible color switching strategies. As solution cluster 4 0 suggests, it is not clear which operation was used. (As for solution 4 2, it is considered an incorrectly created solution, though it did occur three times.) S6A1 Variation to inuence grouping
: :: :

?
40 43

83

Solutions: 1. Group C into single unit and rotate counterclockwise 90 degrees 2. Group C into two separate parts and switch colors of bottom two objects/retain all shapes and rotate color lling in counterclockwise direction The intention of the modication was for the connectedness of the ve instead of four circles to give rise to the preference for the rotation solution, which was the most preferred solution. Again, it is not clear whether a diagonal reection was applied as the two operations cannot be distinguished from each other by mere visualization of the solution. Although only created by two participants, solution cluster 4 3 is shown here to point out that the color switching solution was also considered. S6A2 Variation to test inuence of grouping by color and shape
: :: :

?
40

84

Solution: Group C into single unit and rotate counterclockwise 90 degrees By using all squares that touch each other to create a connected gure in the AB domains, the reection strategy was eliminated. In this way, it can be clearly seen that the rotation solution was the most prevalent of the preferred solutions. S6A3 Variation to inuence separation by color and shape
: :: :

?
45 42

Solutions: 1. Retain shapes for each separate object and rotate black color lling 90 degrees

85

2. Separate C into separate parts and switch colors (black to white and white to black) By adding the grey color and thereby focusing attention on the changes of the contrasting black and white objects, the color switch strategy was enhanced. In the 4 5 solution cluster, the top black circles were considered extraneous (and remained the same color as the grey did in the AB domains) and only the bottom objects were modied; while in the less common 4 2 solution cluster, the black-white color switching was applied to each object. S6C1 Variation to test inuence of separation by shape
: :: :

?
43 40

Solutions: 1. Group C into single unit and rotate counterclockwise 90 degrees 2. Group C into single unit and reect along diagonal axis The modication in C was to include a gure which was symmetrical in shape along the diagonal axis. The asymmetry of the colors seems to have aected the preference for the rotation solution, while the diagonal reection/ color moving solution was slightly enhanced. S6C2

86

Variation to test grouping by color

: :: :

?
41 43

Solutions: 1. Group C into single unit and rotate counterclockwise 90 degrees 2. Group C into single unit and reect along diagonal axis The same eect as in S6C2 was found despite the asymmetrical shape of the gure in C. S6C3 Variation to test inuence of grouping by shape and color
: :: :

?
40

87

Solution: Group C into single unit and rotate counterclockwise 90 degrees By removing all color discrepancies, the rotation/diagonal reection solution was clearly preferred. Categorization of comments As mentioned, the sample size was too small to determine any signicant similarities between the explanations that participants gave for the creation of their solutions. Furthermore, the data to be analyzed was so disorganized that it was not possible to do a systematic classication of the operations used, as reported by the participants. In the Check condition, the number of checkboxes and their possible combinations made it impossible to achieve commonality among the 44 participants. As for the Radio condition, the constraint placed on the freedom of expression by the pre-given statement may have caused unease among participants who were not certain as how to verbalize their intuitions on their analogy-solving strategy. Given the inability to distinguish between competing Gestalt principles at play in the analogies from pure visualization of the solutions, it is of utmost importance to analyze the comments given by the participants so as to further classify the solution clusters intoGestalt clusters, which would be used for the formalization of the analogies. In sum, the conclusion drawn for the nal test from the second pretest, whose primary purpose was to decide between the use of checkboxes and radio buttons for the facilitation of data analysis, was to return to voluntary commenting with explicit instructions as to the vocabulary of operations to be used.

3.3.3

Mini tests: HIT and Technologietag

Participants The participants were visitors of two public events, the Technologietag 20075 (TT) and the Hochschulinformationstag6 (HIT), who volunteered to take part in the experiment. In total there were 112 participants, from which 103 usable datasets were collected (50 from the TT and 53 from the HIT). The participants age ranged across all age groups - from younger than 20 to
The Technologietag 2007 was an exhibition for people interested in computer science and technology (http://project.informatik.uni-osnabrueck.de/tt2007/). 6 The Hochschulinformationstag is an annual exposition day of the study programs at the University of Osnabrck intended for pupils of the local schools (http://www.zsb.uniu osnabrueck.de/hit.html).
5

88

over 40 years. Fifty participants were female, 52 were male, and one did not specify gender. Seventy-four participants (72% of total number) were German; the others were from various countries worldwide. The participants were not paid for their contribution to the experiment. Materials and design For the present study we created 21 analogies composed of circles, squares, triangles and rectangles in three dierent colors - white, black and grey. From these 21 analogies three were so-called original analogies, from each of which six variations were derived. We varied each original analogy three times within the source domain, and three times within the target domain. Thus in the end we had three original analogies and eighteen variations. Figure 3.19 shows an analogy with two variations. All analogies were ambiguous and allowed for dierent plausible solutions. The variations were created in such a way that dierent Gestalt laws were emphasized to a dierent degree and therefore meant to trigger dierent solutions. We focused on four Gestalt principals: proximity, similarity, symmetry and good continuation. The extent of varying the analogies was limited to position, shape, number, and color. Figure 3.19 (top) shows an example of a variation in the source domain. Adding new elements to the B domain of the source creates a new notion: guided by the Gestalt law of good continuation humans might perceive the circles to be arranged in a descending order. The second variation in Figure 3.19 (bottom) adds a new element to the target domain, causing a change in perception, which is this time triggered by the Law of Proximity: the new element (white circle) creates a new relation by means of proximity. In the experiment we used a software tool especially developed for this purpose: the Analogy Lab7 (Figure 3.20) is a web-based framework for constructing solutions to geometric proportional analogies via drag and drop. It also provides a clustering algorithm to classify the solutions constructed by the participants. For the construction of the solutions a toolbar with all the necessary geometric objects was available and there was a trash can symbol for throwing away undesired objects (Figure 3.20). The objects in A, B and C were not movable. There was neither a time limit to construct the solutions, nor a limit to the number of objects in the toolbar that could be used. A button was available for proceeding to the next page. In the experiment, the Analogy Lab was running on standard laptop computers (two at the TT and three at the HIT) with a 15 screen and a resolution of 1024x768 pixels or above.
7

http://mvc.ikw.uos.de/labs/cc.php

89

Figure 3.19: An analogy, a variation in the source domain and a variation in the target domain

Figure 3.20: Screenshot of the Analogy Lab while creating a solution, with additionally labeled toolbar and trash can 90

Figure 3.21: Screenshot of the Analogy Lab while writing in the optional comment box Procedure Participation in the experiment would begin as the participants sat down in front of one of the laptops at the site of the experiment. The Analogy Lab browser window was always open showing the starting page of the experiment before a participant started taking part. The experiment took place in a semicontrolled environment: an experimenter was present at all times, but people passed by freely in the vicinity of the experiment site. The experiment consisted of solving a sequence of analogy problems, and each solution consisted of two phases: the construction phase and the comment phase. At the beginning, the participants went through a one-example practice session. In the construction phase (Figure 3.20) participants saw the rst three (A, B, [source domain] and C [target domain]) objects of an analogy and solved the problem by constructing the object D in order to complete the analogy. The input was made by drag and drop with a standard mouse. The object could be dragged to the trash can symbol if not desired. Once the participants were satised with their solution, they clicked on the OK button to proceed to the second phase (Figure 3.21), where participants were asked for optional comments on their constructed solutions. The participants were able to see what they had constructed while they were commenting. They were asked to give a short description of the steps of reasoning they had made in the process of constructing the solution using the keyboard. In order to facilitate and shorten the process of commenting, some keywords were 91

suggested above the comment box, such as: rotate, mirror, remove/change objects, shapes, colors, position etc. Again a button click was required, this time for proceeding to the next analogy. The stimuli were presented in a pseudo-randomized manner. The set of stimuli was divided into three combinations: each combo contained one out of the three original analogies, and two variations of each of the three original analogies. One of the three combos was randomly selected for each participant. Thus each participant had to solve seven analogies in total. As soon as a participant had nished the test, a message on the screen indicated the end of the test and gave the subject a feedback statement on their performance. The feedback statement was based on the comparison of the sum of their time needed to complete the individual analogies (excluding the time it took them to comment) with the average sum of the other participants times, and on the comparison of a participants choice of solutions with the average choice solutions for the particular analogies. The average choice solution was determined by the mean distribution of the answers based on the performance of the previous participants. Results For each participant the constructed solutions for the analogy tasks were stored in the experiment log le together with the comments entered and the time that was needed for the construction. The complete log le was then parsed and the data was clustered to identify groups of solutions. The dierent solutions were then analyzed and compared to nd preferred solutions. The comments were evaluated with the aim of identifying the transformations used by the participants in the process of creating the solutions, and furthermore to determine the strategies for solving geometric analogies. There were 31 to 37 participants per combo. The answers for one original analogy and for its respective variations of the source and target domain were compared. We will focus here on analogy A1Original as well as two of its variations and we give a complete overview of the results for the remaining analogies in Appendix C. The results for the original analogy A1Original are depicted in Figure 3.22: 93% of the participants constructed the solution shown with the letter (a) while the rest created other solutions, labeled (b) and (c) in the gure. 87% of the participants who constructed solution (a) provided a comment describing their opinion on how this solution was created. The analysis of the comments revealed that the participants applied dierent transformations to construct the same solution. The participants explained their solutions with the following transformations: 38% of the participants argued that they left 92

Figure 3.22: Original analogy A1Original the middle object intact, 27% stated that they removed the outer objects, another 27% removed the white circles, whereas 8% stated that the black square should stay. Figure 3.23 shows the results for analogy A1S3, one of the variations of the analogy A1Original. In this variation the source domain has been altered by moving the black circles in A from the center to the top while the rest of the analogy was kept unchanged. Here 44% of the participants constructed the solution (a), 38% solution (b) and the rest created sundry solutions. All participants who constructed solution (a) provided comments, from which the following transformations were identied: 80% removed the white objects, 13.3% kept the center object and 6.6% ipped the gures in the center. For solution (b) 84% of the participants provided a comment and described the following transformations: 55% kept the rst row objects, 15% rotated along the Z-axis in a 3D manner, 15% moved the upper row one position down. Figure 3.24 shows the results for A1T3, which is another variation of A1Original. Here the target domain has been altered by inserting additional elements. In this case, 50% of the participants constructed solution (a), 35% solution (b), 10% solution (c) and the rest created miscellaneous solutions. 76% of the participants who constructed solution (a) commented on it and from their descriptions the following transformations were identied: 85% kept the center object and 15% removed all types of circles. All participants who constructed solution (b) argued that they took the white gures away. 66% of those who created solution (c) provided comments and stated that the outermost circles had to be removed. The results for the remaining variations of the original analogy A1Original 93

Figure 3.23: Results for a variation of the source domain (A1S3)

Figure 3.24: Results for a variation of the target domain (A1T3)

94

are summarized in Figure 3.25. The analogy tasks are depicted together with the preferred solutions. For each solution, the number of participants who created that solution as well as the transformations that were described in the comments are given. The results for the analogies A2Original and A3Original and their variations are documented in Figures C.1 and C.2 in Appendix C. Discussion By varying the number or the features of the elements in the analogies we could distinguish which features of the elements were more salient in the process of solving. In A1Original the saliency of the central black objects is pronounced, with resulting removal of the remaining objects. Changing the color of the central object, as in A1S1 (in the source domain) or in A1T2 (in the target domain), still leads to a pronounced saliency of the centeredness, suggesting that position is a more important feature than color. However, the robustness of color increases when a third color is added, as in A1T1, A1T2 and A1T3. Here the ambiguous third color (grey) is perceived, by means of applying the Gestalt Law of Similarity, as belonging to the group of dark objects. This might be explained by the hypothesis that if too many features are present in an analogy, then humans apply Gestalt principles in order to group objects and thereby reduce the cognitive load. This can also be seen in the second analogy (A2Original and its variations), in which the solution of the analogy is reached more often by applying the Law of Proximity to group separate objects and then to rotate them as a whole. That can be seen in all the variations. Another observation is that rotation is preferred to reection. Although it was not possible to distinguish between these two transformations by analysis of the constructed solutions, the additional comments of the participants in our experiment provided more insight and suggested that the preferred transformation is rotation. The commenting feature of this experiment let us identify that the Law of Good Continuation inuenced the perceptions of the third analogy (A3Original and its variations). Conclusions We designed the geometric proportional analogies so that they have multiple plausible solutions. We had the participants construct solutions on their own. This is dierent from most other setups, where participants select from pre-dened solutions, as in standard intelligence tests. Our method 95

Figure 3.25: Results for A1Original and its variations

96

has many advantages. Letting the participants construct their own solutions does not constrain their reasoning processes to a limited set of answers, which is particularly important for ambiguous analogies such as the ones used in our experiment. Moreover, having the freedom of constructing solutions causes the participants to reect more cautiously on which transformations are necessary for them to complete the analogy, in contrast to selecting from pre-dened solutions, which might be accomplished by only considering supercial features and not the relational structure. Another important feature of our setting was that the participants had the possibility to provide comments. The participants comments were crucial in revealing how they actually experienced the analogy-making process. The analysis of the comments conrmed that there are several strategies for solving geometric proportional analogies, as discussed above. The comments revealed very helpful information that could not have been derived solely by analysis of the constructed solutions. The data of these comments showed that the participants often applied transformations to several objects at a time, which indicates that participants formed groups of objects. These groupings can be explained by Gestalt-based perception: most of the time, objects with common color or shape (Law of Similarity) or nearby objects (Law of Proximity) have been grouped together. The number of objects in an analogy and the number of colors and shapes determine the complexity of an analogy problem. The observations we made revealed that more complex analogies lead to a greater variety of created solutions. A possible explanation for this eect is the increased cognitive load: The strategies that dierent individuals employ for the purpose of coping with this load dier from each other. Varying an analogy by introducing new objects or new features of an object, in particular in the target domain, increases the cognitive load imposed on the participants in the process of solving. A good strategy to cope with an increased cognitive load is to apply Gestalt principles to group objects and thereby reduce the complexity of the problem. Thus, when dierent Gestalt principles are applied, then dierent solutions arise, as observed in our experiment.

3.4
3.4.1

Final experiment
Participants

238 participants completed the online experiment over the course of three weeks. Participants were recruited from among the experimenters friends, the pool of cognitive science students at the University of Osnabrck, and a u 97

number of online fora. Participants ranged in age from below 20 to over 40, with over 60% of all participants falling into the 21-30 age group. 85 female and 146 male participants took part in the experiment. 205 participants were right-handed, 16 left-handed, and 14 ambidextrous. Half of all participants had never taken an IQ test before. Most participants (62%) were German, followed by 24% Indonesians. 46% of participants were students, 32% were employees. Over half of all participants did not indicate their area of expertise this is probably due to the high percentage of cognitive science students participating in the study, for whom there was no item cognitive science as opposed tonatural sciences, computer science, and humanities. For detailed distributions of age, IQ history, nationality, expertise, and occupation, see Appendix D.1.

3.4.2

Procedure

The procedure was a modied version of that employed in the pretests, taking into account the results from all pretests. In particular, the second pretest had yielded a high number of participants who aborted the experiment before reaching the end. We believe this is due to the pretests long duration, which we attributed to two main factors: 1. The high number of analogies in the second pretest (42). This was solved by reducing the number of analogies to 30. 2. The necessity of choosing a radio button or (more than one) checkbox to comment on ones solutions. This considerably increased the time that participants had to invest in the solving of one analogy. This was solved by removing the radio buttons/checkboxes and making commenting optional (as in the rst pretest). Again, participants sat in front of a computer screen and completed the pretest online within the framework of the AI Analogy Lab 8 . They were instructed to solve geometric analogies of the form A:B::C:D. Their task would be to complete D. Instructions were modied so as to incorporate some suggestions of vocabulary that might be used in the optional commenting (e.g. rotate, mirror, remove, change objects / shapes / colours / positions etc.). These suggestions were repeated above the comment box after the construction of every analogy. Before the actual test began, participants completed an information form to provide information about their gender, nationality, handedness, occu8

see Chapter 3.2.1 for documentation

98

Solution order Analogies 123 S1OR S2A2 S3A3 S4C2 S5C4 132 S1A1 S2A3 S3C2 S4C3 S6OR 213 S1A2 S2C1 S3C3 S5OR S6A1 231 S1C1 S2C2 S4OR S5A1 S6A3 312 S1C2 S3OR S4A1 S5A3 S6C1 321 S2OR S3A1 S4A2 S5C3 S6C2 Table 3.1: Solution distribution in the choice condition pation, area of studies (if applicable) and when they had last taken an IQ test. The test consisted of three blocks of ten analogies each. The rst and third block were in the choice (C) condition and the second block was in the make (M) condition. C and M conditions were as in the rst pretest (see Section 3.3.1). The only dierence was that commenting in the M condition was explicitly made optional. We hoped that this would further decrease the number of premature experiment dropouts. A progress bar on the top of the screen displayed participants progress in the experiment.

3.4.3

Materials

Participants were presented with 30 analogies, randomized for presentation position (and, consequently, for condition). We ensured that analogies were equally distributed over the C and M conditions. Thus, each participant saw each analogy, albeit in a dierent order. Appendix D.2 contains the full set of stimuli. Analogy nomenclature is as follows: the rst number after the S indicates the analogy class. OR indicates that this analogy is this class original analogy, that which is being varied. Analogies containing variations of the AB domain have an A in their name, those with variations in the C domain a C. For example, analogy S3A2 belongs to analogy class 3 and contains a variation of S3ORs AB domain. The solution order for the C condition was pseudo-randomized over analogies in the following way. For each analogy, the three most frequent solutions according to the data from the second pretest were selected (when available). The solutions over the whole analogy set were then distributed over the six dierent ordering possibilites based on their frequency ranking. Distribution of solution order is shown in Table 3.1. For example, solution order 231 means that the solution in position 1 is the second most frequent, followed 99

::

Figure 3.26: S2Original pretest (top) vs. S2Original nal test (bottom) by the third most frequent, followed by the most frequent one. The number of analogies was reduced compared to the second pretest, which had consisted of 42 dierent analogies falling into six dierent classes. Thus, while an analogy class in the second pretest consisted of one original, three variations of the AB domain and three variations of the C domain (7 analogies per class in total), we reduced this to the original and two variations each in the nal test (5 analogies per class in total). Those variations with the least ambiguity in solutions according to the second pretest were excluded from the nal test. In addition, the C domain of the S2 class original analogy was modied (see Figure 3.26). This was due to the fact that in both pretests, as well as in the mini tests from the Technologietag and the Hochschulinformationstag, over 99% of all participants chose the same solution. This solution (single black square) could be reached both by retaining the black objects as well as by retaining the central object. To test which of these operations was more prominent, the objects colors in the C domain were switched. Thus, retention of black objects would lead to one solution (two black circles), while retention of the central object would lead to another (single white square). A dummy solution (single white circle) was created to t the schema of three solutions per analogy. Other analogies with dummy solutions are S3A3. A further change in stimuli concerns class S5 (see Figure 3.27). The downward pointing rectangle in S5C3 was intended to facilitate the good continuation (absolute downward movement) solution. For the nal test an additional analogy S5C4 was created that was identical to S5C3, with the exception that the black rectangle pointed upwards. We hoped to nd a 100

Figure 3.27: S5C3 (top) vs. S5C4 (bottom) preference for the proximity solution in comparison to S5C3. Appendix D.3 contains a classication of all solutions in terms of a) the transformations that, when applied to the gure in C, yield the solution and b) the Gestalt laws we assume to be in play when the application of the respective transformations is salient. This classication, which we constructed prior to conducting the experiment, reects our own judgment about the possible ways to get to the dierent solutions. Its validity was tested by comparing it with the results from participants optional commenting. Figure 3.28 shows the distribution of solutions according to applicable transformations.

3.4.4

Results

The number of datasets obtained ranged between 75 and 85 for each analogy in the M condition, and between 153 and 163 in the C condition. In what follows, plots of relative frequency for each solution in each analogy are shown, alongside the analogy itself and any additionally created solutions in the M condition. Solutions that were created by less than 5% of participants were not taken into account. The same color over C and M plots indicates the same solution. The solution numbers in the C plot are coupled to the solution position in the analogys solution set as presented to participants. For example, in analogy S1OR, the bars in the C condition solution plot are labelled as 2=n53, 3=n54, and 1=n52. 2,3, and 1 indicate the solutions position as presented in the analogy, which is displayed above the plot. Due to the fact that there are too many factors that potentially inuence the perception of analogies and the investigation of what these factors are

101

Overview of transformations:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 colour saliency / object movement deletion of grey objects deletion of non-black objects deletion of non-central objects deletion of non-central objects / colour switch deletion of non-top group / object movement deletion of white objects object movement reflection reflection / switch colour rotation rotation partially rotation / switch colour switch colour switch position

Figure 3.28: Distribution of solutions according to applied transformations would have well exceeded the scope of the CoUGAR project, we we were not able to systematically vary the analogies so as to make them comparable across analogy classes. This also had the eect of making statistical analysis of solution preferences impossible. Therefore, if we speak of e.g. the data reecting a preference for rotation over reection, we are speaking not of an actual statistical eect, but rather of a tendency. Analogy results are listed and discussed individually and are only compared to other analogies in cases of minimal feature variation. At the end of each analogy section the transformations and Gestalt laws associated with solutions 1 to 3 are listed. S1 S1OR

102

S1ORC

S1ORM

0.8

0.8

0.6

0.6

n56

0.4

0.4

0.2

0.2

n55

0.0 2=n53 3=n54 1=n52

0.0 n53 n52 n54 n56 n55 4_1 1_0 1_1 4_3 4_5 4_6

Ambiguity in this analogy stems from the possibility of perceiving the analogy in one or many of the following ways: 1. The whole gure in A is perceived as being rotated. 2. All objects are perceived as switching their color. 3. The black circle is perceived as moving upwards. 4. The objects are perceived as being reected over a diagonal axis going from the upper left to the lower right. 5. The gure is perceived as consisting of an upper and a lower group, which exchange positions. Data from the C condition indicates that the rotation solution is preferred (i.e. all objects in C are grouped together according to the Law of Proximity), followed by solutions 3 and 1. Thus, in this analogy rotation is preferred over color switching (according to the Law of Similarity) and object movement (according to the Law of Good Continuation - GC). The M condition reects the preferences revealed in the C condition and additionally yielded solutions corresponding to 4.) and 5.). Solutions: transformations and Gestalts 1. Object Movement GC 2. Rotation Proximity 3. Color switch Similarity 103

S1A1

S1A1C

S1A1M

0.8

0.8

0.6

0.6

0.4

0.4

4_26

0.2

0.2

0.0 2=n35 1=n33 3=n34

0.0 n35 n33 n34 4_26 0_0 4_15 4_21 4_25 4_6 4_7 4_9 4_12 4_13 4_17 4_18 4_2 4_20 4_23 4_24 4_27 4_3 4_5 4_8

In this variation we varied the AB domain by introducing an additional color, with the aim of testing whether this would lower the preference for (binary) color switching. This was not the case. Instead, among the two most preferred solutions in the C condition is solution 2, which is created by switching the color of all black objects to grey and of all white objects to black. Indeed, this was the preferred solution in the M condition, and was created twice as often as the next preferred solution. Thus, instead of decreasing the preference for color switching, this variation in fact increased it. The M condition also yielded a novel solution, created by a combination of object movement and global color switching. This solution stands in contrast to solution 1, where the two circles unaected by the black circles movement retain their white color. In the novel solution not only does the aected white circle change its color to grey, but rather all white circles do. Solutions: transformations and Gestalts 1. Object movement GC 2. Color switch Similarity 3. Rotation Proximity

104

S1A2

S1A2C

S1A2M

0.8

0.8

0.6

0.6

n38

0.4

0.4

0.2

0.2

n40

0.0 3=n37 2=n36 1=n39

0.0 4_12 4_13 n36 n37 n38 n40 n39 0_0 1_0

In this variation we varied the AB domain by changing the shape of the objects. Instead of circles, the analogy contained rectangles. The purpose of this was to invoke the image of a line consisting of two shapes, along which the shapes could move. Based on this, we predicted a higher preference for the movement solution. This was the case in the M, but not in the C condition. Here there was still a strong preference for the rotation solution. The relative frequency of movement solution preference in the C and M conditions are overall not so dierent (32% vs. 38%). It is the dierence between the rotation solution preference which is striking (57% in the C condition vs. 25% in the M condition). This dierence might be due to the fact that 19% of participants in the M condition constructed a color switch solution (n38) which was not available as a choice in the C condition. Thus, rotation may have been the second best solution for a large part of participants in the C condition. The overall comparison of frequency of movement solution construction between S1A2 (average of C and M conditions: 34%) and S1OR (average of C and M conditions: 25%) provides support for the prediction that variation of the AB domain such as to facilitate perception of object movement along a line increases preference for the movement solution. Solutions: transformations and Gestalts

105

1. Position switch Similarity 2. Object movement GC 3. Rotation Proximity S1C1

0.8

S1C1C

S1C1M

0.8

0.6

0.6

S1C1C S1C1C
0.6 0.4 0.8 0.5 0.8 0.5 0.4 0.6

n44 S1C1M S1C1M

n41

0.2 0.6

0.4

0.2 0.6

0.4

n44

n41

n43

0.3 0.0 3=n46 1=n46 2=n42 0.4 0.4 0.0

0.3 4_10 4_11 4_20 4_11 4_22 4_20 4_22 n44 n46 n41 n43 n42 n45

0.2

0.2

0.2 0.1

0.2 0.1

n43

0.0 3=n44 1=n46 2=n42 0.0 3=n46 1=n46 0.0 2=n42 n44 n46 n41

0.0 n42 n44 n45 n46 4_10 n41 4_11 n43 4_20 n42 4_22 n45 n43 4_10

n=162

Analogous to S1A1, the C domain was varied by introducing a new color. The goal, as in S1A1, was to test whether (in the M condition) this would lead to a lower preference for color switching. This seems to be the case solution n43, corresponding to the color switch solution (black turns white, white turns black) was created in only 14% of the cases, as opposed to 22% in this class original analogy. Again, there was a strong preference for the rotation solution in the C condition (62%) which decreased substantially in the M condition (36%). As in S1A2, we attribute this decrease to the solution construction freedom in the M condition, which, if absent, seems to trigger the second best solution choice. The solution construction freedom in the C condition leads to the construction of two new solutions: the color switch solution, mentioned above, and the solution corresponding to 5.) as described in section 3.4.4. Here, there is a similarity-based grouping of the C domain into an upper and lower group, which switch their position to yield solution n41. Solutions: transformations and Gestalts 106

1. Color switch Similarity 2. Reection Similarity 3. Rotation Proximity S1C2

S1C2C

S1C2M

0.8

0.8

0.6

0.6

n49
0.4 0.4

0.2

0.2

0.0 2=n51 3=n50 1=n48

0.0 4_10 4_13 4_14 n48 n51 n50 n49 n47 4_0 4_1

Analogous to S1A2, the circles in the C domain were replaced by rectangles, analogous to S1A2. A higher preference for the movement solution was predicted. Compared to S1OR (25% movement solution), this prediction was not conrmed the movement solution on average in the C and M conditions was chosen in 25% of all cases. However, there is a dierence of 10% in movement solution preference between the C and M conditions. Why this is the case is not entirely clear. Rotation solution preferences display the same pattern as above: high preference in the C condition (52%), with a decrease in the M condition (31%). Participants in the M condition additionally constructed the solution corresponding to the switch of upper and lower group of circles. Preference for the color switch solution was comparable to the original S1 analogy. Solutions: transformations and Gestalts 1. Object movement GC 107

2. Rotation Proximity 3. Color switch Similarity Summary of S1 series In this series we attempted two things: to decrease the preference for color switching by introducing a new color, and to increase the preference for object movement by employing rectangles instead of circles to create the image of alinealong which objects can move. While the former did not seem to work out as planned, the data indicates that the latter did. S2 S2OR

S2ORC
0.8 0.8

S2ORM

0.6

0.6

0.4

0.4

0.2

0.2

0.0 3=1_0 2=n64 1

0.0 1_0 n64 2_1 2_2 3_0

The variations in this series are of the old, not the new S2OR9 . All variations were aimed at teasing apart the application of the following four transformations. 1. Non-black objects are deleted. 2. Non-central objects are deleted.
9

see Figure 3.26 for the dierence; see section 3.4.3 for the explanation of this dierence.

108

3. White objects are deleted. 4. Outer objects are deleted. The results of S2OR show a clear preference for the retention of the central object over the retention of the black objects10 . Solutions: transformations and Gestalts 1. DUMMY 2. Object deletion (non-black) Similarity 3. Object deletion (non-central) Similarity S2A2

S2A2C
0.8 0.8

S2A2M

0.6

0.6

0.4

0.4

0.2

0.2

0.0 1=1_1 2=3_0 3

0.0 1_1 3_0 0_0 1_0 1_2 2_0 5_0 5_1 5_2

An additional layer of grey circles was added to the top and bottom of the gure in A, with the purpose of forcing participants to decide for a solution corresponding to the deletion of either grey or non-central objects
Retention of black objects is used synonymously with deletion of non-black objects. Analogous with retention of central objects and deletion of non-central objects. While participants comments never contained the negative formulation (i.e. deletion of noncentral/non-black objects), we use this expression nevertheless for the sake of uniformity.
10

109

(solutions 2/3 and 1 respectively). As in S2OR, centrality seems to be the more prominent feature - there was a clear preference for the solution corresponding to the retention of the central object as opposed to the deletion of the grey objects. Solutions: transformations and Gestalts 1. Object deletion (non-central) Proximity 2. Object deletion (grey) Similarity 3. Object deletion (grey) Similarity S2A3

S2A3C
0.8 0.8

S2A3M

0.6

0.6

n58

0.4

0.4

0.2

0.2

0.0 1=n59 3=n61 2=n60

0.0 n59 n61 n60 n58 0_0 n62 3_0

This variation was aimed at teasing apart application of non-black object deletion and combined deletion of outer objects with color switch. In addition, there was a solution corresponding to the downward movement of the topmost object. This time, there was a clear preference for retaining the black object, followed by the solution corresponding to downward movement of the topmost object. Retaining the central object, which was the preferred transformation in the previous analogies, was applied in the C and M conditions in only 19% of the cases (as opposed to 79% in S2OR). This is a drastic dierence. 110

However, solution 3 not only requires retaining the central object, but also in addition switching that objects color. It seems reasonable to conclude that the application of more than one operation is more complex and thus less preferred. Therefore, although centrality seems to be an important aspect in geometric analogy perception, the number of transformations that have to be applied to reach a certain solution might be claimed to be the more constraining factor in solution construction. A novel solution was also created by 9% of participants: this solution is arrived at by the application of two transformations, retention of black objects followed by downward movement of retained objects. Solutions: transformations and Gestalts 1. Object deletion (non-black) Similarity 2. Object deletion (non-central) and color switch Proximity 3. Object deletion (non-top group)/movement GC S2C1

S2C1C
0.8 0.8

S2C1M

0.6

0.6

3_1

0.4

0.4

0.2

0.2

3_4

0.0 1=4_1 3=n57 2=1_0

0.0 1_0 3_1 n57 3_4 4_1 2_1 3_5 3_6 2_0 3_3 4_0 7_0

This variation of the C domain is designed to yield dierent results for the application of all four of the transformations mentioned in section 3.4.4.

111

Thus, deletion of non-central objects yields solution 2, deletion of white objects solution 1, deletion of non-black objects solution 3. The restriction to three solutions in the C condition did not allow presentation of the fourth solution (that results from application of outer object deletion). This solution was created, however, in the M condition (solution 3 1). Interestingly, the results of this analogy in the C and M conditions do not reect the same preferences. The data from the M condition shows a clear preference for deletion of non-central objects (solution 2, constructed 49%), followed by deletion of outer objects (solution 3 1, constructed 19%). In the C condition, on the other hand, solution 2 was very dispreferred. In this condition, white object deletion was most preferred, followed by non-black object deletion. Why is this the case? First of all, participants might simply be lazy. Solution 2 is the one that contains least objects and is consequently fastest to construct. Thus, when confronted with a number of intuitively sensible choices, they may have simply chosen the easiest solution in the M condition. In the C condition, on the other hand, the eort for choosing a solution is equal for each solution. This is a factor that was not controlled for - here we may be seeing the eects. A second reason for this distribution of solution preferences might be the nature of the C domain, which initially looks quite complex. It contains three dierent colors and two dierent shapes. Of the ve objects it contains, only two are equal. Thus, it might be that participants were overwhelmed by the analogys complexity and simply chose the rst option presented to them, which happened to be that corresponding to the deletion of white objects. We are potentially seeing here the eects of not randomizing solution position and of not controlling for analogy complexity in terms of number and kind of shapes. However, if we disregard the laziness hypothesis as an explanation of the data in the M condition, but take it as is, it falls in line with the previous data exhibiting a clear preference for retaining the central object. Solutions: transformations and Gestalts 1. Object deletion (white) Similarity 2. Object deletion (non-central) Proximity 3. Object deletion (non-black) Similarity

112

S2C2

S2C2C
0.8 0.8

S2C2M

0.6

0.6

0.4

0.4

0.2

0.2

0.0 3=n63 1=2_0 2=1_0

0.0 n63 1_0 2_0 2_4 1_2 2_6 0_0 2_3 2_5

In this variation of the C domain, the position of the black object was changed, and a new color was introduced. Thus, the application of white object deletion, non-black object deletion, and non-central object deletion leads to dierent results. The obtained results t in well with the other data from the S2 series: solution 3, corresponding to the retention of the central object, is highly preferred. Solutions: transformations and Gestalts 1. Object deletion (white) Similarity 2. Object deletion (non-black) Similarity 3. Object deletion (non-central) Proximity Summary of S2 series This series was aimed at determining whether object position (centrality) or color (blackness) were more salient, which we expected to see directly mirrored in the deletion/retention of certain objects. Results generally indicated that object centrality is very prominent, thus leading to central objects retention. The only factor that seems to be stronger than centrality is analogy complexity in terms of the amount of transformations that have to be applied to reach a certain solution. Where

113

reaching a solution required not only retaining the central object (the generally preferred strategy), but additionally performing a color switch, this solution was very dispreferred. S3 S3OR

S3ORC
0.6 0.6

S3ORM

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0.0 2=n5 3=2_6 1=2_5

0.0 n5 2_6 2_5 0_0 2_1 1_0 2_2 2_3

n=153

This analogy is ambiguous in that one can perceive the relation between A and B to be one of the following: 1. Rotate the triangle. 2. Rotate the whole gure. 3. Reect the gure horizontally. The results indicate a preference for global rotation, i.e. for perceiving the gure as a whole, presumably proximity-based, and rotating this whole. Second most preferred is the reection solution, followed by the dispreferred partial rotation solution. Results are similar in the C and M conditions. Solutions: transformations and Gestalts 1. Rotation Similarity 2. Rotation Proximity 3. Reection Symmetry 114

S3A1

S3A1C

S3A1M

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0.0 2=2_2 3=2_7 1=2_6

0.0 2_2 2_7 n1 0_0 2_4 2_6 1_0 2_0 2_1 2_5

In order to inhibit perception of the gure as a whole, and rather facilitate perception of individual objects, we introduced a new color in AB. This led to a strong decrease in the global rotation solution (here solution 1), while preference for partial rotation increased (solution 2). Solutions: transformations and Gestalts 1. Rotation Proximity 2. Rotation Similarity 3. Reection Symmetry

115

S3A3

S3A3C

S3A3M

0.8

0.8

n3

0.6

0.6

0.4

0.4

0.2

0.2

0.0 1=n2 2=2_0 3

0.0 n2 n3 0_0 2_0

In order to further enhance perception of the individual objects, we contrasted white and black instead of white and grey objects, in AB. In addition, objects in A were aligned such that applying global rotation would not lead to the constellation in B. This indeed led to a clear preference for individually rotating the objects in C, yielding solution 1. Solutions: transformations and Gestalts 1. Rotation Similarity 2. Rotation Proximity 3. DUMMY

116

S3C2

S3C2C

S3C2M

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0.0 3=n4 1=3_6 2=3_7

0.0 3_6 n4 3_7 3_1 0_0 3_0 3_3 3_4 3_5 5_0

To further test whether the gure in C was being perceived as a whole or as individual objects, we introduced an additional white triangle, which would lead to a dierent solution if rotated individually (solution 2) instead of as part of the whole gure (solution 1). Both the C and M conditions supported the preferences obtained in S3OR. Global rotation (53% - S3OR: 61%) is preferred over reection (33% - S3OR: 29%) and partial rotation (11% - S3OR: 7%). Solutions: transformations and Gestalts 1. Rotation Proximity 2. Rotation Similarity 3. Reection Symmetry

117

S3C3

S3C3C S3C3C

S3C3M S3C3M

0.40.8

0.40.8

2_1
0.30.6 0.30.6

0.20.4

0.20.4

2_4

0.10.2

0.10.2

0.00.0 2=2_5 2=2_5 1=2_2 1=2_2 3=2_3 3=2_3

0.00.0 2_5 2_5 2_2 2_2 2_3 2_3 2_1 2_1 2_4 2_4 2_6 2_6 0_0 0_0 2_0 2_0 2_7 2_7

n=158

For the purpose of blocking perception of the gure as a whole and instead facilitate perception of the individual objects, we introduced a new color in C (analogous to S3A1). In addition, the shape of the square was changed to a triangle - this was done to see whether the white shape would be rotated as well (which is obviously not visible when using squares). Solution 1 corresponds to the global rotation solution, solution 2 to that of partial rotation (rotation of triangles initially standing upright), and solution 3 to the individual rotation of all triangles. The same ranking of solution preferences was obtained in both C and M conditions. There was a preference for solution 2, indicating that object orientation is a salient feature in perception. This was even stronger than the preference for global rotation, which is surprising given the ndings above. In addition a solution was created which was arrived at by rotating only the grey triangle (solution 2 4). However, this occurred much less frequently than the solution corresponding to the rotation of either all or only upward pointing triangles. Thus, one might conclude that object orientation is a more salient feature than object color when grouping according to the Law of Similarity. However, solution 2 may also be construed as a reection solution, arrived at by reecting the gure in C along a horizontal axis. The just mentioned conclusion then is not valid. Solutions: transformations and Gestalts 1. Rotation Proximity 118

2. Reection Symmetry 3. Rotation Similarity Summary of S3 series This series shows that participants prefer grouping of all objects and performing global rotation on the resulting group when possible. A similar observation was already made for the S1 series, where rotation of the whole gure was a highly preferred strategy. This is quite intriguing, since grouping is generally considered to be a complex, costly process. However, these results suggest that the opposite may be the case: global grouping may be the default, with splitting being a costly process only to be applied if certain features make individual objects particularly salient (as in e.g. S3C3). S4 S4OR

S4ORC
0.8 0.8

S4ORM

0.6

0.6

0.4

0.4

0.2

0.2

0.0 3=2_0 1=2_1 2

0.0 2_0 2_1 2_2 n12 0_0

This series was aimed at further testing the preference of rotation vs. reection, including also the switching of object position. The dierent applicable operations in this analogy are: 1. The gure is reected along a central vertical axis. 2. The whole gure is rotated by 90 degrees to achieve a laying position. 119

3. The objects switch position / The whole gure is rotated by 180 degrees. Results show a preference for solution 3 this may be arrived at both by global rotation or object position switch. While the reection solution was also constructed in a quarter of the cases, the 90 degrees rotation was highly dispreferred. Solutions: transformations and Gestalts 1. Reection Symmetry 2. Rotation Proximity 3. Rotation Proximity S4A1

S4A1C
0.8 0.8

S4A1M

0.6

0.6

2_6

0.4

0.4

0.2

0.2

0.0 2=n6 3=2_1 1=2_5

0.0 n6 2_1 2_5 2_6 0_0 1_0 2_0 2_3 2_4 3_1

The colors in the B domain were switched. This had the eect that solution 1, corresponding to global rotation was only a partial solution. The only correct solutions left over were solution 2 and 3. The former was reached by the combination of global rotation with additional object color switch, or by object position switch with additional color switch. The latter was reached by vertical reection with additional object color switch. Also, an additional partial solution was created (solution 2 6), which was reached by simply reecting over the vertical midline axis. Although participants chose and constructed the correct solutions 60% of the time (again with a preference for global rotation/object position switch), 120

there was a surprisingly high amount of partial solutions in both the C and M conditions (19%). This might be attributed the the analogys complexity, arising from the high number of dierent shape/color combinations present in the AB domain - no two objects are the same. This supports the ndings from analogy S2C1. Solutions: transformations and Gestalts 1. Rotation Proximity 2. Rotation Proximity 3. Reection Symmetry S4A2

S4A2C
0.8 0.8

S4A2M

0.6

0.6

0.4

0.4

0.2

0.2

0.0 3=2_1 2=2_3 1=2_5

0.0 2_1 2_3 2_5 0_0 2_0 2_2 2_4

In this variation, an additional white circle was added in the AB domain, with the purpose of facilitating a 90 degrees rotation solution. This indeed was the case. 90 degrees rotation and reection were equally preferred. Solutions: transformations and Gestalts 1. Rotation Proximity 2. Reection Symmetry 3. Rotation Proximity

121

S4C2

S4C2C
0.8 0.8

S4C2M

0.6

0.6

n8

0.4

0.4

0.2

0.2

0.0 1=n7 2=2_5 3

0.0 n7 2_5 n8 0_0 1_0 2_1 2_3 2_4 4_0

In this variation, the circle in C was replaced by a triangle. This has the consequence that global rotation and object position switch lead to dierent results, which was not the case in the previous S4 analogies. Results in both C and M conditions indicate a preference for global rotation. Reection was also a preferred transformation (solution 2). The solution corresponding to object position switch (n 8) was constructed in 9% of the cases. Thus it is safe to say that rotation is more preferred than position switch. This again supports the hypothesis that (assuming low complexity is preferred over high complexity) grouping is in fact cognitively less complex than splitting. Solutions: transformations and Gestalts 1. Rotation Proximity 2. Reection Symmetry 3. Rotation Proximity

122

S4C3

S4C3C
0.8 0.8

S4C3M

0.6

0.6

0.4

0.4

0.2

0.2

0.0 1=n9 3=n11 2=n10

0.0 n9 n11 n10 0_0 1_0 2_1 2_2 2_3 2_5 2_6

Similar to S4C2, in this variation the shapes in C are not directly adjacent. The intention behind this was to test whether this would facilitate perception of the individual objects as opposed to a whole gure. If so, solution 2 should be preferred. The results indicate nothing of the sort, but instead were congruent with the results obtained from S4C1. This may show that the Law of Proximity does not apply stronger to directly adjacent objects than to objects that stand slightly apart. Solutions: transformations and Gestalts 1. Rotation Proximity 2. Position switch Proximity 3. Reection Symmetry Summary of S4 series This series further investigated rotation preferences. Further support for the hypothesis that global rotation is a generally preferred solution strategy was provided. In addition, these data show that rotation is preferred over object position switch. However, a potential problem may be that the analogies in general are not controlled for the transformations that lead to their solutions. The eect this has on our analogy set

123

is that there are 21 dierent solutions that can be arrived at by the application of global rotation, but only three that are reached by switching objects positions. This may have biased participants to construct rotation solutions when possible, i.e. it might be that once they began employing the rotation strategy, they became blind to the alternatives. S5 S5OR

S5ORC
0.7 0.7

S5ORM

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0.0 1=n23 2=n22 3

0.0 n23 n22 0_0 1_0 2_1

Ambiguity in this analogy comes about by potentially perceiving the analogy in the following ways: 1. The upper object moves towards the lower object. 2. The circle moves towards the square. 3. The circle moves downwards. While there seems to be a slight preference for the upper object moves towards lower object solution, the absolute downward circle movement solution was highly dispreferred, indeed was not created at all in the M condition. Since pretests had already indicated that this solution was not among the 124

preferred, variations of the analogy aimed at facilitating absolute downward movement. Solutions: transformations and Gestalts 1. Object movement GC 2. Object movement GC 3. Object movement GC S5A1

S5A1C
0.7 0.7

S5A1M

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0.0 1=n13 3=n14 2

0.0 n14 n13 6_5 0_0 2_0 6_2 6_7

By varying B such that a line of circles is formed, we hoped to facilitate perception of absolute circle downward movement. Thus, solution 2 should have been chosen more often than the corresponding solution 3 in the original. This was not the case, however (3% chosen in S5OR, 4% in S5A1). Again, it was not created in the M condition. The preference for solution 3 was even higher, though. In addition, there is an incongruence between solution preferences in the C and M conditions - while solution 1 is preferred over solution 2 in the C condition, this preference is reversed in the M condition. It is not clear why this is the case. One hypothesis is that it might be an eect of solution 125

position in the C condition: instead of reasoning about the solution before choosing, it may be that participants see the rst solution and try to make it t the analogy, thus leading to a higher preference for the rst solution. Solutions: transformations and Gestalts 1. Object movement GC 2. Object movement GC 3. Object movement GC S5A3

S5A3C
0.7 0.7

S5A3M

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0.0 2=n15 1=n16 3

0.0 n15 n16 6_6 0_0 1_0 2_0 2_1 2_2 6_2 6_7

To further enhance perception of absolute downward movement, the circles in AB were replaced by rectangles, analogous to S1A2 and S1C2. This again had no eect in the M condition, though there is a slight increase in preference for the absolute downward solution in the C condition (6%). However, the overall preference for solution 2, the upper towards lower object solution, became even clearer. Solutions: transformations and Gestalts 1. Object movement GC 126

2. Object movement GC 3. Object movement GC S5C3

S5C3C
0.7 0.7

S5C3M

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0.0 1=n19 2=n17 3=n18

0.0 n19 n17 n18 2_0 2_8

Analogous to S5A3, the circle in C was replaced by a downward slanted rectangle to facilitate absolute downward movement perception. Although this led to the construction of the absolute downward solution twice in the M condition, it was chosen even fewer times than in the original. Preference for the upper towards lower object movement solution was found once again. Solutions: transformations and Gestalts 1. Object movement GC 2. Object movement GC 3. Object movement GC

127

S5C4

S5C4C
0.7 0.7

S5C4M

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0.0 3=n21 1=n20 2=2_4

0.0 n21 n20 0_0 1_0 2_4

In this variation, the rectangle in the C domain was slanted upwards from left to right instead of downwards, as was the case in the previous analogy. We thus expected a higher preference for solution 1 (rectangle moves towards square) than for the corresponding solution 2 in S5C3, as well as a lower preference for the absolute downward movement solution 2 than the corresponding solution 3 in S5C3. This was not the case. In addition, the preference for the upper towards lower object solution seems robust. Solutions: transformations and Gestalts 1. Object movement GC 2. Object movement GC 3. Object movement GC Summary of S5 series This series aimed at varying the perception of lines created by slanted rectangles and lined up circles. We hypothesized that the more salient the line perception, the more often the absolute downward movement would be constructed, in line with the Law of Good Continuation. 128

However, this was not the case. One reason may be that the spatial region that needs to be used for the absolute downward movement solution lies outside the region used by the A, B, and C domain. Thus, the absolute downward movement requires outside-the-box thinking. In general, object position seems to be a more salient feature than object shape, i.e. preference for solutions corresponding to the movement of objects based on their position (upper towards lower) was higher than for solutions corresponding to the movement of objects based on their shape (circle moves towards square). S6 S6OR

S6ORC

S6ORM

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0.0 1=n31 2=n32 3=4_3

0.0 n31 n32 0_0 1_0 2_0 4_1 4_3

Similar to the S4 series, this series aimed at investigating the preference of global rotation over individual object movement or feature switch. The results of the original analogy S6OR show a strong preference for global rotation (solution 1), with the solution corresponding to the grouping of the four objects into two groups (upper right and lower left), with objects in the lower left undergoing a color switch (solution 2) being much less preferred. However, the latter was constructed much more often in the M condition (17%) than in the C condition (6%). This might again be due to a solution position eect in the C condition. Solutions: transformations and Gestalts 129

1. Rotation Proximity 2. Color switch Similarity 3. Object movement GC S6A1

S6A1C

S6A1M

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0.0 3 2=n25 1=4_3

0.0 n25 4_3 0_0 1_0 3_0 4_1 4_2

To test whether rotation preference could be further increased, we added a fth circle to the middle of the gure in the A and B domain, with all circles now being directly adjacent. We intended this to increase perception of the gure as a whole, this in turn increasing preference for global grouping and thus global rotation. While there seems to be no dierence in the C condition as compared to the original, the rotation solution was constructed in 89% of cases, as opposed to 77% in the original. The color switch solution (solution 1) was much dispreferred in the M condition (4%) as opposed to the original. Thus, while our hypothesis was supported in the M condition, it was not in the C condition (which might also be due to the solution position eect hypothesis put forward previously). Solutions: transformations and Gestalts 1. Color switch Similarity 2. Rotation Proximity 3. Object movement GC 130

S6A3

S6A3C

S6A3M

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0.0 2 3=n26 1=n27

0.0 n26 n27 4_9 0_0 1_0 4_0 4_1 4_3 4_8

To increase perception of two groups as described in the original, we introduced a new color in the AB domain, to facilitate visual discrimination of these two groups. We predicted a higher preference for the color switch solution. This indeed seems to be the case. Solution 1, which is arrived at by switching the color of all black and white objects, was highly preferred in both conditions. Solutions: transformations and Gestalts 1. Color switch Similarity 2. Reection Symmetry 3. Color switch Similarity

131

S6C1

S6C1C

S6C1M

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0.0 1 2=n28 3=4_5

0.0 n28 4_5 0_0 4_0 4_2 4_3 4_4

To further increase perception of two groups instead of one in the C domain, we varied the objects shape the upper group consisted of circles, the lower group of squares. We predicted a higher preference for the color switch solution over the global rotation solution. While results indicate that global rotation is still highly preferred, it is less so than in the original. Conversely, there seems to be a slight increase in color switch preference. Thus, the prediction may be said to have played out, albeit not strongly. Solutions: transformations and Gestalts 1. Object movement GC 2. Rotation Proximity 3. Color switch Similarity

132

S6C2

S6C2C

S6C2M

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0.0 3=n29 2=4_6 1=n30

0.0 4_10 4_11 n29 4_6 n30 3_0 4_0 4_3 4_5 4_7 5_0

Complementary to S6C2, we varied the C domain such that the lower left group contained objects of the same color (white) than of the same shape. We predicted this to result in an increased preference for object position switch. This does not seem to be the case. Global rotation is still highly preferred. Solutions: transformations and Gestalts 1. Color switch Similarity 2. Position switch Proximity 3. Rotation Proximity Summary of S6 series The goal of this series was to investigate the robustness of global rotation as solution strategy. There seems to be a slight inuence of shape on grouping preferences, as demonstrated in S6C1, while color grouping in this case does not seem to be very salient.

3.4.5

Comment classication

The detailed comparison of comment classication and solution classication is listed in Appendix D.4.

133

Chapter 4 Closing discussion

Clemens Bauer, Judith Degen, Irena Dorceva, Maxim Haddad, Martin Schmidt, Rolf Stollinski, Kae Sugwara, Martesa Tandra & Radomir Zugic Institute of Cognitive Science, University of Osnabrck u

In this report presents our research on human strategies to solve geometric proportional analoges and documents the ndings from experimental investigation of human analogy perception, inspired by Gestalt psychological considerations. We conducted a number of experiments two pretests, two mini-experiments at TdT and HIT, and a nal experiment to investigate solution preferences participants have and solution strategies participants employ in geometric analogy-solving. Pretest results showed that, while we can extract preference tendencies, there are a) many diverse solution strategies leading to one and the same solution, and b) many dierent solutions to one and the same analogy that can be explained in terms of the same Gestalt law. This made analogy stimulus construction and variation a very complex problem, indeed resulting in an uncontrolled distribution of transformation (solution strategy) possibilities over the set of analogies in the nal experiment. Thus, our main result, that participants tend to group globally and apply rotation when possible, is to be taken with a grain of salt. That rotation was the preferred solution strategy may simply be due to the high number of possible rotation solutions compared to solutions that may have been reached by application of other transformations. In short, there may have been strong priming eects towards rotation. This is one point one could further improve in future experiments. It requires elaborate analysis of possible object group134

ings as well as of kind and amount of transformations that can be applied to a given scene in order to control for available solution strategies over analogies. A related point is that it is not clear what eect a variation in color, shape, size, etc. really has on scene perception. Since we did not vary only color, only shape, or only number of objects in any given analogy, we could not systematically investigate the contribution of each of these features. This would require a large-scale investigation of each feature in isolation. In addition, there may have been eects of solution position in the Choice condition. That is, in some cases the rst solution (read from left to right) was preferred, even though it was not constructed (or constructed less frequently) in the Make condition. To factor this out, solution position should be randomized in future experiments. Also, a dummy solution should be included in the solution set, to control for participants who are not properly engaging in the task. Until the end, the problem persisted of how to collect useful comments from participants about their solution strategies. This is obviously an important part of the experiment, since often the only way to determine which transformations participants applied to reach a given solution is by looking at their comments. We tried optional commenting with and without keywords, choosing one of six pre-formulated comments, and commenting by checking one or more keywords (see Section 3.3.2 for a more precise description of the latter two methods). In the nal experiment, we decided to use free optional commenting with keywords as support. However, this was not optimal, as many participants chose not to comment. This is a dicult problem, which seems to be based partly on the diculty of accessing ones own perceptual processes. A further big project is to develop a complexity measure for geometric analogies. For the project itself it would have been interesting to investigate whether correlations between analogy complexity and either number of created solutions or number of wrong/partial solutions could be found. An attempt at developing such a measure based on supercial analogy features was made in the course of the project. The features initially taken into account were a) number of objects in the scene, b) number of dierent colors and shapes involved, and c) number of permutations of shapes and colors in the objects. These features were deemed insucient to develop an adequate complexity measure it is necessary to take into account e.g. relative distance of objects to one another, the number of possible object groupings, whether the number of objects in A and C is the same (which facilitates object mapping), how many dierent transformations may lead from A to B, to name just a few. This is certainly a very interesting task and should be further investigated. 135

Bibliography
W. Bechtel and G. Graham, editors. A Companion to Cognitive Science. Wiley-Blackwell, 1999. P.W. Cheng and K.J. Holyoak. Pragmatic reasoning schemas. Cognitive Psychology, 17(4):391416, 1985. M. Dastani and R. Scha. Language for gestalts of line patterns. Journal of Mathematical Psychology, 47:429449, 2003. T. G. Evans. A heuristic program to solve geometric analogy problems. Technical Report Technical Report: AIM-46, Massachusetts Institute of Technology, 1962. Gentner. Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7(2):155170, 1983. D. Gentner, M.J. Rattermann, and K.D. Forbus. The roles of similarity in transfer: Separatincl-retrievability from inferential soundness. COGNITIVE PSYCHOLOGY, 25(52):524575, 1993. Holyoak K.J. & Kokinov B.N. Gentner, D. The Analogical Mind: Perspectives from Cognitive Science. MIT Press, 2001. M.L. Gick and K.J. Holyoak. Schema induction and analogical transfer. Elsevier, 1983. W. Gordon. Synectics: The development of creative capacity. Harper and Row, New York, 1961. D. Hofstadter and C.D. Dennett. The Minds I: Fantasies and Reections 1 on Self and Soul 0A 4 ed. Basic Books, 1981. K. Holyoak and P. Thagard. Analogical mapping by constraint satisfaction. Cognitive Science, 13:295355, 1989.

136

Indurkhya. Modes of analogy. En Analogical and Inductive Inference, pages 217230, 1989. W. Khler. Gestalt Psychology. Liveright, New York, 1929. a M.T. Keane. On adaptation in analogy: Tests of pragmatic importance and adaptability in analogical problem solving. The Quarterly Journal of Experimental Psychology Section A, 49(4):10621085, 1996. K. Koka. Principles of gestalt psychology. Psychological Bulletin, 19:531 585, 1935a. K. Koka. Principles of Gestalt Psychology. Harcourt, New York, 1935b. B. Kokinov and French. Computational models of analogy-making. En L. Nadel, ed. Encyclopedia of Cognitive Science. London, pages 113118, 2003. A.B. Markman. Constraints on analogical inference. Cognitive Science, 21 (4):373418, 1997. W. Metzeger. Certain implications in the concept of gestalt. American Journal of Psychology, 40:162166, 1928. E. Mullally and ODonoghue. Geometric proportional analogies in topographic maps: Theory and application. En Cambridge, UK, pages 95108, 2005. L. Novick. Analogical transfer, problem similarity, amd expertise. Journal of experimental psychology. Learning, memory, and cognition, 14(3):510520, 1988. U. Schmid, J. Wirth, and K. Polkehn. A closer look at structural similarity in analogical transfer. Cognitive Science Quarterly, 3(1):5790, 2003. D. Schn. Displacement of concepts. Humanities Press, New York, 1963. o A. Schwering and et al. Using gestalt principles to compute analogies of geometric gures. In CogSci2007 Austin: TX, 2007. B.A. Spellman and K.J. Holyoak. Pragmatics in analogical mapping. Cognitive Psychology, 31(3):307346, 1996. M. Wertheimer. Gestalt Theory. The Gestalt Journal Press, New York, March 1924. Reprint (1997). 137

M. Wertheimer. Experimentelle Studien uber das Sehen von Bewegung. Zeitschrift fr Psychologie, 61:161265, 1929. u M. Wertheimer. Productive Thinking. Harper and Row, New York, 1954.

138

Appendix A First pretest

A.1 List of analogies

Red numbers 1 - 3 indicate the numbers of the answers in the multiple choice test. Those circled in red are considered the correct solutions in the case of the unambiguous analogies.

A1-1 1 2 3

A1- 2 1 2 3

A1- 3 1
A1- 4 1 2 3

139

A2 - 1 1 2 3

A2 - 2 1
A2 - 3

A2 - 4 1 2 3

U1 - 1 1 2 3

U1 - 2 1 2 3

140

U1 - 3 1
U1 - 4 1 2 3

U2 - 1 1 2 3

U2 - 2

U2 - 3 1 2 3

141

U2 - 4

A.2

Analogy combinations

Combo 1: U2c + A2c + A1m + U1m Combo 2: U1c + A2c + A1m + U2m Combo 3: U2c + A1c + A2m + U1m Combo 4: U1c + A1c + A2m + U2m A1 includes A1-1 to A-1-4, analogous for A2, U1, and U2. c indicates choose condition, m indicates make condition.

A.3

Participant information

Age Fifty-four (54) participants indicated their age. The participants ranged from 19 to 48 years of age. Mean age was 27.17 and median age was 26 (13.0%). Age group below 20 1 20-24 18 25-29 19 30-34 11 35-39 3 40-44 45 and above 2

142

Gender

Gender Of the 54 participants who indicated their gender, there were 30 males (53.6%) and 24 females (42.9%).
male

female

NA's

Handedness

Handedness There were 51 right-handed participants (91.1%) and two (2) ambidextrous participants (3.6%). No participant indicated left-handedness and three participants did not select their handedness.

right

ambidextrous NA's

IQ test history With respect to familiarity of geometric proportional analogies, participants were asked about the last time they took an IQ test. Most of the participants (26 participants or 46.4%) had never taken an IQ test before, while 5.4% of the participants (or 3 participants) had taken one recently within the past year. The rest had taken an IQ test either 1-5 years ago (12 participants or 21.4%), 5-10 years ago (5 participants or 8.9%) or more than 10 years ago prior to 1997 (7 participants or 12.5%). Three participants did not select an answer.
0.4

Time of last IQ test never 26 less than 1 year ago 3 1-5 years ago 12 5-10 years ago 5 more than 10 years ago 7

0.0

0.1

0.2

0.3

never

< 1 yr

15 yrs

510 yrs

> 10 yrs

143

Area of study In order to see if a mathematical/logic background would have an eect in terms of familiarity in solving analogy problems, a question was included on area of study. Only 41 participants gave a response. The subject areas were diverse and furthermore input was not uniform. Four participants (7.1%) were studying or lecturing in linguistics and there were three students each in computer science and statistics (5.4% respectively). Other studies included biology, cognitive science as well as disciplines in the humanities and natural sciences. Area of study Linguistics 4 Computer science 3 Statistics 3 Biology 2 Cognitive science 2 Other 27

Nationality Aside from half (50%) of the participants being German (28), the nationalities of the participants were varied. Five participants (8.9%) came from Mexico, followed by four (7.1%) from the USA, as well as three (5.4%) from Macedonia and two (3.6%) from Serbia. The rest of the nationalities were split among Europe (6 participants), Asia (3 participants) and one from South Africa. Four participants did not indicate their nationality.
0.5

Nationality German 28 Mexican 5 American 4 Macedonian 3 2 Serbian Other 10

0.0

0.1

0.2

0.3

0.4

DE

MX

US

MK

RS

Other

144

Occupation Looking at the responses of the 52 participants that indicated their occupation, more than half were students (60.7%), including high school, university and graduate studies. Five other participants additionally worked in the education sector (either as an employee or teaching) and two participants (3.6%) were apprentices. Occupation Student 34 Education (other than student) 5 IT 2 2 Banking Apprentice 2 2 Employee Other 5

A.4

Test environment

Two language versions (German and English) were set up to pool participants from within Germany as well as abroad. All 16 examples (A1-1 through A14, A2-1 through A2-4, U1-1 through U1-4 and U2-1 through U2-4) used were identical in both versions. Participants were randomly assigned to one of four analogy combinations (Combo1, Combo2, Combo3 and Combo4).
Language version

Language version
English

Of the 56 participants, 21 (37.5%) opted for the English version and 35 (62.5%) for the German version.
German

145

Combination

Randomized combination assignment

1

Answer 2 Answer 3 Answer 4 Answer 5 Answer 6 Answer 7 Answer 8 Answer 9 Answer 10 Answer 11 Answer 12

A1-1-D A1-2-D A1-3-D Despite some eort to achieve a A1-4-D quasi8 10 2 7 randomized assignment of the four dier15 5 6 7 1 1 5 ent analogy1 combinations, automatic as2 1 1 1 signment was unbalanced. 1 1 1 1 1 8 1 2 1 1 1 1 1 1 1

A2-1-D
2

A2-2-D 2 8 1 2 1
3

A2-3-D 11 1
4

A2-4-D 9 0 2 11 4 1 1 2

A.5
100

Results by analogy
A1-1
100

A1-2

75

75

50

Good continuation Proximity

50

25

25

7
1 2 3 4 5 6 7

Proximity/Similarity
Answer 1 Answer 3 Answer 5 Answer 7

Make

Choice A1-4

Make

MakeChoice A2-2

Choice

A1-1
100

100

75

75

Proximity
50

Good continuation
50

Similarity

25

25

146

100

100

75

75

50 100 25

A1-4

50 100 25

A2-2
Proximity

9
75 100 0 50 75 25 50 1 2 3 4 5 6 7 8 9 75 100 0 50 1

6
Symmetry
2 3

75 Similarity 50

A1-4
1 2 3 4 5 6 7 8 9

25

100 0 25 75 0

0 100 25

Proximity/Symmetry
1 2

A2-2
3

Make

Choice
3 4

A1100 50
75 25 50 0 25

41

A2-1
5

75 0
100

Make
1

MakeChoice A2-3
2

Choice
3

50

75

3
25

Proximity
1 2 3 4 5 6 7 8

50 Good continuation 0 9 25

Similarity
1 2 3

Make A2-1
1 2 3 4

Choice
5 6

Make
100
1 2

0 100

MakeChoice A2-3
3 4

Make
75

Choice

Make
75

MakeChoice Proximity

Choice

50

50

25

6
1 2 3 4 5 6

25

Proximity
0 1 2 3 4

Make

Choice

Make

MakeChoice

A2 - 1 1 2 3

Proximity

Good continuation

Similarity

147

A2-4
100

75

50

6
25

Proximity Similarity

0
A1-3-D A1-4-D 2 6 1 1 A1-4-D 1 1 1 1 1 2 1 1 1 1 1 1 A2-1-D 7 7 5 1 A2-1-D 2 1 1 1 1 1

Answer 1 Answer 2 Answer 3 Answer 4 Answer 5

A2 - 4

A1-3-D 1 1 1 2 1 1 1 1

Make A2-2-D A2-3-D 2 11 8 1 1 A2-3-D 2 A2-2-D 1

9 0 2

Choice A2-4-D 11 4 1 A2-4-D 1 1 2 2

2
Similarity

3
Proximity

Good continuation

1-1 A1-1
100 100 7575 5050 2525 0 0

A1-2 A1-2

Good continuation

Make Make
11 2 2 3 3 44 5 5 6 6 7

Choice Choice

4 4

5 5

6 6

7 7

Choice Choice

Make Make

MakeChoice MakeChoice A2-2 A2-2

Choice Choice

A1-4 A1-4

A1- 2
100 100 7575 5050 2525 0 0

1
Good continuation

2
Proximity

3
Proximity

148

100

75

50 100 25

A2-2

100 75 0 75 50 50 25 100 25 0 75 0 50 100

A2-2

Choice
7 8 9

Make
1

6 2-1

MakeChoice 2 3 A2-3

Choice
Good continuation

1
25 75

3
Symmetry

Proximity
6 7 8 9 0 50 1 2 3

Choice

-1
4 5 6

25

Make

MakeChoice A2-3
2 3 4

Choice

100 0 75

Choice

Make

MakeChoice

Choice

50

Proximity

25

Choice

Make

MakeChoice

Choice
Good continuation Good continuation Good continuation

A2 - 3

1
Proximity

3
Good continuation

149

A1-3
100

75

6
50

Similarity

25

8
1 2 3 4 5 6 7 8 9 10 11 12

Proximity

Make Choice

MakeChoice

Choice
Similarity

A1- 3 1
U1-1-C Answer 1 Answer 2 Answer 3 0 1 20 U1-2-C 0 21 0 U1-3-C 0 0 21 U1-4-C

2
U2-1-C 1 0 4 0 31

3
U2-2-C 1 0 34 U2-3-C 0 31 4 U2-4-C 0 35 0

Similarity
20

Proximity/Similarity

U1-1
100 100

U1-3

75

75

Symmetry

50

50

25

25

Make

MakeChoice

Choice

Make
Symmetry

Choice

U1 - 1 1
100

U1-4

75

50

25

150

100

75

50

U1-2
100

U1-4

100 25 75 0 50

75

50 25 25 0 0 1 2 3

Make

MakeChoice

Choice

Make

Choice

U1 - 2 1
U1-3-C 0 21 0 0 0 21 U1-4-C 1 0 20 U2-1-C 4 0 31 U2-2-C 1 34 0 U2-3-C 0 31 4 U2-4-C 0 35 0

Similarity

U1-3
100

75

50

25

Choice

Make

Choice

Make
Similarity/Proximity

Choice

U1 - 3
100

U1-4

75

50

25

151

100

75

50 100 25 75 0 50

U1-4

25

0 3

Choice

Make

Choice

Make

Similarity/Good Choice continuation

U1 - 4
U1-1-CP Answer 1 Answer 2 Answer 3 0 4.7619047619 95.238095238 U1-2-CP 0 100 0 U1-3-CP 0

1U1-4-CP
0

U2-1-CP 2

U2-2-CP 3

U2-3-CP 0

U2-4-CP 0 100 0

0 4.7619047619 11.428571429 2.8571428571 100 95.238095238 88.571428571

0 97.142857143 88.571428571 0 11.428571429

U2-1
100 100

U2-3

75

75

4
50 50

Symmetry

25

25

Make

MakeChoice

Choice

Make
Proximity

MakeChoice

U2 - 1 1 2

152

U2-2
100

U2-4

U2 - 2

100

75

75

50

50

25

25

Good continuation
0
U1-3-CP 0 U1-4-CP 0 4.7619047619 0

Make 11.428571429

U2-1-CP

MakeChoice 2.8571428571

U2-2-CP

U2-3-CP 0

U2-4-CP

Choice 0
0

Make

MakeChoice

Choice

0 97.142857143 88.571428571 0 11.428571429

100

100 95.238095238 88.571428571

1
100

U2-3

U2 - 3
75

50

Proximity
25

Choice

Make

MakeChoice

Choice

U2-4
100

U2 - 4

75

1
50

2
Proximity/Good continuation

25

Choice

Make

MakeChoice

Choice

A.6

Feedback

The following two sections comprise a summarized evaluation of the feedback that the subjects gave during the pretest.

153

Pros Instructions were very understandable and precise The vast majority knew exactly what to do at all times Embedded exercise examples & trash can received a positive feedback Apparently, the usability of the resources toolbar was quite easy and intuitive All in all, no severe technical problems occurred (except for the screen resolution issue, see Cons below) Commenting their solutions helped some subjects to clarify their taken actions (during the construction phase) or to realize mistakes afterwards A lot of people considered the rst part to be easier than the second part (which was our intention when we decided to let the subjects to the multiple-choice task rst) With respect to the ambiguous analogies, some subjects indeed realized that there were several plausible solutions in contrast to the unambiguous analogies (Question: Is this really a positive feedback or just a conrmation of our intentions?) Some subjects considered the analogies to have dierent levels of diculty (Question: Once again, rather a positive feedback for us or simply a conrmation of our intention to have examples that dier in their complexity?) Cons In the end, the experiment was not entirely compatible with resolutions below 1024x768 / 1280x720 (usage of arrow keys / space key necessary to display certain buttons) The analogy images seem to load very slowly when using slow internet connections like a 56k modem or ISDN (see general improvements below for a possible solution)

154

 Some subjects wanted for some feedback showing them how they performed throughout the experiment (similar to typical IQ tests => Note: providing exact percentages is impossible due to our ambiguous examples) Subjects needed signicantly less than 30 minutes to complete the experiment (the corresponding remark on the introduction page was obviously overstated) Certain analogy examples and its variations were used too often in comparison to other examples => even a small number of subjects repeatedly pointed out that some examples were quite familiar to them while they proceeded in the experiment (see general improvements below for a possible solution) A very small number of participants felt bored when they had to read again and again certain instructions like Please comment on your solution for D etc. Some participants had severe problems to comment on what they just did in the construction step before => that is, they did not know exactly what to write as they considered their solutions to be somewhat logical or obvious (this might be one possible reason why a rather high number of subjects did not comment their solutions at all) It was suggested to display the analogies in front of a visible grid such that the exact object positions become more obvious than it is the case now without a grid Some subjects skipped back some pages during the experiment e.g. to correct their constructions (see general improvements below for a possible solution)

A.7

Suggestions for improvement

The following section comprises a listing of general improvements that one has to think of for the real experiment. Comment box: message alert if no text was entered Disable back / forward buttons to prevent the subjects from skipping back and forth the pages: 1) for all browsers, 2) corresponding mouse buttons 155

 Return key issue (information page): suppress return key, otherwise form is submitted immediately Implement nice looking buttons instead of ugly local links Consistent position of button: always in same location at bottom left of screen Load all images before showing whole page (to circumvent gradual appearance of images when using modem / ISDN connection) Implement example-specic toolbars Make sure: equal size for all geometric elements Participant results to be stored in separate les Information form: values to be same for both language If possible: prevent shifting of colons (dierent browsers = unfortunately dierent positions of colons) Disallow multiple trials with dierent browsers Enable formal (real) randomization (to allow for a more sophisticated statistical analysis compared to the current one) Session management for unstable internet connections (continue where last left o) Hide address bar in IE popup window Snapshot images as explanation (especially on exercise pages) Flash animated instructions Set cookie after entering welcome page Enable scrolling in popup window (could resolve the resolution issue) Make sure: Everything is visible even when using low screen resolutions (could be problematic as we would lose space on each page) Remark: Some suggestions for improvement obviously become null and void provided that the real experiment is conducted oine (or online, but in an controlled environment)

156

Appendix B Second pretest

B.1 List of analogies
S1Original
: :: :

?
:: :

::

?
S1A2

?
S1A3

::

S1A1

::

?
S1C2

::

?
S1C3

::

S1C1

157

S2Original
: :: :

?
?
S2A3

::

?
S2A2

::

::

S2A1

::

?
S2C2

::

?
S2C3

::

S2C1

S3Original
: :: :

?
:

S3A1

::

S3A2

::

S3A3

::

::

S3C1

S3C2

::

S3C3

::

158

S4Original
: :: :

?
:

S4A1

::

? ?

S4A2

::

? ?

S4A3

::

? ?

::

S4C1

S4C2

::

::

S4C3

S5Original
: :: :

::

?
S5A2

::

::

?
S5A3

S5A1
: :: :

?
S5C2

::

?
S5C3

::

S5C1

S6Original
: :: :

?
:: :

::

?
S6A2

?
S6A3

::

?
?

S6A1
: :: :

?
S6C2

::

?
S6C3

::

S6C1

159

B.2

Stimuli combinations

There were a total of six original examples with six variations which can be grouped into two sets of three AB xed and three C xed, respectively. In addition to all original examples, the variation sets (A & C) were divided equally to create two dierent combinations of 24 examples each. Mix 1 Mix 2 S1 C S1 A S2 C S2 A S3 C S3 A S4 A S4 C S5 C S5 A S6 C S6 A

B.3

Example randomization

Two combinations each of the two mixes were created to give four combinations each in the Checkboxes and Radio buttons conditions. All examples within the combinations were completely randomized. Combo 1 S1C3 S3C3 S4A2 S2A3 S5Original S4A1 S6A2 S4Original S1Original S6A3 S3C1 S1C1 S5C1 S2Original S5C2 S4A3 S2A1 S6Original S5C3 Combo 2 S2A1 S2A2 S1Original S1C3 S1C1 S2Original S4A1 S3C3 S4A3 S5Original S5C2 S4A2 S3Original S5C3 S1C2 S6A1 S2A3 S5C1 S3C2 Combo 3 S2Original S3A3 S4Original S4C3 S5Original S3A2 S2C2 S3Original S4C2 S6C1 S6Original S3A1 S1A3 S4C1 S1A2 S6C3 S2C3 S5A3 S5A2 160 Combo 4 S4C1 S1A2 S5A3 S2Original S4C2 S5Original S1Original S4Original S6C3 S3A2 S3A3 S6C2 S2C3 S2C2 S5A1 S5A2 S2C1 S6Original S3Original

S3Original S6A1 S2A2 S1C2 S3C2

S4Original S3C1 S6A2 S6Original S6A3

S5A1 S2C1 S6C2 S1A1 S1Original

S1A1 S6C1 S1A3 S4C3 S3A1

Thus, each participant was given a set of 24 examples, in either the Checkboxes or the Radio buttons condition exclusively, from one of the mixes with all examples in a randomized order.

B.4

Randomization of Check/Radio options

The order in which the Check and Radio options were presented was randomized for each combination. The nal options (j in Check, f in Radio) in both of the conditions was the commenting option. Option e in the Radio buttons comments was always an incorrect solution. Checkboxes S1 a. Switch colors b. Move object(s) down c. Rotate by 180 d. Flip along diagonal axis e. Flip along horizontal axis f. Switch positions g. Move object(s) up h. Rotate by 90 i. Remove object j. Other S2 a. Black object(s) remain b. Move object(s) downwards c. Grey object(s) remain d. Remove white object(s) e. Remove black object(s) f. Remove grey object(s) g. Central object(s) remain h. Remove outer object(s) 161 Radio buttons

a. The colors of the objects are switched. b. The gure in C is rotated by 180 . c. The gure in C is ipped twice. d. The black object moves upwards. e. The black object moves downwards. f. None of the above

a. The central objects remain. b. The black objects remain. c. The white objects are removed. d. The outer objects are removed. e. The white and black objects switch colors. f. None of the above

i. Remove central object(s) j. Other S3 a. Rotate white object(s) b. Rotate grey object(s) c. Flip along vertical axis d. Rotate triangle(s) e. Flip along horizontal axis f. Rotate left object(s) g. Rotate right object(s) h. Rotate object(s) by 180 i. Rotate object(s) by 90 j. Other S4 a. Move object(s) upwards b. Move object(s) downwards c. Move object(s) around d. Flip along vertical axis e. Flip along horizontal axis f. Rotate object(s) by 90 g. Change positions h. Switch colors i. Rotate object(s) by 180 j. Other S5 a. Change position b. Move square(s) upwards c. Rotate object(s) by 180 d. Change shape e. Add object(s) f. Move circle(s)/rectangle(s) upwards g. Move circle(s)/rectangle(s) downwards h. Remove object(s) i. Move square(s) downwards j. Other

a. The gure in C is ipped horizontally. b. The gure in C turns 180 . c. The triangle in C is ipped. d. The gure in C is ipped vertically. e. The square in C is rotated by 90 . f. None of the above

a. The gure in C is ipped. b. The gure in C turns 180 . c. The objects in C change positions. d. The gure in C is reected vertically. e. Grey objects turn black. f. None of the above

a. The circle moves downwards. b. The circle moves upwards towards the square. c. The square moves downwards towards the circle. d. The square moves upwards. e. Gravity pulls the circle. f. None of the above

162

S6 a. Move square(s) diagonally downwards b. Change positions c. Move circle(s) diagonally upwards d. Flip along diagonal axis e. Rotate object(s) by 90 f. Switch colors g. Change shape h. Move circle(s) diagonally downwards i. Move square(s) diagonally upwards j. Other

a. The gure in C turns 90 . b. The white square moves downwards. c. The white square changes position with the lower circle. d. The objects switch color. e. The square moves upwards. f. None of the above

B.5
Age

Participant information
Age

Age group below 20 4 20-25 17 26-30 9 11 31-40 3 above 40

0.0

0.1

0.2

0.3

<20

2025

2630

3140

>40

Gender There were 33 males and 10 females.

Gender

female

male

163

Handedness There were 40 right-handed participants, one left-handed participant and one ambidextrous participant.

Handedness

left ambidextrous right

IQ test history
IQ

Time of last IQ test never 19 3 less than 1 year ago 1-5 years ago 12 5-10 years ago 5 more than 10 years ago 4

0.0

0.1

0.2

0.3

0.4

never

<1

15

510

>10

Expertise, nationality & occupation Area of study Humanities Natural sciences Computer science Engineering Social sciences Languages Maths Other 7 5 4 3 3 2 1 9 Nationality German 21 Macedonia 4 4 USA Indonesia 3 Switzerland 2 Other 7 Occupation Student 18 Employee 14 Self-employed 6 Unemployed 3 Other 3

164

Appendix C Mini tests: HIT and Technologietag

165

Figure C.1: Results for A2Original and variations

166

Figure C.2: Results for A3Original and variations 167

Appendix D Final test

D.1
0.35 0.4

Demographics
Age IQ

0.30

0.20 0.2 <10 1115 1620 2125 2630 3140 >40 0.0 less_than_1 0.1 0.05 0.00

0.15

0.10

0.3

0.25

15

510

more_than_10

never

Expertise Engineering Natural sciences Computer science Humanities Other NA

40 36 23 13 77 49

Nationality Germany 148 Indonesia 57 USA 8 Italy 4 Switzerland 3 Canada 2 Other 16

Occupation Student 110 Employee 77 Self-employed 19 Civil servant 8 Pupil 7 Unemployed 6 Other 11

168

D.2
S1OR

Stimuli

S1A1

S1A2

S1C1

169

S1C2

S2OR

S2A2

S2A3

170

S2C1

S2C2

S3OR

S3A1

S3A3

S3C2

171

S3C3

S4OR

S4A1

S4A2

S4C2

S4C3

172

S5OR

S5A1

S5A3

173

S5C3

S5C4

S6OR

S6A1

174

S6A3

S6C1

S6C2

175

D.3

Solution classication
Analogy
Gestalt S1OR S1A1 S1A2 S1C1 S1C2 S2OR S2A2 S2A3 S2C1 S2C2 S3OR S3A1 S3A3 S3C2 S3C3 S4OR S4A1 S4A2 S4C2 S4C3 S5OR S5A1 S5A3 S5C3 S5C4 S6OR S6A1 S6A3 S6C1 S6C2 GC GC similarity similarity GC DUMMY proximity similarity similarity similarity similarity proximity similarity proximity proximity symmetry proximity proximity proximity proximity GC GC GC GC GC proximity similarity similarity GC similarity

Solution 1
Transformation object movement colour saliency / object movement switch position switch colour object movement DUMMY deletion of non-central objects deletion of non-black objects deletion of white objects deletion of white objects rotation partially rotation rotation rotation rotation reflection rotation rotation rotation rotation object movement object movement object movement object movement object movement rotation switch colour switch colour object movement switch colour

Analogy
Gestalt S1OR S1A1 S1A2 S1C1 S1C2 S2OR S2A2 S2A3 S2C1 S2C2 S3OR S3A1 S3A3 S3C2 S3C3 proximity similarity GC similarity proximity similarity similarity proximity proximity similarity proximity similarity proximity similarity symmetry

Solution 2
Transformation rotation switch colour object movement reflection rotation deletion of non-black object deletion of grey objects deletion of non-central objects / colour switch deletion of non-central objects deletion of non-black objects rotation rotation partially rotation rotation partially reflection

176

S4OR S4A1 S4A2 S4C2 S4C3 S5OR S5A1 S5A3 S5C3 S5C4 S6OR S6A1 S6A3 S6C1 S6C2

proximity proximity symmetry symmetry proximity GC GC GC GC GC similarity proximity symmetry proximity proximity

rotation rotation, switch colour reflection reflection switch position object movement object movement object movement object movement object movement switch colour rotation reflection rotation switch position

Analogy
Gestalt S1OR S1A1 S1A2 S1C1 S1C2 S2OR S2A2 S2A3 S2C1 S2C2 S3OR S3A1 S3A3 S3C2 S3C3 S4OR S4A1 S4A2 S4C2 S4C3 S5OR S5A1 S5A3 S5C3 S5C4 S6OR S6A1 S6A3 S6C1 S6C2 similarity proximity proximity proximity similarity similarity proximity GC similarity proximity symmetry symmetry DUMMY symmetry similarity proximity symmetry proximity proximity symmetry GC GC GC GC GC GC GC similarity similarity proximity

Solution 3
Transformation switch colour rotation, switch colour rotation rotation switch colour deletion of non-central objects deletion of grey objects deletion of non-top group / object movement deletion of non-black objects deletion of non-central objects reflection reflection DUMMY reflection rotation partially rotation reflection / switch colour rotation rotation reflection object movement object movement object movement object movement object movement object movement object movement switch colour switch colour rotation

177

D.4

Solution analysis

178

Choose condition solution # of participants per solution 58/161 (36%) 59/161 (37%) 44/161 (27%) # of participants per solution 12/77 (15%) 24/77 (31%) 12/77 (15%) 30/77 (39%) # of comments per solution 3/32 (9%) 1 discarded 13/32 (41%) 5/32 (16%) 1 discarded 11/32 (34%) 3 discarded 32 5/32 (15.6%) cluster ID (same ID was used in the make condition) n33 n35 n34 # of participants per transformation 2

D1 D2

Analogy

D3 Make condition solution

transformation(s) derived from comments switch colours

S1A1

D1

D2

switch position rotation + switch colours

13 4

D3 novel

view next page(s) for details

total # of comments: total # of discarded comments: Novel solutions (make condition) solution # of participants per solution 1/30 1/30

cluster ID 4_2 4_3 4_5

transformation(s) derived from comments

# of participants per transformation

Analogy

1/30

2/30 2/30

4_6 4_7 4_8 4_9

switch colours + switch position

S1A1

1/30 2/30

switch colours + rotation

1/30

4_12

1/30

4_13

179

Novel solutions (make condition) solution # of participants per solution 2/30 cluster ID transformation(s) derived from comments switch colours # of participants per transformation 1

4_15

1/30

4_17

Analogy

1/30

4_18

1/30

4_20 switch colours reflection reflection + switch colours switch colours rotation + switch colours + switch position 1 1 1 1

S1A1

2/30 1/30 1/30

4_21 4_23 4_24

2/30

4_25

Analogy

Novel solutions (make condition) # of participants per solution solution 4/30

cluster ID 4_26

transformation(s) derived from comments

# of participants

S1A1
blank *

1/30 * 2/30

4_27

0_0

Remark: clusters 4_26 and 4_27 basically entail the same solutions, but due to the shifted black circle in 4_27 two distinct clusters were generated in the end!

180

Choose condition solution # of participants per solution 18/161 (11%) 52/161 (33%) 91/161 (56%) cluster ID (same ID was used in the make condition) n39 n36 n37

D1 D2

Analogy

D3 Make condition solution

# of participants per solution 4/77 (5%) 29/77 (38%)

# of comments per solution 1/34 (3%)

transformation(s) derived from comments switch position object movement switch position reflection rotation switch colours rotation switch position + rotation reflection switch position

# of participants per transformation 1 1 2 2 2 1 3 1 2 1

S1A2

D1

D2

12/34 (35%) 4 discarded

D3 novel

19/77 (25%) 25/77 (32%)

8/34 (24%) 1 discarded 13/34 (38%) 34 5/34 (14.7%)

view next page(s) for details

total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution 1/25 cluster ID 1_0 transformation(s) derived from comments # of participants per transformation

Analogy

1/25

4_12

rotation

15/25

n38

switch colours

S1A2

5/25

n40

reflection

1/25

4_13

blank

2/25

0_0

181

Choose condition solution # of participants per solution 43/162 (27%) 17/162 (10%) 102/162 (63%) # of participants per solution 17/76 (22%) 3/76 (4%) # of comments per solution 4/26 (16%) cluster ID (same ID was used in the make condition) n46 n42 n44 # of participants per transformation 1 3 2 1 2 2 5

D1 D2 D3 Make condition solution

Analogy

transformation(s) derived from comments switch position switch colours reflection switch colours + position switch position reflection rotation

S1C1

D1 D2

2/26 (8%)

D3

27/76 (36%)

10/26 (38%)

29/76 novel (38%) total # of comments: total # of discarded comments:

10/26 (38%) 26 0/26 (0.0%)

view next page(s) for details

Novel solutions (make condition) solution # of participants per solution 1/29 cluster ID 4_10 4_11 4_20 4_22 switch colours + position switch colours switch position switch colours 1 1 1 5 switch colours 1 transformation(s) derived from comments reflection # of participants per transformation 1

Analogy

1/29 1/29 1/29

S1C1
12/29 n41 11/29 2/29 n43 n45

182

Choose condition solution # of participants per solution 36/163 (22%) 84/163 (52%) 43/163 (26%) cluster ID (same ID was used in the make condition) n48 n51

D1 D2

n50

Analogy

D3 Make condition solution

# of participants per solution 24/75 (32%)

# of comments per solution 10/30 (33%) 2 discarded

transformation(s) derived from comments switch colours object movement + switch colours switch position reflection reflection rotation switch colours

# of participants per transformation 4 1 1 2 3 2 8

S1C2

D1

D2

23/75 (31%) 15/75 (20%) 13/75 (17%)

5/30 (17%)

8/30 (27%) 7/30 (23%) 30 2/30 (6.6%)

D3 novel

view next page(s) for details

total # of comments: total # of discarded comments: Novel solutions (make condition) solution # of participants per solution 1/13 1/13 1/13

cluster ID 4_0 4_1 4_10

transformation(s) derived from comments rotation

# of participants per transformation 1

Analogy

reflection

1/13

4_13

S1C2
1/13 3/13 5/13 4_14 n47 n49 rotation + switch colours reflection reflection 1 2 2

183

Choose condition solution D1 # of participants per solution 2/161 (1%) 30/161 (19%) D2 129/161 (80%) # of participants per solution 0/77 (0%) # of comments per solution cluster ID (same ID was used in the make condition) 1

n64

Analogy

D3 Make condition solution

1_0 # of participants per transformation

transformation(s) derived from comments

S2OR

D1

0/28 (0%) deletion of white objects deletion of non-black objects deletion of non-black objects 4 3 18

16/77 (21%) D2 D3 novel 58/77 (75%) 3/77 (4%)

8/28 (28%) 1 discarded 19/28 (68%) 1 discarded 1/28 (4%) 1 discarded 28 3/28 (10.7%)

view next page(s) for details

total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution cluster ID transformation(s) derived from comments # of participants per transformation

Analogy
1/3 2_1 1/3 2_2

S2OR
1/3 3_0

184

Choose condition solution D1 # of participants per solution 123/156 (79%) 18/156 (12%) D2 15/156 (9%) # of participants per solution 68/82 (83%) # of comments per solution 29/34 (85%) cluster ID (same ID was used in the make condition) 1_1

3_0

Analogy

D3 Make condition solution

3 # of participants per transformation 29

transformation(s) derived from comments deletion of non-central objects

S2A2

D1

5/82 (6%) D2 D3 novel 0/82 (0%) 9/82 (11%)

2/34 (6%)

deletion of grey objects

0/34 (0%) 3/34 (9%) 1 discarded 34 1/34 (2.9%) view next page(s) for details

total # of comments: total # of discarded comments: Novel solutions (make condition) solution # of participants per solution 1/9 1/9

cluster ID 1_0 1_2 2_0

transformation(s) derived from comments deletion of non-central objects

# of participants per transformation 1

Analogy

1/9

1/9

5_0

S2A2

1/9

5_1

1/9

5_2

add objects

blank

3/9

0_0

185

Choose condition solution # of participants per solution 71/157 (45%) 31/157 (20%) 55/157 (35%) # of participants per solution 28/81 (35%) # of comments per solution 13/34 (38%) 2 discarded cluster ID (same ID was used in the make condition) n59 n60 n61 # of participants per transformation 1 10 5

D1 D2

Analogy

D3 Make condition solution

transformation(s) derived from comments deletion of non-black objects + object movement deletion of non-black objects deletion of non-central objects + switch colours deletion of non-top group + object movement

S2A3

D1

D2

14/81 (17%) 25/81 (31%)

5/34 (15%) 10/34 (29%) 1 discarded 6/34 (18%) 1 discarded 34 4/34 (11.8%)

D3

14/81 novel (17%) total # of comments: total # of discarded comments:

view next page(s) for details

Choose condition solution # of participants per solution 71/157 (45%) 31/157 (20%) 55/157 (35%) cluster ID (same ID was used in the make condition) 4_1 1_0 3

D1 D2

Analogy

D3 Make condition solution

# of participants per solution 5/83 (6%) 41/83 (50%) 6/83 (7%) 31/83 (37%)

# of comments per solution 2/47 (4%)

transformation(s) derived from comments deletion of white objects

# of participants per transformation 2

S2C1

D1 D2

24/47 (51%) 4/47 (9%) 1 discarded 17/47 (36%) 5 discarded 47 6/47 (12.8%)

deletion of non-central objects deletion of non-black objects

24

D3 novel

view next page(s) for details

total # of comments: total # of discarded comments:

186

Novel solutions (make condition) solution # of participants per solution 1/31 2/31 16/31 cluster ID 2_0 2_1 3_1 3_3 3_4 deletion of non-central objects deletion of non-central objects + object movement deletion of non-central objects 1 deletion of non-central objects 9 transformation(s) derived from comments # of participants per transformation

Analogy

1/31 5/31

2/31

3_5

S2C1

2/31

3_6

1/31

4_0

1/31

7_0

Choose condition solution # of participants per solution 29/163 (18%) 23/163 (14%) 111/163 (68%) # of participants per solution 4/75 (6%) 6/75 (8%) 55/75 (73%) 10/75 (13%) # of comments per solution 2/34 (6%) cluster ID (same ID was used in the make condition) 2_0 1_0 n63 # of participants per transformation 2

D1 D2 D3 Make condition solution

Analogy

transformation(s) derived from comments deletion of white objects

S2C2

D1

D2

4/34 (12%) 25/34 (73%) 3 discarded 3/34 (9%) 1 discarded 34 4/34 (11.8%)

deletion of non-black objects deletion of non-central objects

D3 novel

22

view next page(s) for details

total # of comments: total # of discarded comments:

187

Novel solutions (make condition) solution # of participants per solution 2/10 cluster ID 1_2 2_3 2_4 deletion of white objects + switch colours deletion of grey objects 1 transformation(s) derived from comments # of participants per transformation

Analogy

1/10 3/10

S2C2

1/10

2_5

2/10 blank 1/10

2_6 0_0

Choose condition solution D1 D2 # of participants per solution 12/153 (8%) 47/153 (31%) 94/153 (61%) # of participants per solution 5/85 (6%) 51/85 (60%) 22/85 (26%) 7/85 (8%) # of comments per solution 4/37 (11%) cluster ID (same ID was used in the make condition) 2_5 n5 2_6 # of participants per transformation 1 3 5 17 7 1

Analogy S3OR

D3 Make condition solution

transformation(s) derived from comments reflection rotation reflection rotation reflection rotation

D1

D2 D3 novel

22/37 (59%) 10/37 (27%) 2 discarded 1/37 (3%) 37 2/37 (5.0%)

view next page(s) for details

total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution 1/7 2/7 cluster ID 1_0 2_1 2_2 2_3 0_0 reflection 1 transformation(s) derived from comments # of participants per transformation

Analogy S3OR
blank

1/7 1/7 2/7

188

Choose condition solution D1 D2 D3 Make condition solution # of participants per solution 13/159 (8%) 74/159 (47%) 72/159 (45%) # of participants per solution 2/79 (3%) 43/79 (54%) 23/79 (29%) 11/79 (14%) cluster ID (same ID was used in the make condition) 2_6 2_2 2_7 # of participants per transformation 1 4 2 7 8 1 4

Analogy S3A1

# of comments per solution

transformation(s) derived from comments

D1

1/30 (3%)

reflection reflection rotation partial rotation reflection rotation partial rotation

D2

14/30 (47%) 1 discarded 13/30 (43%) 2/30 (7%) 1 discarded 30 2/30 (6.6%)

D3 novel

view next page(s) for details

total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution 1/11 1/11 1/11 cluster ID 1_0 2_0 2_1 2_4 2_5 n1 0_0 rotation 1 transformation(s) derived from comments # of participants per transformation

Analogy S3A1
blank

2/11 1/11 3/11 2/11

Choose condition solution D1 D2 D3 Make condition solution # of participants per solution 138/155 (89%) 9/155 (6%) 8/155 (5%) # of participants per solution 75/83 (91%) 1/83 (1%) 0/83 (0%) 7/83 (8%) # of comments per solution 30/33 (91%) 4 discarded 1/33 (3%) 1 discarded 0/33 (0%) 2/33 (6%) 2 discarded 33 7/33 (21.2%) view next page(s) for details cluster ID (same ID was used in the make condition) n2 2_0 3 # of participants per transformation 12 14

Analogy S3A3

transformation(s) derived from comments reflection rotation

D1

D2

D3 novel

total # of comments: total # of discarded comments:

189

Analogy S3A3

Novel solutions (make condition) solution # of participants per solution 6/7 blank 1/7 cluster ID transformation(s) derived from comments # of participants per transformation

n3 0_0

Choose condition solution D1 D2 D3 Make condition solution # of participants per solution 89/161 (56%) 20/161 (12%) 52/161 (32%) # of participants per solution 36/77 (47%) 5/77 (6%) 26/77 (34%) 10/77 (13%) cluster ID (same ID was used in the make condition) 3_6 3_7 n4 # of comments per solution 16/36 (46%) 5/36 (14%) 2 discarded 12/36 (32%) 3/36 (8%) 36 2/36 (5.6%) transformation(s) derived from comments reflection rotation reflection + rotation partial rotation # of participants per transformation 7 8 1 3

Analogy S3C2

D1 D2

D3 novel

reflection

12

view next page(s) for details

total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution 1/10 cluster ID transformation(s) derived from comments rotation # of participants per transformation 1

3_0 3_1 3_3 3_4 3_5 5_0 0_0

Analogy S3C2

3/10 1/10 1/10 1/10 1/10 blank 2/10

rotation rotation + switch colours

1 1

190

Choose condition solution D1 D2 D3 Make condition solution # of participants per solution 57/158 (36%) 70/158 (44%) 31/158 (20%) # of participants per solution 24/80 (30%) 30/80 (38%) 11/80 (14%) 15/80 (18%) cluster ID (same ID was used in the make condition) 2_2 2_5 2_3 # of comments per solution 14/38 (37%) 1 discarded 18/38 (47%) 1 discarded 2/38 (5%) 4/38 (11%) 38 2/38 (5.3%) transformation(s) derived from comments rotation reflection rotation reflection # of participants per transformation 13 16 1 2

Analogy S3C3

D1

D2

D3 novel

view next page(s) for details

total # of comments: total # of discarded comments: Novel solutions (make condition) solution # of participants per solution 1/15 5/15 5/15

cluster ID 2_0 2_1 2_4 2_6 2_7 0_0

transformation(s) derived from comments

# of participants per transformation

Analogy S3C3
blank

reflection partial rotation reflection

1 1 2

2/15 1/15 1/15

191

Choose condition solution # of participants per solution 44/157 (28%) 2/157 (1%) 111/157 (71%) cluster ID (same ID was used in the make condition) 2_1 2 2_0

D1 D2

Analogy S4OR

D3 Make condition solution

# of participants per solution 14/81 (17%) 0/81 (0%) 62/81 (77%) 5/81 (6%)

# of comments per solution 9/43 (21%)

transformation(s) derived from comments reflection

# of participants per transformation 9

D1 D2

0/42 (0%) 34/43 (79%) 1 discarded 0/43 (0%) 43 1/43 (2.3%) rotation reflection switch position rotation + reflection 15 11 6 1

D3 novel

view next page(s) for details

total # of comments: total # of discarded comments: Novel solutions (make condition)

Analogy S4OR

solution

# of participants per solution 2/5 2/5

cluster ID 2_2 n12 0_0

transformation(s) derived from comments

# of participants per transformation

blank

1/5

Choose condition solution D1 D2 D3 Make condition # of participants per solution 27/159 (17%) 96/159 (60%) 36/159 (23%) # of participants per solution 7/79 (9%) # of comments per solution 1/37 (2%) cluster ID (same ID was used in the make condition) 2_5 n6 2_1 # of participants per transformation
1 9 7 2 4 1 1 2 1 1 3

Analogy S4A1

solution

transformation(s) derived from comments

rotation rotation + switch colours switch colours + position reflection + switch colours change objects/shapes + switch colours colour saliency + object movement switch objects reflection + switch colours rotation + switch colours switch colours + position switch colours

D1

D2

47/79 (59%)

24/37 (65%)

D3 novel

14/79 (18%) 11/79 (14%)

7/37 (19%) 5/37 (14%) 3 discarded 37 3/37 (8.1%)

view next page(s) for details

total # of comments: total # of discarded comments:

192

Novel solutions (make condition) solution # of participants per solution 1/11 1/11 cluster ID 1_0 2_0 transformation(s) derived from comments # of participants per transformation

Analogy S4A1

1/11

2_3

1/11

2_4

reflection + switch colours reflection + retain colours

5/11

2_6

1/11 blank 1/11

3_1 0_0

Choose condition solution # of participants per solution 20/161 (12%) 69/161 (43%) 72/161 (45%) # of participants per solution 9/77 (12%) 29/77 (38%) 34/77 (44%) 5/77 (6%) # of comments per solution 2/36 (6%) 13/36 (36%) 2 discarded 21/36 (58%) 1 discarded 0/36 (0%) 36 3/36 (8.3%) cluster ID (same ID was used in the make condition) 2_5 2_3 2_1 transformation(s) derived from comments reflection rotation + switch colours rotation switch position # of participants per transformation 2 11 19 1

D1 D2

Analogy S4A2

D3 Make condition solution

D1 D2 D3 novel

view next page(s) for details

total # of comments: total # of discarded comments: Novel solutions (make condition) solution # of participants per solution 1/5 1/5

cluster ID 2_0 2_2

Analogy S4A2

transformation(s) derived from comments

# of participants per transformation

1/5 blank 2/5

2_4 0_0

193

Choose condition solution # of participants per solution 105/158 (66%) 49/158 (31%) 4/158 (3%) # of participants per solution 46/81 (57%) 21/81 (26%) 0/81 (0%) # of comments per solution 21/41 (51%) 1 discarded 15/41 (37%) cluster ID (same ID was used in the make condition) n7 2_5 3 transformation(s) derived from comments rotation reflection reflection # of participants per transformation 10 10 15

D1

D2

Analogy S4C2

D3 Make condition solution

D1

D2 D3

0/41 (0%) 5/41 (12%) 1 discarded 41 2/41 (4.9%) view next page(s) for details

14/81 novel (17%) total # of comments: total # of discarded comments: Novel solutions (make condition) solution # of participants per solution 1/14

cluster ID 1_0 2_1 2_3 2_4

transformation(s) derived from comments

# of participants per transformation

Analogy S4C2
*

1/14 1/14 1/14

1/14

4_0

switch position

7/14 blank 2/14

n8 0_0

switch position

* One participant provided two solutions at a time. Only the left figure is a novel solution. The right figure matches D1 and is thus crossed-out in this context.

194

Choose condition solution # of participants per solution 95/155 (61%) 20/155 (13%) 40/155 (26%) cluster ID (same ID was used in the make condition) n9

D1

n10

D2

Analogy S4C3

n11

D3 Make condition solution

# of participants per solution 42/83 (50%) 8/83 (10%) 25/83 (30%) 8/83 (10%)

# of comments per solution 16/37 (43%) 6/37 (17%) 1 discarded 13/37 (35%) 1 discarded 2/37 (5%) 1 discarded 37 3/37 (8.1%)

transformation(s) derived from comments rotation reflection switch position reflection rotation

# of participants per transformation 10 6 5 10 2

D1

D2

D3 novel

view next page(s) for details

total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution 1/8 cluster ID 1_0 transformation(s) derived from comments # of participants per transformation

Analogy S4C3

1/8

2_1

1/8 1/8 1/8

2_2 2_3 2_5

1/8 blank 2/8

2_6 0_0

switch colours + position

195

Choose condition solution D1 D2 # of participants per solution 80/156 (51%) 71/156 (46%) 5/156 (3%) cluster ID (same ID was used in the make condition) n23 n22

Analogy

D3 Make condition solution # of participants per solution 45/82 (55%) 33/82 (40%) 0/82 (0%) D3 novel 4/82 (5%) total # of comments: total # of discarded comments: 1/29 (3%) 1 discarded 29 7/29 (24.1%) view next page(s) for details # of comments per solution 19/29 (66%) 4 discarded 9/29 (31%) 2 discarded transformation(s) derived from comments object movement switch position
switch objects

S5OR

D1

D2

object movement switch position

object movement + rotation

# of participants per transformation 8 6 1 5 1 1

0/29 (0%)

Novel solutions (make condition)

Analogy

solution

# of participants per solution 1/4

cluster ID 1_0

transformation(s) derived from comments

# of participants per transformation

S5OR
blank

1/4

2_1

2/4

0_0

196

Choose condition solution D1 # of participants per solution 81/154 (53%) 6/154 (4%) 67/154 (43%) # of participants per solution 28/84 (33%) 0/84 (0%) # of comments per solution 7/33 (21%) cluster ID (same ID was used in the make condition) n13 2 n14 # of participants per transformation 2 3 2

D2

Analogy

D3 Make condition solution

transformation(s) derived from comments object movement rotation reflection

D1

S5A1
D2

0/33 (0%) object movement switch position switch colours + position switch colours 8 6 3 1

D3 novel

50/84 (60%) 6/84 (7%)

25/33 (76%) 7 discarded 1/33 (3%) 33 7/33 (21.2%)

view next page(s) for details

total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution 1/6 cluster ID 2_0 transformation(s) derived from comments # of participants per transformation

Analogy

1/6

6_2

S5A1

2/6

6_5

rotation

1/6 blank 1/6

6_7 0_0

197

Choose condition solution D1 D2 D3 Make condition solution # of participants per solution 64/157 (41%) 83/157 (53%) 10/157 (6%) # of participants per solution 20/81 (25%) # of comments per solution 7/28 (25%) 1 discarded cluster ID (same ID was used in the make condition) n16 n15 3 # of participants per transformation 2 1 2 1 5 5 3 1 1

Analogy

transformation(s) derived from comments object movement switch position add objects rotation object movement switch position add objects
change objects/shapes + switch colours gravity (object attraction)

D1

S5A3
D2 50/81 (62%) 18/28 (64%) 3 discarded

D3 novel

0/81 (0%) 11/81 (13%)

0/28 (0%) 3/28 (11%) 28 4/28 (14.3%) view next page(s) for details

total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution 1/11 1/11 cluster ID 1_0 2_0 transformation(s) derived from comments # of participants per transformation

Analogy

* 1/11 2_1

1/11 *

2_2

S5A3

1/11

6_2 add objects switch position 1 1 1

4/11

6_6

1/11

6_7

switch colours and objects

blank 1/11 0_0 Remark: clusters 2_0 and 2_2 are obviously the same. Error during clustering?

198

Choose condition solution # of participants per solution 86/159 (54%) 71/159 (45%) 2/159 (1%) D3 Make condition solution # of participants per solution 46/79 (58%) 29/79 (37%) 3/79 (4%) D3 novel 1/79 (1%) 1/32 (3%) 32 0/32 (0.0%) view next page(s) for details total # of comments: total # of discarded comments: # of comments per solution 20/32 (63%) 10/32 (31%) transformation(s) derived from comments object movement switch position object movement switch position object movement # of participants per transformation 14 6 8 2 1 cluster ID (same ID was used in the make condition) n19 n17

D1 D2

n18

Analogy

S5C3

D1 D2

1/32 (3%)

Analogy

Novel solutions (make condition) solution # of participants per solution cluster ID transformation(s) derived from comments # of participants per transformation

S5C3

1/1

2_0

gravity (object attraction)

199

Choose condition solution # of participants per solution 72/156 (46%) 3/156 (2%) 81/156 (52%) # of participants per solution 32/82 (39%) 1/82 (1%) 47/82 (57%) 2/82 (3%) # of comments per solution 15/35 (42%) 2 discarded 1/35 (3%) cluster ID (same ID was used in the make condition) n20

D1

2_4

D2

Analogy

D3 Make condition solution

n21 # of participants per transformation 10 2 1 1 12 5 1 1

transformation(s) derived from comments object movement switch position

switch position + rotation

S5C4

D1

object movement object movement switch position switch colours retain position

D2

D3 novel

19/35 (52%) 0/35 (0%) 35 2/35 (5.5%)

view next page(s) for details

total # of comments: total # of discarded comments: Novel solutions (make condition)

Analogy

solution

# of participants per solution

cluster ID

transformation(s) derived from comments

# of participants per transformation

1/2

1_0

S5C4

blank

1/2

0_0

200

Choose condition solution # of participants per solution 145/160 (91%) 10/160 (6%) 5/160 (3%) cluster ID (same ID was used in the make condition) n31

D1

n32

D2

Analogy S6OR

4_3

D3 Make condition solution

# of participants per solution 60/78 (77%) 13/78 (17%) 1/78 (1%) 4/78 (5%)

# of comments per solution 28/37 (75%) 2 discarded 7/37 (19%)

transformation(s) derived from comments rotation switch colours rotation rotation + switch colours rotation

# of participants per transformation 26 2 3 2 1

D1

D2

1/37 (3%) 1/37 (3%) 1 discarded 37 3/37 (8.1%)

D3 novel

view next page(s) for details

total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution 1/4 1/4 cluster ID transformation(s) derived from comments # of participants per transformation

Analogy S6OR
blank

1_0 2_0

1/4

4_1

1/4

0_0

201

Choose condition solution # of participants per solution 21/163 (13%) 139/163 (85%) 3/163 (2%) # of participants per solution 3/75 (4%) 67/75 (89%) 0/75 (0%) # of comments per solution 2/38 (5%) 35/38 (92%) 2 discarded 0/38 (0%) 1/38 (3%) 38 2/38 (5.2%) view next page(s) for details cluster ID (same ID was used in the make condition) 4_3 n25 3 # of participants per transformation 1 1 31 1 1

D1

D2

Analogy S6A1

D3 Make condition solution

transformation(s) derived from comments rotation switch colours rotation switch position reflection

D1

D2

D3

5/75 novel (7%) total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution 1/5 1/5 cluster ID 1_0 3_0 transformation(s) derived from comments # of participants per transformation

Analogy S6A1

1/5

4_1

rotation

1/5 blank 1/5

4_2 0_0

202

Choose condition solution # of participants per solution 37/159 (23%) 7/159 (4%) 115/159 (73%) cluster ID (same ID was used in the make condition) n27 2

D1

D2

Analogy S6A3

n26

D3 Make condition solution

# of participants per solution 15/79 (19%) 0/79 (0%) 54/79 (68%) 10/79 (13%)

# of comments per solution 9/31 (29%)

transformation(s) derived from comments switch colours

# of participants per transformation 9

D1

0/31 (0%)

D2

19/31 (61%) 3/31 (10%) 31 0/31 (0.0%)

switch colours

19

D3 novel

view next page(s) for details

total # of comments: total # of discarded comments: Novel solutions (make condition) solution # of participants per solution 1/10

cluster ID 1_0

transformation(s) derived from comments

# of participants per transformation

1/10

4_0

partial rotation

Analogy
1/10 4_1

S6A3

1/10

4_3

switch colours

1/10

4_8

3/10 blank 2/10

4_9 0_0

object movement

203

Choose condition solution # of participants per solution 1/158 (1%) 122/158 (77%) 35/158 (22%) cluster ID (same ID was used in the make condition) 1 n28

D1 D2

4_5

Analogy S6C1

D3 Make condition solution

# of participants per solution 0/80 (0%) 56/80 (70%)

# of comments per solution 0/41 (0%) 30/41 (73%)

transformation(s) derived from comments

# of participants per transformation

D1 D2

rotation switch colours reflection rotation rotation + switch colours switch position partial rotation

30 2 1 1 2 1 1

D3 novel

18/80 (22%)

9/41 (22%) 1 discarded

6/80 (8%)

2/41 (5%) 41 1/41 (2.4%)

view next page(s) for details

total # of comments: total # of discarded comments: Novel solutions (make condition) solution # of participants per solution 1/6

cluster ID

transformation(s) derived from comments rotation

# of participants per transformation 1

4_0

Analogy
1/6 4_2 object movement / gravity 1

S6C1

1/6

4_3

1/6 blank 2/6

4_4 0_0

204

Choose condition solution # of participants per solution 14/161 (9%) 23/161 (14%) 124/161 (77%) cluster ID (same ID was used in the make condition) n30 4_6

D1 D2

Analogy S6C2

n29

D3 Make condition solution

# of participants per solution 3/77 (4%) 5/77 (6%) 60/77 (78%) 9/77 (12%)

# of comments per solution 1/39 (3%) 2/39 (5%) 1 discarded 33/39 (84%) 3/39 (8%) 1 discarded 39 2/39 (5.1%)

transformation(s) derived from comments partial rotation + switch colours switch position rotation partial rotation

# of participants per transformation 1 1 32 1

D1 D2

D3 novel

view next page(s) for details

total # of comments: total # of discarded comments:

Novel solutions (make condition) solution # of participants per solution 1/9 1/9 1/9 cluster ID 3_0 4_0 4_3 transformation(s) derived from comments # of participants per transformation

Analogy
1/9 4_5 rotation 1

S6C2

1/9

4_7

2/9

4_10

switch colours

1/9

4_11

1/9

5_0

205