A Sci ence of Reasoni ng: Ext e nde d Abs t ract *

Alan Bundy
Department of Artificial Intelligence
University of Edinburgh
Edinburgh, EH1 1HN, Scotland
Email: bundy@edinburgh.ac.uk, Tel: 44-31-225-7774
Abs t ract
How can we under st and reasoni ng in general and mat hemat i cal proofs in part i cul ar? It
is ar gued t hat a high-level under st andi ng of proofs is needed t o compl ement t he low-level
under st andi ng provi ded by Logic. A role for comput at i on is proposed to provi de this
high-level under st andi ng, namel y by t he associ at i on of proof plans wi t h proofs. Cri t eri a
are gi ven for assessi ng t he associ at i on of a pr oof pl an wi t h a proof.
1 Mot i vat i on: t he unders t andi ng of mat he mat i c al
proof s
We ar gue t hat Logic 1 is not enough t o under st and reasoning. It provi des onl y a low-
level, st ep by st ep under st andi ng, whereas a high-level, st rat egi c under st andi ng is also
requi red. Many commonl y observed phenomena of reasoni ng cannot be expl ai ned wi t h-
out such a high-level underst andi ng. Fur t her mor e, aut omat i c reasoni ng is i mpract i cal
wi t hout a high-level underst andi ng.
We pr opose a science of reasoni ng whi ch provi des bot h a low- and a high-level un-
der st andi ng of reasoni ng. It combi nes Logic wi t h t he concept of proof plans, [Bundy 88].
*The research reported in this paper was supported by SERC grant GR/E/44598 and an SERC Senior
Fellowship to the author. I would like to thank other members of the mathematical reasoning group at
Edinburgh for feedback, especially Frank van Harmelen, Colin Phillips, Mitch Harris and Toby Walsh. I
am grateful for comments on the first draft of this paper from two anonymous referees. The full version
of this extended abstract will appear in ~Computational Logic: Essays in Honor of Alan Robinson", MIT
Press, forthcoming.
1 We adopt the convention of using uncapitalised 'logic' for the various mathematical theories and
capitalised 'Logic' for the discipline in which these logics are studied.
634
We i l l ust r at e t hi s wi t h exampl es f r o m ma t h e ma t i c a l r easoni ng, but it is i nt ended t h a t
t he sci ence s houl d event ual l y a p p l y t o all ki nds of r easoni ng.
2 Th e Ne e d f or Hi g h e r - Le v e l E x p l a n a t i o n s
A pr oof in a l ogi c is a sequence of f or mul a e wher e each f or mul a in t he sequence is ei t her
a n a xi om or is der i ved f r om ear l i er f or mul a e i n t he sequence b y a r ul e of i nference.
Ea c h ma t h e ma t i c a l t he or y defi nes wha t it me a ns t o be a f or mul a, a n a xi om or a rul e
of i nference. Th u s Logi c pr ovi des a l ow-l evel expl anat i on of a ma t h e ma t i c a l pr oof l I t
expl ai ns t he pr oof as a sequence of s t eps a nd shows how each s t ep fol l ows f r om pr evi ous
ones by a set of rul es. I t s concer ns ar e l i mi t ed t o t he s oundnes s of t he pr oof , a nd t o t he
t r u t h of pr opos e d conj ect ur es in model s of l ogi cal t heor i es.
Whi l e Logi c pr ovi des an e xpl a na t i on of how t he st eps of a pr oof fit t oget her , it is
i na de qua t e t o expl ai n ma n y c o mmo n obs er vat i ons a b o u t ma t h e ma t i c a l pr oof s.
Ma t he ma t i c i a ns di st i ngui sh be t we e n unde r s t a ndi ng each s t ep of a pr oof and un-
de r s t a ndi ng t he whol e pr oof .
Ma t he ma t i c i a ns r ecogni se fami l i es of pr oof s whi ch cont ai n c o mmo n s t r uct ur e.
Ma t h e ma t i c i a n s use t hei r exper i ence of pr evi ous l y encount er ed pr oof s t o hel p t h e m
di scover new pr oof s.
Ma t h e ma t i c i a n s di s t i ngui s h bet ween ' i nt e r e s t i ng' and ' s t a n d a r d ' s t eps of a pr oof .
Ma t h e ma t i c i a n s of t en have a n i nt ui t i on t h a t a conj ect ur e is t r ue, but t hi s i nt ui t i on
is fal l i bl e.
St udent s of ma t he ma t i c s , pr es ent ed wi t h t he s ame pr oof s, l ear n f r om t h e m wi t h
var yi ng degr ees of success.
3 Co mmo n S t r u c t u r e i n Pr o o f s
Sever al r es ear cher s in a u t o ma t i c t h e o r e m pr ovi ng have i dent i fi ed c o mmo n s t r uc t ur e in
f ami l i es of pr oof s. For i nst ance,
[ Bundy & We l h a m 81] descr i bes t he c o mmo n s t r uc t ur e i n sol ut i ons t o s ymbol i c
equat i ons. Thi s c o mmo n s t r uc t ur e was i mp l e me n t e d i n a pr ocess of recta-level
inference whi ch gui ded t he s ear ch f or sol ut i ons t o equat i ons.
[ Bundy et al 88] descr i bes t he c ommon s t r uc t ur e in i nduct i ve t he or e ms a b o u t nat -
ur al numbers~ lists, etc. Thi s c o mmo n s t r uc t ur e was i mpl e me nt e d as an inductive
proof plan whi ch was used t o gui de t he sear ch for pr oof s of s uch t heor ems .
635
[Bledsoe et al 72] describes the common structure in theorems about limits of func-
tions in analysis. This common structure was implemented as the limit heuristic
and used to guide the search for proofs of such theorems.
[Wos & McCune 88] describes the common structure in at t empt s to find fixed-
points combinators. This common structure was implemented as the kernel method
and used t o guide the search for such fixed-points.
[Polya 65] describes the common structure in ruler and compass constructions.
This common structure was implemented by [Funt 73] and used to guide the search
for such constructions.
4 P r o o f P l a n s
Common st ruct ure in proofs can be captured in proof plans. A proof plan consists of
two parts: a tactic and a method. A tactic is a procedure which constructs part of
a proof by applying a sequence of rules of inference. High-level tactics are defined in
terms of lower-level sub-tactics. The lowest level tactics will apply individual rules of
inference. A method is a partial specification of a tactic. It consists of preconditions
which must be satisfied before the tactic is executed and some effects which will be true
provided the tactic application is successful.
Proof plans have been implemented within the OYSTER-CLAM system, [Bundy et al 88].
OYSTER is a t heorem prover for Intuitionist Type Theory. CLAM is a plan formation pro-
gram which has access to a number of general-purpose tactics and methods for inductive
proofs. This system has been used to control the search for inductive proofs about natu-
ral numbers and lists. CLAM constructs a special-purpose proof plan for each conjecture
out of its methods and tactics. The tactic of the proof plan is then executed. It in-
structs OYSTER to build a proof of the conjecture. The search for a proof plan at the
meta-level is considerably cheaper than the search for a proof at the object-level. This
makes proof plans a practical solution to the problems of search control in automatic
t heorem proving.
5 Th e Hi g h - Le v e l Un d e r s t a n d i n g o f Pr o o f s
Thus a high-level explanation of a proof of a conjecture is obtained by associating a proof
plan with it. The tactic of this proof plan must construct the proof. The method of
this proof plan must describe bot h the preconditions which made this tactic appropriate
for proving this conjecture and the effects of this t act i cs application on the conjecture.
It must also describe the role of each sub-tactic in achieving the preconditions of later
sub-tactics and the final effects of the whole tactic.
In fact, this association provides a multi-level explanation. The proof plan associated
with the whole proof provides the top-level explanation. The immediate sub-tactics and
636
sub-methods of this proof plan provide a medium-level explanation of the major sub-
proofs. The tactics and methods associated with individual rules of inference provide a
bottom-level explanation, which is similar to t hat already provided by Logic.
The general-purpose tactics and methods which we will use to build proof plans,
and the association of proof plans with proofs will constitute the theories of our science
of reasoning. This extends the way in which logical theories and the association of
logical proofs with real proofs and arguments, constitute the theories of Logic (especially
Philosophical Logic). Just as Logic also has meta-theories about the properties of and
relations between logical theories, we may also be able to develop such meta-theories
about proof plans.
6 Wh a t i s t he Na t u r e of our Sc i e nc e of Re a s o n i n g ?
Before we can dignify this proposed study of the structure of proofs with the epithet
science we must address a fundamental problem about the nature of such a science.
Traditional sciences like Physics and Chemistry study physical objects and the way
they interact. The subject of our proposed science is proof plans. But proof plans are
not physical objects. If they can be said to exist at all it is in the minds of mathemati-
cians proving theorems, teachers explaining proofs and students understanding them.
Physicists assume that the electrons in the apple I am eating as I write are essentially
the same as the electrons in some distant star. But proof plans will differ from mind to
mind and from time to time. There will be billions of such proof plans. Are we doomed
merely to catalogue t hem all? Given the difficulty of discovering the nature of even one
such proof plan, what a difficult and ultimately pointless task this would be. We would
prefer to narrow our focus on a few representative proof plans. But on what basis could
these few be chosen?
Fortunately, this is not a new problem. It is one faced by all human sciences to some
extent and it is one t hat has been solved before. Consider the science of Linguistics.
In Linguistics the theories are grammars and the association of grammatical structure
with utterances. Linguists do not try to form different grammars for each person, but
t ry to form a grammar for each language, capturing the commonality between different
users of t hat language. They t ry to make these grammars as parsimonious as possible,
so t hat they capture the maximum amount of generality within and between languages.
Linguists do not claim t hat everyone or anyone has these target grammars stored in
their head - - nor, indeed, t hat anyone has a grammar at all - - only t hat they specify
the grammatical sentences of the language.
Another example is Logic itself. Again judged by the arguments people produce, the
logical laws differ between minds and vary over time. Logicians do not try to capture
this variety, but confine themselves t o a few logics which specify ' correct' arguments.
As with grammatical sentences, correct arguments are identified by initial observation
of arguments actually used and consultation with experts to decide which of these are
correct.
637
I place our proposed science of reasoning between Linguistics and Logic. Proof
plans are more universal t han grammatical rules, but it is possible to associate different,
equally appropriate proof plans with the same proof. The study of proof plans appeals
both to an empirical study of the way in which mathematicians structure their proofs
and to reflection on the use of logical laws to put together proofs out of parts.
Thus there are strong precedents for a science t hat takes mental objects as its domain
of st udy and tames the wide diversity of exemplars by imposing a normative explanation
informed by reflection and empirical study. It only remains to propose criteria for
associating proof plans with proofs t hat will enable us to prefer one proof plan to
another. This we can do by appealing to general scientific principles. Our proposals are
given in the next section.
7 Cr i t e r i a f or As s e s s i n g P r o o f P l a n s
If there were no criteria for the association of proof plans with proofs, then we could
carry out our programme by associating with each proof an a d hoe tactic consisting of
the concatenation of the rules of inference required to reproduce it, and constructing
an a d hoe method in a similar way. This would not go beyond the existing logical
explanation.
The only assessment criterion we have proposed so far is c o r r e c t n e s s , i . e . t hat the
tactic of the proof plan associated with a proof will construct t hat proof when executed.
We now discuss some other possible criteria.
I n t u i t i v e n e s s : the way in which the proof is structured by a proof plan accords
with our intuitions about how we structure the proof.
P s y c h o l o g i c a l Val i di t y : there is experimental evidence t hat all, most or some math-
ematicians producing or studying proofs also structured a proof in the way sug-
gested by some proof plan.
Ex p e c t a n c y : . there must be a basis for predicting the successful outcome of a proof
plan.
Ge n e r a l i t y : a proof plan gets credit from the number of proofs or sub-proofs with
which it is associated and for which it accounts.
P r e s c r i p t i v e n e s s : a proof plan gets more credit the less search its tactic generates
and the more it prescribes exactly what rules of inference to apply.
S i mp l i c i t y : a proof plan gets more credit for being succinctly stated.
Ej f f i c i e n e y : a proof plan gets more credit when its tactic is computationally effi-
cient.
638
Parsimony; t he overal l t he or y get s mor e cr edi t t he f ewer gener al - pur pos e pr oof
pl ans ar e r equi r ed t o account for some col l ect i on of proofs.
We mi ght s t ar t desi gni ng pr oof pl ans usi ng t he cr i t er i a of i nt ui t i veness and psy-
chological val i di t y as sources of i nspi r at i on, but t he n use t he cr i t er i a of cor r ect ness,
expect ancy, general i t y, pr escr i pt i veness, si mpl i ci t y, effi ci ency and par s i mony t o revi se
t hem.
8 The Rol e of t he Co mput e r
So f ar we have not i nvol ved t he c omput e r i n t hi s met hodol ogi cal di scussi on. One mi ght
expect it t o pl ay a cent r al role. I n fact , comput er s have no rol e in t he theory, b u t pl ay
an i mpor t a nt practical role. Computation pl ays a cent r al rol e in t he t heor y, because t he
t act i cs are pr ocedur es and t he y ar e par t of t he t heor y of our sci ence of reasoni ng. I t is
not , st r i ct l y speaki ng, necessar y t o i mpl ement t hese t act i cs on a comput er , si nce t he y
can be execut ed by hand. However , in pr act i ce, i t is hi ghl y conveni ent . I t makes t he
process of checki ng t ha t t he t act i cs meet t he cr i t er i a of t he 7 b o t h mor e efficient and
less er r or pr one. Machi ne execut i on is conveni ent :
for s peedi ng up cor r ect ness t est i ng, especi al l y when t he pr oof pl ans ar e l ong, or
i nvol ve a l ot of search, or when a l arge col l ect i on of conj ect ur es is t o be t est ed;
t o a u t o ma t e t he gat her i ng of st at i st i cs, e.g. on size of sear ch space, execut i on
t i me, etc;
t o ens ur e t h a t a t act i c has been accur at el y execut ed; and
t o de mons t r a t e t o ot her r esear cher s t ha t t he checki ng has been done by a disin-
t er es t ed par t y.
I n t hi s way t he c omput e r can assi st t he r api d pr ot ot ypi ng and checki ng of hypot hes i s ed
pr oof pl ans. Fur t he r mor e , in i t s ' di si nt er est ed pa r t y' rol e, t he c omput e r act s as a
scept i cal col l eague, pr ovi di ng a second opi ni on on t he mer i t s of hypot hes i s ed pr oof pl ans
t h a t can s er ve as a sour ce of i nspi r at i on. Une xpe c t e d posi t i ve and negat i ve r esul t s can
cause one t o revi se ones cur r ent pr econcept i ons.
9 The Re l a t i o n t o Au t o ma t i c Th e o r e m Pr o v i ng
Al t hough o u r sci ence of r easoni ng mi ght fi nd appl i cat i on in t he bui l di ng of hi gh per f or -
mance, a ut oma t i c t he or e m pr over s, t he t wo act i vi t i es ar e not co-ext ensi ve. The y differ
bot h in t hei r mot i va t i on a nd t hei r met hodol ogy.
639
I take the conventional motivation of automatic theorem proving to be the building
of t heorem provers which are empirically successful, without any necessity t o understand
why. The methodology is implied by this motivation. The theorem prover is applied
to a random selection of theorems. Unsuccessful search spaces are studied in a shallow
way and crude heuristics are added which will prune losing branches and prefer winning
ones. This process is repeated until the law of diminishing returns makes further repe-
titions not wort h pursuing. The result is fast progress in the short term, but eventual
deadlock as different proofs pull the heuristics in different directions. This description
is something of a caricature. No ATP researchers embody it in its pure form, but aspects
of it can be found in the motivation and methodology of all of us, to a greater or lesser
extent.
Automatic t heorem provers based on proof plans make slower initial progress. Initial
proof plans have poor generality, and so few theorems can be proved. The motivation
of understanding proofs mitigates against crude, general heuristics with low prescrip-
tiveness and no expectancy. The ' accidental' proof of a theorem is interpreted as a fault
caused by low prescriptiveness, rather then a lucky break. However, there is no eventual
deadlock to block the indefinite improvement of the theorem prover' s performance. If
two or more proof plans fit a theorem then either they represent legitimate alternatives
bot h of which deserve at t empt i ng or they point to a lack of prescriptiveness in the
preconditions which further proof analysis should correct.
Thus, we expect a science of reasoning will help us build bet t er automatic theorem
proving programs in the long term, although probably not in the short term.
1 0 C o n c l u s i o n
In this paper we have proposed a methodology for reaching a multi-level understanding
of mathematical proofs as part of a science of reasoning. The theories of this science con-
sist of a collection of general-purpose proof plans, and the association of special-purpose
proof plans with particular proofs. Each proof plan consists of a tactic and a method
which partially specifies it. Special-purpose proof plans can be constructed by a process
of plan formation which entails reasoning with the methods of the general-purpose proof
plans. Ideas for new proof plans can be found by analysing mathematical proofs using
our intuitions about their structure and, possibly, psychological experiments on third
part y mathematicians. Initial proof plans are then designed which capture this struc-
ture. These initial proof plans are then refined to improve their expectancy, generality,
prescriptiveness, simplicity, efficiency and parsimony while retaining their correctness.
Scientific j udgement is used to find a balance between these sometimes opposing criteria.
Computers can be used as a workhorse, as a disinterested part y to check the criteria
and as a source of inspiration.
The design of general-purpose proof plans and their association with particular
proofs is an activity of scientific theory formation t hat can be judged by normal sci-
entific criteria. It requires deep analysis of mathematical proofs, rigour in the design
640
of t act i cs and their met hods, and j udgement in t he selection of t hose general-purpose
proof plans wi t h real staying power. Our science of reasoning is normative, empirical
and reflective. In these respect s it resembles ot her human sciences like Linguistics and
Logic. Indeed it includes part s of Logic as a sub-science.
P e r s o n a l No t e
For many years I have regarded myself as a researcher in aut omat i c t heorem proving.
However, by analysing t he met hodol ogy I have pursued in practice, I now realise t hat
my real mot i vat i on is t he building of a science of reasoning in t he form outlined above.
Now t hat I have identified, explicitly, t he science I have been implicitly engaged in for
t he l ast fifteen years, I intend t o pursue it wi t h renewed vigour. I invite you t o j oi n me.
Re f e r e n c e s
[Bledsoe et al 72]
[Bundy & Wel ham 81]
[Bundy 88]
[Bundy et al 88]
[Fnnt 73]
[Polya 65]
[Wos & McCune 88]
W. W. Bledsoe, R.S. Boyer, and W. H. Henneman. Comput er
proofs of limit theorems. Artificial Intelligence, 3:27-60, 1972.
A. Bundy and B. Welham. Using meta-level inference for selec-
tive application of multiple rewrite rules in algebraic manipula-
tion. Artificial Intelligence, 16(2):189--212, 1981. Also available
from Edi nburgh as Research Paper 121.
A. Bundy. The use of explicit plans t o guide i nduct i ve proofs.
In 9th Conference on Automated Deduction, pages 111-120,
Springer-Verlag, 1988. Longer version available from Edi nburgh
as Research Paper No. 349.
A. Bundy, F. van Harmelen, J. Hesketh, and A. Smaill. Ex-
periments with Proof Plans for Induction. Research Paper 413,
Dept . of Artificial Intelligence, Edi nburgh, 1988. To appear in
JAR.
B. V. Funt . A procedural approach to constructions in Euclidean
geometry. Unpubl i shed M.Sc. thesis, University of British
Columbia, Oct ober 1973.
G. Polya. Mathematical discovery. John Wiley & Sons, Inc,
1965. Two volumes.
L. Wos and W. McCune. Searching for fixed point eombina-
tots by using automated theorem proving: a preliminary report.
ANL-88-10, Argonne National Laborat ory, 1988.