Alhassanain(p) Network for Heritage and Islamic Thought

4.3.1 On the Inferential Roles of Evidence

I focus now on Twining's comment given above and his phrase: …"information has a potential role as relevant evidence if it tends to support or tends to negate, directly or indirectly, a hypothesis or probandum". Wigmore has given us a very useful classification of the roles evidence can play in the context of proof[153]. In discussing these evidential roles, Wigmore used terms that are encountered in adversarial processes in our system of laws. But I can make them quite general in their application to inferences in any context. I recall here Stephen Toulmin's comment on adversarial processes and the case we try to make from evidence in any context, and his saying that "logic is generalized jurisprudence"[154].

To make the following comments as general as I can I am going to assume just one person making an inference, namely you.  We don't care about the context or substance of this inference. You are trying to reach a conclusion concerning whether hypothesis H is true or not. We might suppose that you have proposed hypothesis H. The only thing we will assume is that you are open-minded and are willing to consider evidence bearing upon both of the hypotheses: H and not-H. The evidence you will encounter may be tangible or testimonial in form and you may receive it as a result of your own efforts as well as the efforts of others from whom you may request information. You will note in this example that several of the forms, mentioned in Figure 1, and combinations of evidence that I said were "substance-blind" will arise in this example. This will satisfy Poincaré's emphasis on the importance of classification in science.

Supporting Hypothesis H.

First suppose you believe that if event E occurred, it would be directly relevant but not conclusive evidence favouring H. You then find evidence E*, that this event E occurred. This evidence may be a testimonial assertion or an item of tangible evidence. In Wigmore's analysis this would be called aproponent's assertion , if we regarded you as the "proponent" of hypothesis H. You are concerned of course about the credibility of the source of E*. Suppose you obtain ancillary evidence favourable to the credibility of this source. This would enhance your belief that event E did occur. Suppose, in addition, you query another source who/that corroborates what the first source has said; this second source also provides evidence that event E occurred. You then gather ancillary evidence that happens to favour the credibility of this second source. But you also gather evidence in support of the generalization that licenses the relevance linkage between event E and hypothesis H. This evidence would strengthen this linkage.

But you know about other events, if they occurred, would also support hypothesis H. In particular, you believe that event F would converge with event E in favouring H. You gather evidence F*, that event F occurred. You might gather ancillary evidence favourable to the credibility of the source of F, and you might also gather corroborative evidence from another source that event F occurred. You might, in addition, gather further ancillary evidence to strengthen the generalization that licenses your inference from event F to hypothesis H. So, now you have two lines of evidence which, together, would increase the support for hypothesis H. You may of course know of think of other events that would converge in favouring H.

Wigmore was no probabilist, that would have been asking too much of him, given all of his other accomplishments regarding evidence and proof. Study of probabilistic matters concerning the inferential force of evidence suggests additional ways in which we you can use evidence to support hypothesis H. First, suppose that evidence E* is testimonial and came from a source named Mary. You begin to think that the fact that Mary told you that event E occurred means more than it would if someone else had told you about E. In fact, what you know about Mary's credibility makes her testimony more valuable than if you knew for sure that event E occurred. In short, what we know about a source of evidence can often be at least as valuable as what the source tells us. I have captured this subtlety associated with testimonial evidence mathematically in a recent report[155].

Another subtlety, this time involving events E and F. can be captured that would often greatly increase the support that evidence of these events could provide for hypothesis H. Earlier in Section 4.1.2 I mentioned how items of convergent evidence, favouring the same hypothesis, can often be synergistic in their inferential force. You might believe that evidence of events E and F, taken together, would favour hypothesis H much more strongly than they would if you considered them separately or independently. This synergism can be captured in probabilistic terms[156].

These are all roles that evidence can play in support of some hypothesis. But we agreed that you are open-minded and will carefully consider counterevidence that would negate H or would favour not-H.

Negating Hypothesis H

A colleague appears who has never shown any great enthusiasm for a belief that hypothesis H is true. So far you have said that events E and F would favour H. But your colleague challenges the credibility of the sources of evidence about these events by first producing ancillary evidence that disfavours their credibility. In addition, your colleague might produce contradictory evidence from sources who/that will say that either or both of the events E and F did not occur. Wigmore referred to this evidence as constitutingopponent's denial that events E and F occurred.

But your colleague might instead, or in addition, have ancillary evidence that weakens the generalizations you have asserted that link events E and F to hypothesis H. Such evidence would tend toexplain away what you have said was the significance of events E and F. This ancillary evidence would allow your colleague to say: "So what if events E and F did occur, they have little if any bearing on hypothesis H". Wigmore termed this situationopponent's explanation .

Your colleague has yet another strategy for negating H. She might say the following: "So far you have only considered events [E and F] that you say would favour H. Are you only going to consider events you believe favour H? I have gathered evidence J* and K* about events J and K that I believe disfavours H. Wigmore termed this situationopponent's rival evidence . In this situation you would have to cope with what I termed divergent evidence. There is no contradiction here since J and K involve different events and might have occurred together with events E and F. Events J and K simply point in a different F.

inferential direction than do events E and F.

But your colleague has a final approach to undermining an inference that H is true; it involves what you might have said about the synergism involving events E and F. Your colleague says: "You have said that evidence about events E and F taken together have much more force than the would have if we considered the separately. In other words you are   saying that event F has more force in light of the occurrence of E than it would have if we did not consider E. But I argue that the occurrence of E would make F redundant to some degree and so I argue that they mean less when taken together than they would do when considered separately.

In providing this illustration of the various roles evidence plays, it was not actually necessary for me to suppose that you experienced a colleague who would use these evidential strategies for undermining a belief in H. If you were indeed open minded in your inferential approach, you would have played your own adversary by considering how evidence favourable to H could be attacked or countered in these various ways by other evidence. Your conclusion may well have been that not-H is true in spite of your initial belief that H was true. What this example illustrates is the necessity for us to be unbiased or objective in the gathering and evaluation of evidence in the inferences we make from evidence. We might say that we are well served when we play the role of our own "loyal opposition" in the inference tasks we face. This is true even if we often face the often "not so loyal opposition" from our critics.

4.3.2 Stories from Evidence and Numbers

I begin by acknowledging the many studies currently being undertaken in which masses of evidence and complex processes are being analyzed in probabilistic ways in a variety of important contexts including law, medicine and intelligence analysis. Consider Figure 2 again that shows two very simple illustrations of the different forms of inference networks that have been analysed probabilistically. I will use both the Wigmore analysis of a mass of evidence and a certain process model to illustrate how it is possible, and usually necessary, to construct alternative stories that might be told about the inferential force of a mass of evidence. My examples will involve the use of Bayes' rule, but similar analyses can be performed using Shafer's belief functions or Cohen's Baconian probabilities.

Both of the examples I will discuss involve what is termed "task decomposition" or "divide and conquer". In such decompositions, an obviously complex inference task is broken down into what are believed to be its basic elements. Wigmore's analytic and synthetic methods of analyzing a mass of evidence is a very good example. We first list all of the evidence and sources of doubt we believe appear in arguments from evidence to what we are trying to prove or disprove from it [a key list], and then construct a chart [or inference network] showing how we believe all of these pieces fit together. As I have mentioned, we can describe this process as one of trying to "connect the dots".

Suppose we have a mass of evidence in our analysis and an inference network based on this evidence that has survived a critical analysis designed to uncover any disconnects or non sequiturs in the arguments we have constructed. The next step is to assign probabilities that will indicate the strength with which the probanda, sources of doubt, or probabilistic variables are linked together. These probabilities come in the form of the likelihoods I described in Figure 5. All the arrows in the two diagrams in Figure 2 indicate these probabilistic linkages expressed in terms of likelihoods. Let us suppose that we all agree that the inference network we have constructed captures the complex arguments or elements of the process we are studying. But where do these linkage probabilities come from? In some very rare instances we might have a statistical basis for estimating these probabilities from relative frequencies. But in most instances many or most of these probabilities will rest on epistemic judgments we make. In the two examples I will provide,all of the probabilities rest on subjective judgments. Here is where the necessity for telling alternative stories arises.

Though we agree about the structure of our inference network we may find ourselves in substantial disagreement about the likelihoods linking elements of our argument together. Suppose we are interested in determining the overall force or weight of the evidence we are considering. How do our differences in these likelihood ingredients affect the force of the evidence we are considering? We might say that our different beliefs about these probabilistic ingredients allow us to tell different "stories" about the force of the evidence we are considering. The "actors" in our stories consist of the items of evidence we have. The "plots" of our stories are provided by the likelihoods we have assessed. When your likelihoods are different from mine we are essentially telling possibly different stories based on the same evidence, or involving the same actors. We are of course interested in the extent to which our stories end in telling us about the force of the evidence we have. How your story ends may be quite different from the ending of my story, but not necessarily. The metaphor of telling stories from evidence is certainly appropriate. It describes a process that is repeated every day involving the different stories told from the same evidence by opposing attorneys in trials at law. They use the same actors to tell different stories.

How do we tell how our different stories about the force of evidence will end? This is where mathematics comes to our assistance. It involves the process of what is termedsensitivity analysis . We have equations stemming from Bayes' rule that tell us how to combine your likelihoods and my likelihoods in calculations of the force of all the evidence we are considering.  In short, these equations supply endings to your story and my story about the aggregate force of the evidence. Here comes Carnap's comparison and numerical concepts again in a science of evidence. Our stories are told numerically but they can easily be translated into words. Then we can compare our stories to observe the extent to which our differing likelihood ingredients have affected determinations of the force of the evidence.

But there is an important characteristic of the equations we use to combine these  likelihood ingredients; they are allnon-linear . What this means is that these equations can produce many "surprises" that would never result from linear equations in which "the whole is always equal to the sum of its parts". What will happen is that, on some occasions, the fact that our assessed likelihoods are quite different makes little difference in the ending of our two stories; we are telling two stories that have the same or nearly the same ending. But on other occasions even exquisitely small differences in our likelihood ingredients will produce drastic differences in our stories about the force of our evidence.

Jay Kadane and I used the process of sensitivity analysis just described in our probabilistic analysis of parts of the evidence in the case of Sacco and Vanzetti. We used this process to tell different stories on behalf of the prosecution and of the defence in this case[157].  As I mentioned above, we used the same evidence in telling these stories on behalf of the parties involved in this trail. Here are two examples of the kinds of stories we told and how we told them.

The first involved what Wigmore termedconcomitant evidence, that involved what Sacco was doing at the time of the crime. Two of the witnesses I have already mentioned, Lewis Pelser and Lewis Wade. They were the prosecution's "star" witnesses. Recall that Pelser said he saw Sacco at the scene of the crime when it occurred, and Wade said he saw someone who looked like Sacco at the scene of the crime when it occurred. But there were five defence witnesses, four of whom were just across then street from where the payroll guard Berardeiil was shot. They all said the neither Sacco nor Vanzetti was at this scene when the shooting occurred. A fifth defence witness testified that Sacco was not at the scene about 15 minutes before the crime occurred.

The Wigmorean argument structure for the evidence just described is simple enough so that were able to write out the exact equations necessary for combining the likelihoods in this argument structure. Using these equations we told ten different stories, five on behalf of the prosecution and five on behalf the defence[158]. In the process of telling these ten stories we varied the value of likelihoods such as those that concern the credibility of the witnesses, and other matters such as those concerning the probability of someone looking like Sacco being at the scene, given that Sacco was not there. The ten stories we told all have different endings as far as the weight of this evidence favouring either the hypothesis that Sacco did shoot the payroll guard or the hypothesis that Sacco did not shoot the payroll guard. As I noted above, these stories often have surprising endings. For example, by varying Pelser's credibility by apparently just a small amount we can effectively destroy the contribution of his testimony to the weight of the combination of evidence being considered. Some of the stories we told illustrated how Bayes'  rule captures the possible redundance of Pelser's and Wade's testimonies.

We also considered much more complex aggregates of the evidence in this case[159]. One combination of evidence concerned Bullet III I mentioned earlier and the Colt automatic alleged to be the one Sacco had in his possession when he was arrested. The argument structure here is sufficiently complex that it prevents us from writing out equations for the force of evidence. The inference network we constructed for this ballistics evidence requires 42 assessments of likelihoods. But thanks to the truly outstanding works of several probabiliists, we now have computer-based systems that "know" what the necessary equations should be for any appropriate argument structure [a DAG] we can construct, whether it is a Wigmore evidence chart or a process model such as the ones I illustrated in Figure 2[160]. The stories we told on behalf of the prosecution and defence were designed to capture important conditional dependencies among probanda or propositions appearing on our inference network. For example, evidence that Bullet III was fired through Sacco's Colt points toward Sacco's guilt, but not very strongly. Someone else may have fired this weapon during the crime. But this evidence means more when we take into account other evidence that Sacco fired a weapon at the scene of the crime when it  was committed. The credibility of all of this evidence is so important.

We were pleased to note that the stories we told from numbers in the Sacco and Vanzetti case were included in a work by John Allen Paulos, a mathematician whose books enjoy a very wide following. In a recent book on what he terms the hidden mathematical logic behind stories[161], he describes the usefulness of the stories we told from numbers in the Sacco and Vanzetti case and argues that the Wigmorean analysis underlying these stories would have been useful in other complex cases such as the trial of O. J. Simpson.

I make one final point about telling stories from numbers based on inference networks constructed from evidence. I have said nothing so far about the role ofexperiments in a science of evidence. The process of sensitivity analysis is a form of experimentation in which we vary the probabilistic ingredients of equations based on a given inference network. We do so in order to see how the equations will behave [i.e what different stories they will tell] in response to these changes in their ingredients. But do all of the stories told based on an inference network make sense?  Sensitivity analysis is also a process for testing the inference network itself. As I have mentioned, inference networks we construct are products of our imaginative and critical reasoning. How do we test to see if the network we have constructed makes sense in allowing us to draw conclusions of interest to us? Sensitivity analysis allows one kind of test: Does our network allow us to tell stories that make sense when we vary the ingredients of these stories in more or less systematic ways? This form of experimentation is one I have used for years in testing the inferential consequences of arguments I have constructed from a wide variety of forms and combinations of evidence[162]. A science of evidence does allow experimental tests of our views about how we believe evidence to be related to matters we are trying to prove or disprove.

4.4. Discovery in the Science of Evidence

The third OED definition of science, that I mentioned in my opening remarks in Section 4.0, is restrictive in the sense that it makes necessary for a science to have "reliable methods for the discovery of new truths in its own domain". I have two responses here, the first concerns how we extend our knowledgeabout evidence and its properties and uses; this involves how we learn more about evidence itself. The second concerns how a science of evidence can provide methods for enhancing discovery-related activities in any discipline or context in which this science may be applied. So in this sense we have both basic and  applied interests to consider as far as concerns discovery in the science of evidence.

4.4.1 Discoveries about Evidence

In Section 3.0 I considered how the concepts ofevidence and ofscience have emerged and changed over the ages. At no point did I argue that these two concepts are now fixed for all time. Thus, a science of evidence is in the process of emergence in which we hope to learn more about evidence as we continue to study it. For example, I have offered a categorization of recurrent forms and combinations of evidence that I advertised as being "substance-blind" only in the sense that it tells us what kinds of evidence we have. But I never advertised that my classification could not be improved or that other useful classifications could never be made. You might, for example, think of a type of evidence that I have left out of the rows in Figure 1. If so, please tell me about it.

What we wish to avoid are problems the philosopher Imre Lakatos identified in connection with what he calledresearch programs [163]. He called research programsprogressive to the extent that they continue to have heuristic power in generating new facts about the phenomena of interest. If they fail to do this, and just continue to account for already observed facts, Lakatos called themregressive research programs. Given the richness of the contexts in which evidence is used and studied, we run little risk of a science of evidence becoming regressive any time soon, provided that we share our experiences and thoughts about evidence with each other. Phil Dawid's emphasis in his proposal to the Leverhulme Foundation emphasized anintegrated science of evidence. I have always interpreted an integrated science of evidence to involve the necessity of identifying and exploiting our varied experiences and thoughts about evidence. I will give just two examples of how such sharing has benefited me in my work on evidence. I return to the integrated nature of a science of evidence in Section 5.0.

I have always marvelled that there are so many important evidential and inferential subtleties or complexities that often lie just below the surface of even the "simplest" of evidence-based reasoning tasks [if there are any such things as "simple" inferential tasks]. I spent many years of my work on evidence trying to identify and understand these subtleties so that they could be exploited in the conclusions we must draw from evidence. These subtleties involve any or all of the evidence credentials I have mentioned: relevance, credibility and inferential force or weight. Many years ago I recognised that nearly every law case I read revealed some new subtlety I should examine. Many of those I have studied were first revealed to me in law cases, some that took place centuries ago.

But I have often observed subtleties revealed in the works of others interested in the study of various kinds of evidence. As an example, I now refer to the work of C. A. J. Coady, a philosopher interested in the study of testimony[164]. As expected, Coady carefully examines the epistemological foundations for testimonial evidence. But one chapter of his work was especially interesting to me, his Chapter 10  entitled;Astonishing Reports . This chapter concerns testimonial evidence of events either contrary to the laws of nature, or testimony about unusual events that would conform to these laws. The study of testimony about miracles has occupied the attention of probabilists and others for centuries[165]. And it is still a still a topic of research today[166]. I hope to give further study to these topics in future. In my probabilistic studies there are very interesting terms appearling that concern the rareness or improbability of the events reported in testimony. Since the 1600s it has been expected that the rareness of the event reported, in addition to the credibility of the source reporting the event, are factors that determine the force of testimonial evidence about this event. But the relationship between these ingredients was never made clear in any studies I could find. I am just vain enough to conclude that my analysis of this rareness-credibility problem makes the nature of this relationship clear. It turns out that I was "murdered" for doing so[167].

4.4.2 Evidence Science and the Discovery of New Evidence

I understand that the discovery of new evidence depends on the nature of the investigations that take place in the substantive area of concern. The evidence we generate or discover in any context depends upon the questions we ask. People in different contexts will naturally ask different questions. Thus, a historian, a sociologist, an accident investigator, and a physician will ask different questions and will generate the different kinds of evidence of interest to them. The path to expertise in any area depends in no small part on knowing what kinds of questions to ask and how answers to them might be obtained. It is also true that persons having different standpoints will ask different questions in the same context. Suppose a police officer, a news reporter, and an attorney all arrive at the scene involving an injured or a dead person. The police officer will ask whether a crime has been committed. The reporter wil ask whether there is a story here. And the attorney will ask whether there is a  civil complaint or a criminal charge that can be levied in a legal action of some sort.

But it is also true in so many situations involving discovery or investigative efforts that we have hypotheses in search of evidence at the same time we have potential evidence in search of hypotheses. It turns out that we can generate new evidence in both of these situations. I have already noted that from hypotheses we generate new evidence and new lines of reasoning. But new evidence can be generated by several other ways as I now illustrate.

Bayes' Rule and the Force of Evidence

Reasoning from evidence is a dynamic process in which we revise our existing or prior beliefs about hypotheses or propositions on the basis of relevant evidence to form new or posterior beliefs about these hypotheses. There is a consequence called Bayes' rule that tells us how this dynamic process should occur when our beliefs are expressed probabilistically in accordance with three basic rules most of us learn in school. These rules are:

Probabilities are positive numbers or zero [i.e. there are no negative probabilities]

The probability of a "sure" event [one certain to happen] is 1.0,

If two or more events cannot happen together [i.e. they are mutually exclusive], the probability that one or the other of these events occurs is equal to the sum of their separate probabilities.

All probabilities are dependent, or conditional upon, what else we know or find out. Here is an event E whose probability we are interested in determining. But we also learn that event F has occurred; so we have an interest in determining the probability of E, given or conditional on F. Conditional probabilities obey the same three rules given above. There is a consequence of rules for conditional probabilities, called Bayes' rule, that tells us how much, and in what direction, we should revise orprior beliefs about some hypothesis based on new evidence we obtain. The result of this determination is what is called aposterior probability . This rule originated in the work of Thomas Bayes [1702 - 1761] a dissenting clergyman who lived in Tunbridge Wells. Those interested in finding more about Bayes can consult a recent biography of him[136].  Bayes' rule as been called the first mathematical canon for inductive reasoning.

Now, there are terms in Bayes' rule calledlikelihoods that wlll tell us by how much, and in what direction, we should revise our prior beliefs into posterior beliefs, given some new evidence.  We often consider ratios of these likelihoods. In any case, these terms are indications of the inferential force of the evidence we have obtained. Figure 5 is a picture of what these likelihoods express. Suppose defendant Dick is on trial for shooting victim Vick. Here is an item of evidence received during Dick's trial: Dick owned the revolver used in the shooting of Vick.

The force of this evidence on whether Dick shot Vick is given by the ratio of Likelihood 1 to Likelihood 2. If you believe Likelihood 1 is greater than Likelihood 2, you are saying that this evidence favours the proposition that Dick shot Vick by an amount indicated by the size of this ratio. Likelihood ratios express both the inferential force and direction of evidence. If you said that this ratio was 5, you are saying that this evidence is five times more likely if Dick shot Vick than if he didn't shoot Vick. Directionally, this evidence points to Dick shooting Vick. Bayes' rule would say that you are entitled to increase your prior belief that Dick shot Vick by a factor of 5. Thus, in accordance with FRE 401, discussed above concerning relevance, this evidence would indeed be relevant since it allowed you to change your belief in the probability of a material or consequential issue. This is what Richard Lempert observed to be a virtue of the way Bayes' rule grades the inferential force of evidence[137].

For many years now I have studied likelihood ratio formulations for the force of every form and combination of evidence I could think of. I have reviewed many of these studies in another work[138]. Bayes' rule is a marvellous device for capturing a very wide array of evidential subtleties or complexities for study and analysis. Likelihood ratios can be expressed for collections of evidence and not only for individual items as shown in Figure 5. Bayes' rule incorporates a property calledconditional dependence that is the finest property I know of for capturing evidential subtleties or complexities. I will return to Bayes' rule again when I discuss the discovery of evidence. We normally view probability theories as being involved just in the inductive justification of hypotheses. But application of this rule can prompt us to ask questions we may never have thought of asking. These questions may open up new lines of inquiry and new lines of evidence. But as rich as it is, Bayes' rule does not have all there is to say about the force, weight or strength of evidence.

Evidential Support and Evidential Weight: Nonadditive Beliefs

The formal system leading to Bayes' rule rests on axioms taken by many to be self-evident. The person who first formed the three axioms I mentioned above was the Russian mathematician A. N. Kolmogorov[139]. In his works Kolmogorov makes clear that his axioms assume situations involving replicable events in which probabilities can be determined by counting. The two basic examples arealeatory probabilities in games of chance andrelative frequencies in statistics. But there are many situations in which we have doubt or uncertainty about events that are the result of processes that cannot be repeated and in which no counting is possible. I refer here to unique, singular, or one-of-a-kind events. These situations are very common in a variety of contexts such as history, law, intelligence analysis, and in everyday experience. I have my own belief about the probability that Nicola Sacco was guilty of killing Berardelli, but I cannot play the world over again 1000 times to observe the number of occasions on which he did it. Various attempts have been made to apply probabilistic concepts in these non-enumerative situations.

Many persons take the view that probabilities can be epistemic, subjective, or judgmental in form and rest on whatever information we happen to have that we believe to be  relevant to an assessment of the probability of interest. Applications of Bayes' rule requires at least one epistemic probability; we need an initial prior probability in order to get the dynamic probability revision process started. Regarding some hypothesis or proposition H, we need to assess how likely is H before we begin to gather relevant evidence. Many persons have no hesitation in supplying epistemic judgments of prior probabilities and other ingredients of Bayes' rule, including likelihoods, provided that these probabilities conform to the Kolmogorov axioms. As I mentioned at the close of Section 3.2, this is what led Professor Mario Bunge to refer to colleagues as "charlatans", engaged in "pseudoscience", who are willing to assess the prior probability ingredients of Bayes rule in the form of epistemic or subjective judgements. In short, Bunge and others rejectany view of probability as making sense when we have nothing to count. Bunge would really come unstuck if he read the works of the person whose views of the force of evidence I now mention.

Professor Glenn Shafer [Rutgers University]  has given very careful thought to epistemic or judgmental probabilities necessary in situations in which we have nothing to count[140]. He begins by denying the self-evident nature of the third of Kolmogorov's axioms; it is called theadditivity axiom . Recall that this axiom says that if event E and F cannot occur together, then the probability that one or the other occurs is always equal to the sum of their separate probabilities. But there is an added consequence of this axiom. When we have mutually exclusive events that are also exhaustive [one or the other must occur] then the sum of their probabilities is 1.0, in accordance with the second axiom for "sure" events. Thus, if we have two hypotheses H and not-H, their probabilities must sum to 1.0, a priori, or a posteriori, given any evidence. In short, if you believe the probability of H is p, you must also believe the probability of not-H is (1 - p). Thus, if you increase the probability if H, you must decease the probability of not-H. Shafer says this is an unfortunate property in many situations involving epistemic probability judgments. He offers several reasons why this additivity property causes trouble.

Shafer is well aware of some important ideas that have been around for a long time, such as Jakob Bernoulli's distinction between  "mixed" and "pure" evidence that he described in his treatise:Ars Conjectandi in 1713. Mixed evidence offers some degree of support to every hypothesis being considered. But pure evidence says nothing about certain hypotheses and offers them no support at all. As an example of pure evidence, suppose that Tom, Dick and Harry are suspects in the theft of a valuable object from the home of the owner.  There were no signs that the house had been broken into. Tom is found with a key to this house. This would be pure evidence since it offers support for Tom's having stolen the object; but it says nothing about Dick or Harry.

Shafer says we need a different measure of the support that evidence may provide hypotheses. So, he defines a measure ofevidential support, S, which he equates to theinferential weight of evidence. Like ordinary probabilities, 0 ≤ S ≤ 1.0. But S has a different meaning than do the likelihoods discussed above that indicate the force of evidence in Bayes' rule. When S = 0, all this means is that evidence provides no support to some hypotheses. But when a likelihood has zero value this means that this hypothesis is impossible. It says the the probability of the evidence we have is zero, given this hypothesis. Bayes' rule then assigns zero probability to this hypothesis. There is an entirely different meaning of the role of zero in ordinary probability and Shafer's S scale. On the ordinary probability scale, zero indicates impossibility ordisbelief , On the S scale, S = 0 meanslack of support or lack of belief . Our belief in hypothesis H can be revised away from zero when we do have evidence that supports it to some degree. But we cannot revise the probability of a hypothesis away from zero that has been determined to be impossible. Disbelief and lack of belief are different judgmental conditions.

There is a very important consequence associated with the manner in which Shafer support S is assigned. We are allowed towithhold support assigned to hypotheses in various ways when we cannot decide what the evidence means. As I will illustrate in an example, this characteristic of Shafer's system leads to conditions in which our beliefs are nonadditive, as they must be using Bayes' rule. Here is how S is assigned.  Suppose we have some number n of hypotheses that are disjoint or mutually exclusive, but they are not necessarily exhaustive. We might think of others later on, or revise the ones we are considering at the moment. All the set of n hypotheses represents is how we see or inferential situation at the moment. Shafer call this collection of n hypotheses ourframe of discernment , F. We do not assign S to just these n hypotheses by themselves, as we must do in assigning likelihoods in Bayes' rule, but we assign S to subsets of these hypotheses in our frame F. When there are n hypotheses in F, then there are 2n possible subsets of hypotheses in our frame. The set of all 2n hypotheses is called apower set . Here is the simplest case in which we have two mutually excusive hypotheses that are also exhaustive, H and not-H. The power set of hypotheses in our frame F consists of: {H}, {not-H}, {H, not-H} and Ø, where Ø is the set of none of them [Ø is called the "empty set"]. Also, read the set {H, not-H} as: "either H or not-H". We are allowed to assign S in any way we please across the non-empty subsets of a power set, except that they must sum to 1.0 and with the additional provision that Ø always gets S = 0.

Here is an example of support assignment that involves a very good instance in which being indecisive about what evidence means, and being able to reflect our indecision in our beliefs, is a major virtue of Shafer's evidential reasoning system. This example involves William Twining's favorite law case:Rex v. Bywaters and Thompson , that was tried at the Old Bailey, on December 6 -11, 1922[141]. Edith Thompson was charged with either conspiring with Freddy Bywaters to kill her husband Percy Thompson on the particular occasion when he did it [October 3, 1922], or she incited Freddy to kill Percy whenever an occasion presented itself.  A classic love triangle appears in this case. Freddy boarded in the Thompson's home; but he was frequently away; he worked aboard ships. Freddy and Edith became lovers and carried on their affair until the time of Percy's death. Edith and Freddy corresponded daily when Freddy was away, either through the mails or by what were then called Marconigrams. Freddy kept all of the correspondence he received from Edith, but Edith kept none of those she received from Freddy.

I have never encountered finer examples of ambiguous evidence than the letters Edith wrote to Freddy. What is clear is that these letters appear to have convinced the twelve male jurors of her guilt. She was hanged on January 9, 1923 at Holloway; Freddy was hanged the same day at Pentonville. Some of these letters mention poisons of various sorts, some mention broken glass, others contain comments suggesting that Edith had tried to kill Percy herself. Other letters seem to give the impression that Edith and Freddy had made plans to do away with Percy. But the Shakespearean scholar Professor Rene Weis [also at UCL] puts a different interpretation on her letters that he provides in a very careful analysis of Edith's case[142].  Twining and Weis agree that Edith was innocent but do so from different standpoints and using different methods[143]. Twining uses this case to give examples of the truly complex situations in which Wigmore's argument structuring methods can be employed.

Using Shafer's method for assigning evidential support, or weight, here is how I view Edith's letters as supporting her being guilty, G , or not guilty, not-G, as she was charged. Let SL represent the support I have assigned to the entire collection of her letters to Freddy. The power set of these hypotheses is: {G}, {not-G}. {G, not-G] and Ø

{G}            {not-G}                   {G, not-G}                Ø

   SL:     0.3          0.2                  0.4              0

Here is what my S assignment means. I think the letter evidence supports her guilt to degree 0.3, and her being not guilty to degree 0.2. But I am undecided to degree 0.4 about what this letter evidence says, and so I assign this amount to the set {G, not-G} because I cannot tell whether this ambiguous evidence specifically supports G or not-G. This setting of S represents the amount of support I have withheld from either {G} or {not-G}.

The above assignment of support corresponds to my beliefs [Bel] in a way that Shafers system allows I have Bel{G} = 0.3 and Bel{not-G} = 0.2. My beliefs in this case are non-additive since Bel{H} = 0.3 + Bel{not-G} = 0.2 = 0.5, which is less than 1.0. If I had used a Bayesian approach, I would be required to say that Bel{H}  + Bel{not-G} = 1.0 since G and not-G are mutually exclusive and exhaustive. In short, Bayes' rule does not allow me to be indecisive about what I think the evidence means.

Shafer's system, often called a system ofbelief functions , is very useful in capturing elements of our probabilistic beliefs that are difficult, or impossible to capture with ordinary probabilities. Because one of the Kolmogorov axioms is violated, Bayes' rule does not appear is Shafer's belief function system. It is replaced by what is calledDempster's rule , which allows us to combine support assessments S for successions of evidence. This rule allows to calculate what is called theorthogonal sum of S assignments for different items of evidence. This system has found application in a number of important contexts in which epistemic judgmental assessments are necessary in the reasoning tasks at hand.

Evidential Completeness and the Weight of Evidence in Baconian Probability

Francis Bacon [1561 - 1626] is usually credited with being the first to argue that we can never justify hypotheses about how nature works just by compiling instances that are favourable to them. What he argued was that negative instances are at least as informative as positive instances. In fact, what we should do in testing hypotheses is to perform experiments designed toeliminate possible hypotheses. The one or ones that resist our best efforts to eliminate any of the hypotheses we are considering are the ones in which we should have the most confidence. This view has been calledeliminative induction . But Bacon was never specific about what eliminative methods could be employed. As I noted in Section 3.1, John Stuart Mill is usually credited with being the first to identify methods designed to eliminate possible causes for the effects we observe in nature. But such methods were known much earlier to the four Oxford scholars I mentioned.

But there is another important element in the eliminative testing of hypotheses. The tests we perform must bevariative in the sense that we must establish the array of conditions under which we may expect a hypothesis to continue to remain valid. We cannot do this by performing the same test over and over again. The only thing this repetitive testing would accomplish is to increase our confidence in the reliability of this single test's results.  The more varied are the conditions under which some hypothesis holds, the more confidence we can place in it. But this variative testing raises another important question, namely:how complete has been our eliminative testing of our hypotheses? There may be other important tests of our hypotheses that we have not performed whose results might serve to eliminate hypotheses we are still considering.

Neither Bacon, Mill, Popper, nor anyone else was successful in relating problems associated with the eliminative and variative testing of hypotheses to ordinary probabilistic concepts. The first person to study this relation was L. Jonathan Cohen [now emeritus, Queen's College, Oxford]. In a work that had a great influence on probabilistic thinking in law and philosophy, Cohen was the first to generate a theory of probability expressly congenial to the eliminative and variative testing of hypotheses[144]. He refers to this theory asBaconian probability to acknowledge its roots in the works of Francis Bacon. On occasion, he also calls it a theory ofinductive probability . In his works Cohen takes a decidedly ecumenical [or "polycriterial", as he calls it] view of probability in evidence-based reasoning. He allows that conventional views of probability make perfect sense in some but not all situations. He further argues that conventional views of evidence-based reasoning, such as Bayes' rule, overlookhow much evidence has been considered and how complete is this evidential coverage of matters believed to be relevant in the inference at hand. Eliminative and variative inference requires special considerations. In fact,evidential completeness , in Cohen's view, is the major factor associated with the weight of evidence.

In Figure 6 below is a diagram I have used to illustrate some of Cohen's key ideas in Baconian probability. I have tried my best to generate interest in the importance of Cohen's views among persons in a variety of contexts who should be aware of his ideas regarding evidential completeness. I have gone to great lengths in some contexts, but not always with any great success[145]. There are two basic questions that arise in Cohen's views about the weight of evidence: (i) How much uncounteracted favourable evidence do we have on some hypothesis that has arisen in answer to relevant questions we have asked?, and (ii) How many relevant questions, that we know about, remain for which we have no evidential answers? In short, the weight of evidence in Cohen's view depends not only on answers to questions we have asked, but also upon how many questions remain unanswered. Cohen's Baconian views about the weight and amount of evidence bring to mind ideas expressed by John M. Keynes in his very influential treatise on probability[146]. Keynes's ideas about the amount and the weight of evidence have often been misunderstood.  Cohen has written on various questions that have arisen regarding the views of Keynes on the weight of evidence[147].

Here are some details of the cover story surrounding Figure 6. Some time ago we were asked to assess which of three hypotheses, H1, H2 and H3, is most likely because it will have an important bearing on a decision we must make.  Initial evidence pointed very strongly to H1 being true, so we took an action based on H1.  What we are now doing is engaging in a post mortem analysis trying to see what went wrong; H3 happened to occur and our decision miscarried. Our decision produced a disastrous result. Someone says: "How could we have gone wrong? We used Bayes' rule to aggregate what our assessments of likelihoods for the evidence we had, and we all agreed that the prior probabilities we were using made perfect sense. Bayes' rule said that the posterior probability of H1 was 0.997 based on the evidence we incorporated in our inference".

If Jonathan Cohen happened to be present during our post mortem, here is what he might have said: "You were out on an inferential limb that was much longer and more slender than you believed it to be, just based on the answers your existing evidence provided. How many relevant questions do you now realise to have been unanswered in your analysis?" We begin to make a list of questions we believe also relevant that we did not attempt to answer; this list grows quite large. It also contains questions we knew about at the time of our analysis. However, we believed the evidence we did take account of was sufficiently strong that we did not hesitate to conclude that H1 was true. Here is a picture of the actual inferential limb we were on.

Jonathan Cohen goes on to explain the two parts of this inferential limb on which we found ourselves. He says: "The strong part consists of the evidence you had that was favourably relevant to H1. The weak part consists of relevant questions that remained unanswered. What you did in concluding that H1 was true was to assume essentially that the answers to all of the questions that you did not ask would have been favourable to H1. The problem is that a very high Bayesian posterior probability is not a good indicator of the weight of evidence because it does not grade the completeness or sufficiency of evidence".

In another work I have compared Baconian and Bayesian approaches when we encounter chains of reasoning in arguments we construct[148]. There is nothing incompatible about these two  approaches to evidence based reasoning. The reason is that they each respond to different, but equally important, considerations. Bayes' rule provides very useful measures of how strong is the evidence you do have, but Cohen's Baconian probabilities allow us to grade the completeness of our evidence. I ended up concluding that both forms of hedging conclusions would be necessary on many occasions.

Verbal Assessments of the Force of Evidence: Fuzzy Probabilities

In so many situations we talk about the force of evidence, and express the strength of our conclusions, in words rather than in numbers. There are no better examples than those occurring in the field of law. Forensic standards of proof such as, 'beyond reasonable doubt", "clear and convincing evidence", "probable cause", and so on, are verbal assessments that seem to defy efforts to translate them into numerical probabilities. In his analysis of what we now call inference networks, Wigmore understood perfectly well that the arrows linking evidence and probanda, such as those illustrated in Figure 2A, are probabilistic in nature. But he always used words rather than numbers to indicate the force with which one element of an argument is linked to others[149].  He used terms such as "strong force", "weak force", and "provisional force" to indicate the strength of these linkages. The use of words rather than numbers to indicate the force of evidence appears in many other contexts, especially when there is no attempt to employ and combine any of the views of evidential force described above.

There are algorithms for combining numerical probabilities, such as Bayes' rule and Dempster's rule, but how do we combine assessments of the force of evidence that are given in words? Wigmore gave no hint about how we should combine his verbal assignments of evidential force in order to grade the force of an overall mass of evidence. Verbal assessments of probabilities in grading the force of evidence, and in stating the strength of an overall conclusion, are today referred to asfuzzy probabilities , in part to acknowledge their imprecision. But thanks to the work of Lotfi Zadeh and his many colleagues worldwide, there is a logic that underlies the expression and combination of verbal or fuzzy probabilities[150]. This system of fuzzy logic and probabilities has found wide acceptance in many situations in which persons must perform a variety of tasks based on fuzzy or imprecise ingredients. But it does have its detractors[151].

I have now completed my comments on the essential properties or credentials of evidence: relevance, credibility, and inferential force, weight, or strength. I have taken some care in discussing these properties in order to illustrate how study of them involves the classificatory, comparative, and quantitative concepts that both Poincaré and Carnap said were involved in science. I next comment on the uses of evidence and will show how these same concepts arise.

4.3 On the Uses of Evidence

We all use evidence every day of our lives in connection with our inferences and decisions, whatever their substance and objectives might be. William Twining has provided a characterization of evidence that seems to cover the use of evidence in any context you can think of. He says[152]:

'Evidence' is a word of relation used in the context of argumentation (A is evidence of B). In that context information has a potential role as relevant evidence if it tends to support or tends to negate, directly or indirectly, a hypothesis or probandum. One draws inferences from evidence in order to prove or disprove a hypothesis or probandum. The framework is argument, the process is proof, the engine is inferential reasoning.

I am going to provide two examples of uses of evidence. The first will illustrate Poincaré's assertion that science relies upon classifications and is the study of relations, some of which can be expressed in quantitative terms. The second involves Carnap's comparative and quantitative concepts and their importance in science and in our everyday lives.