Something in the air or a simple twist of fate?
This essay was written by Stephen R. Martin and Maria J. Schilstra and was first published in the 2008 Mill Hill Essays. An updated version was published in the Mill Hill Essays anthology.
The fathers of two young cancer sufferers meet in a waiting room at their local hospital. During the ensuing conversation they discover not only that they are near neighbours, but also that between them they know of three other local children who have been diagnosed with the same disease. Surely this is a suspiciously high number of cases to be found in such a small area. Or is it?
The presence, either real or perceived, of a larger than expected number of cases of any specific disease within a particular group of people, or within a defined geographical area, is known as a disease cluster. All types of disease clusters, including suspected cancer clusters, are investigated by epidemiologists. These are scientists who study the causes, the distribution, and the frequency of occurrence of diseases within populations. Several major breakthroughs in the control of infectious diseases have resulted from epidemiological studies of clusters of disease cases. One of the first, and probably the best-known, epidemiological study was undertaken by Dr John Snow, who analysed the distribution of victims of the London cholera outbreak in 1854. Snow suspected that water being taken from the pump in Broad Street in Soho was the source of the disease and he tested this theory by reviewing the records of those who had died from the disease and by interviewing the surviving relatives. The results showed that a large majority of those who became ill had indeed drunk water from the Broad Street pump. Snow presented his findings to the local authorities whereupon the pump handle was removed and then the epidemic ended, thereby supporting Snow’s theory that cholera was a waterborne disease. More recent disease clusters include the appearance of a rare lung disease in a group of American Legionnaires in 1976 in Philadelphia caused by Legionella pneumophila (Legionnaires’ disease); several cases of an unusual type of pneumonia, caused by the fungal organism Pneumocystis carinii, among young homosexual men in Los Angeles (1981) that led to the identification of the human immunodeficiency virus (HIV) and acquired immunodeficiency syndrome (AIDS); and the outbreak in 2003 of a respiratory illness, subsequently identified as severe acute respiratory syndrome (SARS), caused by a previously unrecognized virus.
Investigations of non-infectious disease clusters have also resulted in important examples of medical breakthroughs in which a particular health effect was linked to exposure to a specific environmental toxin or pollutant. The fact that some diseases are more common amongst workers in particular occupations has been known since ancient times. As early as 1700, Bernardo Ramazzini published the book De Morbis Artificum Diatriba (The Diseases of Workers) in which he described the particular hazards of some fifty different occupations. It was not, however, until 1775 that Sir Percival Pott, a surgeon at St Bartholomew’s Hospital in London made what is widely regarded as the first scientifically sound connection between cancer and an environmental agent, by linking scrotal cancer in chimney sweeps to their exposure to the carcinogens (cancer causing agents) present in soot. This work was one of the earliest accounts of a cause and effect relationship being established for a carcinogen present in the workplace, and effectively marked the beginning of the study of occupational cancer. More recently, several well-known cancer clusters have been linked to exposure to other occupational toxins. The development of mesothelioma (a rare cancer of the lining of the chest and abdomen) was traced to exposure to asbestos, a material that was at the time used extensively in the construction and manufacturing industries. Liver angiosarcoma was linked to exposure to the vinyl chloride monomer, used in the production of polyvinyl chloride (PVC). Osteosarcoma, the commonest form of malignant bone cancer, was linked to exposure to radium, once widely used to paint watch dials so that they would glow in the dark. Systematic studies of particular groups of workers have identified several other carcinogens and have provided the much-needed motivation to find ways to reduce or eliminate such exposures in the workplace, and elsewhere. However, such examples of proven cause and effect are relatively rare. The examples given above have several things in common: the exposure to the toxin was enormous, the exposure occurred over long periods, and the disease was rare.
Residential or neighbourhood disease clusters, where a number of cases occur in a particular neighbourhood, can cause enormous and understandable public concern, particularly where the disease is one affecting children, such as leukaemia. Requests are often made to health authorities for further investigation into such clusters. And yet, in spite of years or even decades of trying, scientists have almost invariably failed in their efforts to identify any plausible cause. This often generates widespread suspicion amongst the public and the media because it seems to be somewhat paradoxical: there appears to be an unusually high occurrence of a particular disease, potential causes (toxins, pollutants, mobile phone masts, power lines etc.) are everywhere, and science has often been successful in identifying the causes of infectious disease outbreaks and occupational disease clusters. Why should it be so difficult to identify the cause of the residential cluster?
Concerns about possible disease clusters are not exclusive to cancer cases; similar concerns relate to birth defects and various neurological disorders. However, the majority of suspected clusters do involve such cases and here we will focus on cancer. Cancer is not a single disease but a group of approximately one hundred different diseases that share the same general feature: the uncontrolled growth and spread of abnormal cells within the body. Any individual’s risk of developing one particular form of cancer is thought to be influenced by a combination of several different factors that interact in ways that are still not completely understood. Some of these factors include: personal characteristics such as age, sex, and race; genetic factors shown by a family history of cancer; personal habits such as diet, cigarette smoking and alcohol or drug consumption; the presence of certain medical conditions, and some medical treatments; exposure to certain viruses; and exposure to various different environmental factors. Environmental factors known to be related to cancer include numerous chemicals and different forms of radiation, including sunlight. The incidence of particular cancers will vary quite significantly from place to place because several of the above factors, particularly age and personal habits, have a profound influence. For example, the incidence of cancer in a community consisting predominantly of the elderly would almost certainly be significantly higher than in a community with a much lower average age.
Not all suspected disease clusters will be investigated. No investigation will be carried out unless it can be demonstrated that the number of cases of the disease that have occurred over some particular time period is significantly greater than the number that would be expected. The incidence of the disease in an appropriate background, or reference, population will be used to estimate the expected number of cases. The expected number is simply calculated by multiplying the rate at which cases occur in the reference population during a specified period by the size of the population in the neighbourhood considered to be at risk. The initial task of the investigators is then to demonstrate that the observed number of cases is unlikely to have occurred solely by chance.
If there were no particular cause of a disease in a neighbourhood then the cases should have arisen with some perfectly natural, and predictable, variability. Statisticians use what are known as distributions to calculate the probability (P) that something will occur by chance. For example, the binomial distribution allows one to calculate the probability of obtaining a particular result from a series of trials where each trial has just two possible outcomes, such as when flipping a coin. For disease clusters what needs to be calculated is the probability that the observed number of cases would be found by chance when the expected number is known. This calculation is performed using a closely related distribution, known as the Poisson distribution.
t is common scientific practice to assume that a finding is statistically significant if the probability that it could have happened solely by chance is less than 1 in 20, what statisticians call P<0.05. But what do we actually mean by statistically significant? In normal English usage, “significant” is generally taken to mean important, whereas to a statistician a “significant” finding is one that probably did not happen solely by chance. Furthermore, any particular scientific finding may well be true without being important, and even when statisticians say a result is “highly significant” they simply mean that it very probably did not happen by chance; they do not necessarily mean that it is highly important. It is also worth noting that, although widely used, the 5 percent level is actually a completely arbitrary one, and represents a relatively high level of uncertainty on which to base any scientific judgement.
It is also worth considering at this stage what the P value of 0.05 actually means in practice. What would you expect to find if you were to determine the number of cases of a particular cancer in 100 randomly selected neighbourhoods? If the cases were occurring solely by chance, then one would expect to find approximately five neighbourhoods with a statistically significant elevation, P < 0.05, in the number of cancer cases and, incidentally, five with a statistically significant reduction. Based on these criteria, and given the large number of different cancers, it is inevitable that many neighbourhoods will have cancer clusters that can be demonstrated to be statistically significant. A further complicating factor is that many reported cancer clusters involve a relatively small number of cases. For technical reasons, this makes it appreciably more difficult to exclude the possibility that the cluster arose by chance or coincidence.
At first sight it would appear that calculating an accurate P (probability) value, and thereby identifying a potential cancer cluster, should be a relatively straightforward matter. There are, however, several serious practical issues that make this much more difficult than it might seem. If there is no particular disease determinant in the neighbourhood then the number of observed cases divided by the number of expected cases should be close to, but not of course exactly equal to, one. However, overestimating the number of observed cases or underestimating the number of expected cases would increase this ratio and thereby create a false cluster.
Determining the number of observed cases should, at least in principle, be relatively easy. The most probable source of error is the inclusion of patients who are not actually related to the suspected cluster. For example, only the primary cancer should properly be considered to be a part of the cluster, not one that has metastasized, or spread, from another organ. However, it may not always be easy for doctors to determine the primary site of a patient’s cancer, which makes it hard to determine whether that patient should be included. Other ways in which patients can be inappropriately included will be discussed further below.
Accurately estimating the number of expected cases is a far more difficult problem. As noted above, it would seldom be appropriate to use the incidence of the disease in the general population to estimate the number of expected cases. The reference population used in the calculation should be carefully selected so that it corresponds as closely as possible, in terms of age, gender, and ethnicity, to the neighbourhood population considered as being at risk. This is because all of these factors have a significant effect on the expected incidence of the disease. Furthermore, even with such a carefully selected reference population there is still the problem that it may, by necessity, be a relatively small one. In such cases the estimate of the number of expected cases will be subject to considerable error. Finally, any accurate calculation of the expected number requires complete and up-to-date information on the incidence of the disease within the community. This is only possible if a highly efficient health tracking system is in place, and such systems are not universal for cancer, and do not exist at all for some chronic diseases.
The major problems that can arise in many investigations of suspected clusters relate to the issue of accurately defining the size of the population from which the cases are thought to have arisen, and the time period over which those cases occurred. Suppose, for example, that three children in a single street have been diagnosed with leukaemia; you might choose to draw the geographical boundary around the street – which will almost inevitably create a statistically significant cluster – rather than around a somewhat larger area, which will probably not create a cluster. This is sometimes known as the Texas sharpshooter fallacy: the rifleman fires his gun randomly at the side of a building and only then does he paint a bullseye around the spot where the most bullet holes are seen to cluster.
Narrowing the geographic area around the observed cases will reduce the number of expected cases and make the apparent statistical significance greater. This process, often referred to as boundary tightening, relates not only to the selection of the geographical boundaries but also to the selection of the time period around suspected clusters and to the selection of age groups. A related problem is the temptation to expand selectively the geographic borders of the potential cluster to include additional cases of the suspected disease as they are discovered. The tendency to define the boundaries of a cluster on the basis of where known cases are located, rather than to first define the population and geographic area and then determine if the number of cases is excessive, will inevitably create many clusters that are not real. The conventional P value in statistics is only strictly interpretable with a priori hypotheses; i.e., hypotheses set up without any prior knowledge of the number of cases occurring in the population of interest.
In the United States there are between 1000 and 2000 reports of a suspected cancer cluster each year. The majority of these reports come from members of the public and many of these are resolved at an early stage when those enquiring are made aware of two important facts. First, that cancer is far more prevalent than is often imagined, and, second, that clusters will inevitably appear purely by chance. Truly random patterns often appear to us to be non-random because we have a tendency to assume that the behaviour of any large population will be perfectly replicated in any smaller sample taken from it. A simple example is what happens when one flips a coin several times and notes the sequence of heads (H) and tails (T). There is a natural tendency to assume that the sequence TTTHHH is in some way less random than, for example, HTHHTT. In fact, of course, these two sequences are equally probable.
In only 5 to 10% of all cases of reported/suspected clusters does formal statistical testing confirm that the number of observed cases exceeds the expected number by a significant amount. Even in these cases chance will always remain one of the possible explanations because even very improbable things will inevitably happen. It must always be remembered that truly random events do cluster by chance, and that statistical tests cannot separate observed clusters caused by chance from those due to some unidentified cause.
Whilst the task of identifying a possible cancer cluster is primarily a statistical exercise, the task of identifying a possible cause is an epidemiological one. In practice, further detailed investigation of those cases that do appear to be statistically significant has almost never identified the role that may have been played by environmental factors in the development of the disease. But why should it be so difficult to find a cause?
In purely practical terms one particular aspect of cancer causes some severe problems. Carcinogenesis (the process by which normal cells are transformed into cancer cells) involves a series of changes within cells that probably occur over several years. In many cases, more than 10 years might have elapsed between the exposure to a carcinogen and the subsequent diagnosis of cancer. Therefore, unlike infections, the effect of a carcinogen in a neighbourhood may not be seen for many years and, because we live in a highly mobile society, cancer victims who appear to be clustered may not all have lived in an area long enough for their cancers to have a common cause (i.e., it would not be plausible to link very recent exposure to the onset of the disease). Furthermore, it will be extremely difficult to pinpoint any possible cause of a cancer unless extremely detailed environmental records are available going back over many years.
Even in those cases where particular toxic substances are found, there may be no known biologically plausible link with the cancer being investigated, and it may often not be possible to demonstrate that all the affected individuals could have been exposed. In addition, toxic substances are very often present at levels that are not only relatively low but are also similar to levels found at other locations where there are no clusters. One possible explanation might be that there is an unidentified substance in a mix of pollutants that is unique to one location and that it is the combination of several different pollutants that causes the problem. But that, of course, raises the question of how one would set about finding it. Finally, if there is no direct link, there is always the possibility that the problem may have arisen through some indirect exposure. Might it be that the mothers were exposed during pregnancy or could the fathers have had their sperm affected at some time? The problem is that if you look hard enough and divide the data in enough different ways some possible association will almost certainly emerge. What, if anything, it means is, of course, an entirely different question.
There are those who believe that cluster investigations are not a good way to use increasingly scarce public health resources. Scientists know that a suspected cluster is more likely to be real, and therefore worth investigating, rather than chance or serious calculation error, if it involves one or more of the following factors: an exceptionally high cancer rate, i.e. a very low P value; a large number of cases of a specific type, or closely related types, of cancer rather than many different types; a rare cancer, rather than a common one; and an increased number of cases in an age group that is not generally affected by that type, as when young people develop a cancer that is usually seen only in the elderly. But even in these favourable cases it is extremely rare for a cause to be identified with anything even approaching total certainty.
So what are the parents of the young cancer sufferers introduced in our first paragraph to believe? Was something in the air (was someone, or something, to blame) or was it just a simple twist of fate? Tragically, they will probably never know for sure, as science may simply not be able to give them the answer. Are, then, the resources used in cluster investigations being wasted? Almost certainly not: they may at least serve to allay public fears over some environmental issue. Moreover, the intense scrutiny associated with a cluster investigation might well lead to the identification of real or potential environmental hazards that better be removed whether or not they are related to the suspected cluster. As Donald Rumsfeld once remarked “As we know, there are known knowns. There are things we know we know. We also know there are known unknowns. That is to say we know there are some things we do not know. But there are also unknown unknowns, the ones we don’t know we don’t know.” Perhaps the one real hope for the future is that cluster investigations may one day turn some of our unknown unknowns into known knowns.