Science - Statistics and Objecitivity

There is an increasing tendency in New South Wales for government authorities to use statistical devices to hide the inadequacies of environmental protection measures and to legitimise existing levels of pollution rather than being tough on polluters. The use of medians and percentiles rather than maximums and averages allows private firms and public organisations to discharge unlimited wastes for a certain percentage of time. Geometric means and medians are helping the government to coverup the frequency with which beaches are polluted and to understate the quantities of toxic waste being discharged into the ocean every year.

The Clean Waters Regulations, 1972 stated clearly in plain English what was required for protection of the environment. For example, wastes were not to be discharged if they adversely affected beaches or accumulated in marine life. In the implementation of these regulations the State Pollution Control Commission (SPCC) set numeric standards and criteria which gave the appearance of controls but in fact failed to ensure the simple, easily understood objectives in the Regulations were met.

The recent expose of inadequate regulation of Sydney's sewerage system and the resulting contamination of fish has forced the SPCC to include limits on some toxic wastes in the Water Board's ocean outfall licences and to put forward new water quality standards and design guide criteria. The simple English language goals in the Clean Waters Regulations are to be redefined into complex numeric formulas and statistical measures.

This latest wave of standards and criteria have cleverly utilised mathematical sleights of hand that hide the fact that they are unlikely to have any significant impact on the level of pollutants going into NSW waterways. In fact bathing standards and ocean outfall licence limits are being loosened at a time when the public assumes the government is cracking down on polluters.

The geometric mean of a number (n) of values is found by multiplying the values and finding the nth root of the multiplication. (Whereas an arithmetic mean or average of n values is found by adding the values and dividing by n.)

Geometric means are often used for statistically assessing beach pollution levels and setting standards. The standards set by the State Pollution Control Commission for New South Wales were derived from U.S. standards. Sydney beaches are defined as bathing waters for the months November through to May and during that time the geometric mean of five samples taken in a 30-day period should not exceed 200 faecal coliform units per 100ml. Also, only three samples taken during the November-May period are allowed to exceed 400 faecal coliform/100ml (S.P.C.C. undated). (Faecal coliform are organisms which occur naturally in the human gut and indicate the presence of sewage pollution.)

During the rest of the year it is assumed that the beaches are not bathing beaches (although many people still use the beaches during this time for surfing and swimming) and the geometric mean can be up to 1000 faecal coliform/100ml. During that same time only three samples should exceed 2000/100ml.

Authorities use a geometric mean because it is less effected by a few high values than an average and so it is thought that a geometric mean is more likely to give a typical value than an average. For example if the five samples give the results 10, 100, 100, 320, 10000 then the geometric mean is 200 whereas the average is 2106 and 200 would seem to be a more representative figure for the sample. However this is only true for some types of distributions. If, for example, the results are 10, 10, 320, 1000, 10000 then the geometric mean is 200 and the average is 2268, and the geometric mean of 200 seems to understate the series of readings.

The use of the geometric mean to measure sewage pollution therefore incorporates the unstated and unproven assumption that high readings will be rare and most readings will be at the lower end of the range. Although many countries make similar use of the geometric mean it is a questionable use in that it can mask regular instances of pollution. This becomes clear if we compare Water Board monitoring of Sydney beaches (as required by the State Pollution Control Commission) with Department of Health monitoring of those same beaches.

The Department of Health also uses a geometric mean but in a very different way. They take three samples of water from a beach at the one time and find the geometric mean of those three samples. Whereas the SPCC and the Water Board are attempting to find a figure representative of variations that occur over a month, the Health Department is finding a figure that is representative of variations along the length of a beach. The Health Department considers a beach unsatisfactory for swimming if the geometric mean of samples taken on the beach on a particular day excedes 300 faecal coliform/100 ml. Table 1 shows that using this method the Department of Health monitoring shows that Eastern suburbs beaches are unsatisfactory for swimming a significant amount of the time whereas the SPCC/Water Board use of the geometric mean hides unsatisfactory days altogether, even though they require a lower geometric mean to be met.

For the public the SPCC/Water Board geometric mean tells them little about how often the bathing waters are clean enough to swim in. Compliance with the 90 percentile condition (where only 10% of the samples can exceed 400 faecal coliform/100ml in summer) is usually shown separately in Water Board reports and is fairly meaningless to most people. It is this deceptive use of mathematical statistics that enabled the Water Board to argue throughout the 1980s that beach pollution at Sydney beaches was not too bad when unpublished Health Department figures were showing that many of Sydney's prime beaches were so polluted that they were unsatisfactory for swimming from 30-80 percent of the time.

In the last year or so the use of a geometric mean for bathing water standards has come under attack in Sydney and the SPCC now proposes to replace the bathing water standards for NSW. They are proposing to replace the geometric mean of 200 with a median of 150 faecal coliform/100ml (SPCC1990a; SPCC 1990b). (In other words the median of at least five samples taken in a 30 day period should not exceed 150 faecal coliform units per 100ml). Whilst this might appear superficially to be a tightening of the standards, one must remember that a geometric mean is affected by the magnitude of all the samples whereas a median is only the middle-sized sample and it is not affected by how polluted the other two samples are. So if the five samples give the results 10, 100, 100, 400, 10,000 the geometric mean exceeds 200 but the median does not exceed 150.

This situation is exacerbated by the proposal to replace the 90 percentile of 400 fc/100ml in the old standards by an 80 percentile of 600 fc/100ml in the proposed new standards. (The new standards require that four out of the five samples taken each month do not exceed 600 faecal coliform/100 ml.) Under this new SPCC system, one sample in five can be extremely polluted and at least 40% of the time the beaches can be unsatisfactory for swimming according to the Department of Health standards (ie 2 samples out of five can exceed 300 fc/100ml) A recent epidemiological study in Sydney showed that persons who swam in water with more than 300 fc/100 ml were 35% more likely to report symptoms of illness than people who swam in cleaner water (government media release, 4/6/90).

The use of medians is also a new feature of the licences that the SPCC grants to polluters under the Clean Waters Act. Limits on some toxic substances were first instituted in Water Board interim licences issued in May 1989 after fish studies done in previous years received some unwelcome publicity. The interim licenses issued by the SPCC set maximum concentrations for five toxic substances in the effluent which had been in excessive levels in fish tested in 1987 and 1988. By July 1989, when licenses were re-issued for the following year, the standards in them had been changed so that those maximum levels for the five toxic substances had become median levels (see Table 2). In other words the limits only had to be met half the time.

This use of a median or 50 percentile is incomprehensible in terms of environmental protection. Bioaccumulation of these substances in marine life depends on total volumes being discharged and yet a median limit only regulates the less worrying half of the total flow.

The use of a median has also been proposed for the new design criteria for ocean outfalls. Under these criteria the allowable concentrations of toxic substances at the completion of initial dilution in the ocean are expressed as six monthly medians (SPCC 1990b). These standards are derived from figures called "chronic criteria" (mainly determined by the US Environmental Protection Agency) that represent levels of toxicity which are likely to damage marine organisms over a length of time.

In order to convert these chronic criteria for toxic wastes to 6 month median values the SPCC multiplied them by 0.5. In other words, the SPCC has taken values that if exceeded on average over any four day period in 3 years may damage marine life, divided them by two and said they can be exceeded (with no upper limit) 50% of the time within each 6 month period! It is an extraordinary assumption that this will protect marine life and begs the question of why the SPCC does not set these limits in terms of four day averages.

The reason the SPCC gives for using medians and other percentiles rather than maximums as standards is that it is easier to use such statistical measures for design purposes. Yet the SPCC does not confine their use to design criteria and as can be seen above, uses them for licence conditions and on-going water quality management purposes. This makes far less sense. The Water Board admitted in an internal report in 1989 that more than half of its inland treatment plants violated 1988/89 licence conditions. These conditions are generally expressed as 50 and 90 percentiles. The report noted that because there was no upper bound pollutant concentration specified in the licences but only statistical requirements to be met, it was difficult to know how well treatment plants were complying until a full year's statistics had been completed.

Medians make more sense if you are concerned with good public relations. Medians tend to be much lower than maximums and in the case of Sydney's sewage stream, somewhat lower than averages, because for less than half of the time the concentrations can be very high. In recent times the Water Board has begun calculating total flows of toxic substances by multiplying median concentrations of these substances by median flow volumes. (For years they had argued against any calculation of total tonnages of heavy metals being discharged saying that it was the concentration that mattered.) Using medians as the basis for their estimate the Board calculates that 239 tonnes of heavy metals are discharged via their Sydney outfalls into the ocean every year (e.g. Sun-Herald 29/7/90). This grossly underestimates the actual amount going into the ocean as can be seen in Table 4 for the Malabar outfall.

Table 3 shows the variation in concentrations of heavy metals coming into the Malabar sewage treatment plant. Samples were taken every 6 days whether or not it was a weekend or a holiday. The heavy metals coming into the Malabar plant currently go into the ocean via the effluent and the sludge. The only material removed from the sewage which doesn't go into the ocean is the screenings and scum which are incinerated. The Board claims that there are negligible metals in those.

One can see that the average concentration of heavy metals at the Malabar plant is usually much higher than the median because high concentrations often occur, although for less than half the time. In the worst case a substance such as selenium is going out at less than 3.5 micrograms/litre for half the time but for 15% of the time it is going out at more than 100 micrograms/litre and for 27% of the time at more than 50 micrograms/litre. The average concentration is about 40 micrograms/litre. The median therefore vastly understates the amount of selenium that is being discharged.

In every case the median is significantly lower than the average as can be seen by the table. If one multiplies the concentrations of each heavy metal by the flow on the day of sampling, average those figures and extrapolate for the whole year then the total is about 400 tonnes/year. Similar calculations would give approximately another 200 tonnes/year for the other Sydney outfalls. This is more than double the Water Board's estimate of 239 tonnes per year at all outfalls.

...back to top

The 90 Percentile

The Water Board is able to allow such large quantities of toxic wastes into the sewers because of the weak standards the SPCC puts on ocean discharges. Because the SPCC does not limit maximum concentrations nor maximum total quantities of toxic waste going into the ocean the Water Board does not have to put maximum limits on what industry can put into the sewers leading to the ocean.

The Water Board uses geometric means and 90 percentiles in its trade waste agreements with industry. These trade waste agreements are supposed to regulate what is allowed into the sewers. Firms pay for toxic waste going into the sewers on a sliding scale of charges which increase as the concentration of toxic waste in their effluent increases. Since concentration usually varies the charges are estimated on the basis of a 90 percentile concentration (Water Board 1988). The 90 percentile is the value below which 90% of all samples occur.

In the examples given by the Board in its Trade Waste Policy booklet the sliding geometric mean concentration (the geometric mean of the each 3 samples) is much higher (33%) than the 90 percentile which indicates that the Board expects high concentrations some of the time.

Regulation in terms of 90 percentiles and means severely limits the ability of the Board to prosecute a company when it has a spill for breach of its trade waste agreement. Similarly the use of percentiles rather than maximums by both the SPCC and the Water Board ensure that it is no longer possible for environmental groups such as Greenpeace to catch a firm exceeding its license. In the past it was sufficient to take a sample and if the concentration of toxic waste exceeded the maximum allowable the firm was caught red-handed. Now samples taken on a single occasion are meaningless since the firm could argue that the sampling had taken place on the rare occasion when they were exceeding the median/geometric mean/90 percentile which they are legally entitled to do.

During the 1970s and main part of the 1980s both the SPCC and the Water Board focussed on the acute affects of toxic wastes and regulated in terms of maximum concentrations of toxic wastes in the sewers and in the ocean. They did not concern themselves with total quantities going out over long periods of time which is an important parameter for controlling chronic effects such as the bioaccumulation of these wastes in marine life.

Now that bioaccumulation has been shown to be such a large problem off Sydney's coastline, both organisations have dropped their maximum limits and replaced them with percentiles and means. At first sight this seems to be a better way of regulating chronic (long-term) effects because it is the usual, typical flows that are being regulated rather than just the peak flows. However the use of medians alone can only regulate half the flow and it is the other half that is of concern for both acute and chronic impacts. When medians are complemented by 80 or 90 percentiles the control is increased but there is still the problem of the other 10 or 20 percent that remain unregulated.

Clearly the environment needs to be protected against acute and chronic impacts and for this reason, maximum limits for concentrations of toxic waste and for total quantities of toxic wastes being discharged need to be included in the regulations.

With respect to bathing waters, statistical measures such as the geometric mean and the median need to be assessed for their appropriateness for the statistical distribution of pollution. Moreover, beachgoers do not want to know how polluted the beach is "typically" which is what such measures might tell them. They are more interested in how often the beaches are unsuitable for swimming (e.g. over 300 f.c./100 ml) and what the probabilities are of it being this way when they want to go to the beach.

SPCC, (1990b) Draft WP-1 Design Guide for the Discharge of Wastes to Ocean Waters.

Water Board, (1990), Malabar Sewage Treatment Plant SPCC Compliance Report, 17 Dec.1989-10 Mar.1990.

Sharon Beder, The Many Meanings of Means, Search, vol.22, no.3, April/May 1991, pp. 88-90.