Monitoring Susceptibility to Spinosad in Three Major Stored-Product Insect Species from Punjab, Pakistan

Hafiz Azhar Ali Khan1,*, Waseem Akram2, Sumi Lee3, Shakir Manzoor1, Syed Rajab Ayub1, Khalil Ur Rehman1, Shinawar Waseem Ali1, Muhammad Bilal Chattha1 and Sumaira Maqsood1

1Institute of Agricultural Sciences, University of the Punjab, Lahore, Pakistan

2Department of Entomology, University of Agriculture, Faisalabad

3Changwon National University, Gyeongsang, South Korea

ABSTRACT

Spinosad, a reduced risk environment friendly insecticide, is increasingly being used as a grain protectant in different countries. Geographical variation in susceptibility to spinosad in stored grain insect pests has been reported worldwide; however, there is a lack of information in Pakistan. In the present study, one laboratory reference strain and five field strains each of, Rhyzopertha dominica, Sitophilus oryzae and Tribolium castaneum, were collected from Punjab, and assessed for their susceptibility to spinosad via diet incorporation bioassays. Based on median lethal concentration (LC50) values, all the strains of S. oryzae and R. dominica were more susceptible to spinosad than T. castaneum strains. The resistance ratios (RRs) of field strains at LC50 values were in the range of 2.24 to 3.24 fold for T. castaneum, 3.33 to 9.00 fold for R. dominica, and 1.73 to 3.45 fold for S. oryzae. The results revealed a variation in susceptibility to spinosad in field strains. Very low levels of resistance to spinosad in field strains warrant designing resistance management strategy for stored insect pests. The baseline data regarding spinosad susceptibility determined here for these stored insect pests will therefore help to monitor resistance in the future.

Article Information

Received 27 August 2017

Revised 12 January 2018

Accepted 24 March 2018

Available online 28 May 2018

Authors’ Contribution

HAAK designed and performed the experiment, analyzed the data and wrote the manuscript. WA and SL helped in writing the manuscript. SM, SRA, KUR, SWA, MBC and SM helped in data collection and maintenance of stored-insects strains.

Key words

Biorational insecticide, Microbial insecticide, Insecticide resistance, Ecotoxicology.

DOI: http://dx.doi.org/10.17582/journal.pjz/2018.50.4.1355.1360

* Corresponding author: azhar.iags@pu.edu.pk

0030-9923/2018/0004-1355 $ 9.00/0

Introduction

The coleopteran insects Rhyzopertha dominica (F.), Sitophilus oryzae (L.), and Tribolium castaneum (H.), are among the most destructive stored grain insect pests around the globe with the ability to cause 20% or more postharvest losses in developing countries (Phillips and Throne, 2009). Chemical control has been the primary measure to manage these pests; however, development of resistance to these chemicals made control of stored insects difficult (Arthur, 1996; Vayias et al., 2010). Whereas, the use of the conventional grain protectants are being questioned due to long lasting residual effects, high mammalian toxicity and environmental pollution. Therefore, evaluation of environment friendly, reduced risk insecticides for an effective stored product pest management program is necessary.

Spinosad has been considered as a safe, environment friendly microbial based insecticide. It is extracted from Saccharopolyspora spinosad bacterium (Mertz and Yao, 1990), having very low mammalian toxicity (Athanassiou et al., 2011; Khan et al., 2013, 2014). It has been used against a number of stored insect pests under the laboratory and field conditions and showed promising results (Toews et al., 2003; Huang and Subramanyam, 2004; Athanassiou et al., 2008; Vayias et al., 2010). It is recommended in the USA at a label rate of 1 ppm of stored grains, and this rate has been reported to control completely the adults of different beetles including R. dominica and S. oryzae (Fang et al., 2002b; Kavallieratos et al., 2010), but this rate provides partial control of T. castaneum (Athanassiou et al., 2011). However, spinosad has not yet been registered as a grain protectant in Pakistan.

When an insecticide develops, regular susceptibility monitoring surveys should be conducted in order to sustain the effectiveness of the insecticide for long (Khan et al., 2015a). Such surveys enable the scientists or pest managers to develop proactive resistance management programs, and judicious use of the insecticide which also protect the environment. Susceptibility to insecticides changes with time and space (Zhao et al., 2006; Ilyas et al., 2017). Previously, a spatial variation in spinosad susceptibility has been reported in different stored insects like T. castaneum, R. dominica, and Cryptolestes ferrugineus (S.) from different states of the USA (Huang et al., 2004; Subramanyam et al., 2007; Bajracharya et al., 2013; Sehgal et al., 2013), Australia (Daglish, 2008) and Iran (Sadeghi and Ebadollahi, 2015), and T. confusum strains from Europe (Athanassiou et al., 2008). However, this information is lacking in Pakistan. Keeping in view the importance of spinosad with respect to environmental safety, low mammalian toxicity, and potential as a grain protectant, the present study was planned to evaluate geographical variation in the susceptibility to spinosad in laboratory and field strains of T. castaneum, R. dominica and S. oryzae collected from different localities of Punjab. The baseline data obtained in the present work could benefit spinosad resistance monitoring in the future.

Table I.- Field collection history of three major stored insect pests.

Species/Site		Host	Strain ID	Collection year
T. castaneum
	Faisalabad	Wheat	Lab-TC	2011
	Lahore	Wheat flour mill	LHR-TC	2015
	Multan	Rice	MTN-TC	2015
	Jhang	Wheat	JHG-TC	2015
	Bahawalpur	Rice	BWP-TC	2014
	Sahiwal	Rice	SHL-TC	2015
R. dominica
	Multan	Wheat flour mill	Lab-RD	2011
	Lahore	Wheat flour mill	LHR-RD	2014
	Multan	Wheat flour mill	MTN-RD	2014
	Jhang	Wheat	JHG-RD	2015
	Bahawalpur	Rice	BWP-RD	2015
	Sahiwal	Wheat	SHL-RD	2015
S. oryzae
	Multan	Rice	Lab-SO	2011
	Lahore	Rice	LHR-SO	2015
	Multan	Rice	MTN-SO	2014
	Jhang	Rice	JHG-SO	2015
	Bahawalpur	Wheat flour mill	BWP-SO	2015
	Sahiwal	Wheat	SHL-SO	2015

Materials and methods

Stored-insects collection and maintenance

Adult stored grain insects T. castaneum, R. dominica, and S. oryzae were collected from wheat flour mills or from round metal bins containing stored wheat or rice (Table I), and reared in one-liter glass jars under the laboratory conditions 28 ± 1oC, and 65% relative humidity. Rhyzopertha dominica and S. oryzae were reared on whole wheat grains (12% moisture), while T. castaneum was reared on wheat flour containing 5% (w/w) Brewer’s yeast. Bioassays with field strains of all insects were conducted using third generation adults. The laboratory strains: Lab-TC, Lab-RD and Lab-SO, maintained under the laboratory conditions for over five years without exposure to any insecticide, served as laboratory susceptible reference strains. The Lab- strains were also reared in the same manner as mentioned above.

Bioassays

Spinosad susceptibility was assessed following the bioassay protocols reported by Huang et al. (2004), with a little bit modification. In brief, clean wheat grains (variety Seher-06; moisture contents ≈12%) were treated with a spinosad formulation (Tracer 24SC, Arysta Life Sciences, Pakistan) to make different concentrations ranging from 0.063-2, 0.125-4, and 0.25-8 mg/kg for assaying R. dominica, S. oryzae, and T. castaneum, respectively. These ranges of spinosad formulation were prepared in distilled water with the principle of bioassays i.e., >0% mortality at the lowest concentration and <100% mortality at the highest concentration (Khan and Akram, 2017; Robertson et al., 2007). 0.1 ml of the respective insecticide concentration was applied to 100 grams wheat grains in a glass jar of a half-liter capacity. For the purpose to evenly distribute insecticide solution, the treated grains were agitated manually for five minutes (Athanassiou et al., 2011; Khan et al., 2016). After agitation, 25 mixed sex adults (2-3-weeks-old) were introduced, and infested jars were closed with filter papers and incubated for seven days. After that, the grains were sieved to record mortality in each jar.

Data analysis

The median lethal concentrations (LC50s) were calculated using the software SPSS v. 10 for windows (Bryman and Cramer, 2002). The LC50 values of each insect strain were compared with those of the corresponding laboratory susceptible reference strains, and the status of the spinosad susceptibility were scaled by following Khan et al. (2017).

Results

Response of T. castaneum

The results of bioassays revealed differences in susceptibility to spinosad in different strains (Table II). The Lab-TC strain was the most susceptible strain with LC50 value 0.17 mg/kg. The LC50 values among the field strains ranged from 0.38 to 0.56 mg/kg. The adults of the MTN-TC strain were less susceptible (resistance ratio (RR) = 3.24 fold) to spinosad when compared with the Lab-TC strain. The RR values of the rest of the field strains were: 2.24 fold for LHR-TC, 2.65 fold for BWP-TC, and 2.76 fold for JHG-TC.

Table II.- Toxicity of spinosad to adults of stored insects from Punjab, Pakistan.

Strain	n*	LC50 (95% CI) (mg/kg)	Fit of probit line				RR
Strain	n*	LC50 (95% CI) (mg/kg)	Slope (SE)	χ2	df	p	RR
T. castaneum
Lab-TC	450	0.17 (0.09-0.25)	1.28 (0.18)	0.11	5	0.99
LHR-TC	450	0.38 (0.25-0.52)	1.13 (0.13)	2.27	5	0.68	2.24
MTN-TC	450	0.56 (0.41-0.72)	1.27 (0.14)	4.48	5	0.34	3.24
JHG-TC	450	0.47 (0.36-0.59)	1.60 (0.16)	5.13	5	0.27	2.76
BWP-TC	450	0.45 (0.34-0.57)	1.51 (0.15)	2.18	5	0.70	2.65
R. dominica
Lab-RD	450	0.03 (0.004-0.06)	1.56 (0.44)	0.67	5	0.95
LHR-RD	450	0.10 (0.04-0.16)	1.19 (0.24)	0.51	5	0.97	3.33
MTN-RD	450	0.17 (0.06-0.31)	0.82 (0.21)	0.49	5	0.97	5.67
JHG-RD	450	0.23 (0.15-0.34)	1.37 (0.24)	0.39	5	0.98	7.57
BWP-RD	450	0.25 (0.17-0.35)	1.55 (0.25)	1.35	5	0.85	8.33
SHL-RD	450	0.27 (0.18-0.41)	1.32 (0.22)	2.26	5	0.68	9.00
S. oryzae
Lab-SO	450	0.11 (0.06-0.16)	1.27 (0.16)	1.29	5	0.86
LHR-SO	450	0.19 (0.13-0.24)	1.28 (0.14)	2.96	5	0.56	1.73
MTN-SO	450	0.22 (0.14-0.31)	1.04 (0.13)	5.44	5	0.24	2.00
JHG-SO	450	0.23 (0.17-0.36)	2.11 (0.20)	7.18	5	0.12	2.09
BWP-SO	450	0.30 (0.25-0.36)	2.12 (0.18)	6.55	5	0.16	2.73
SHL-SO	450	0.37 (0.24-0.54)	1.76 (0.15)	7.68	5	0.10	3.45

Response of R. dominica

The adults of R. dominica were more susceptible than those of the T. castaneum. Among the different strains of R. dominica tested against spinosad, Lab-RD was the most susceptible one with the LC50 value 0.03 mg/kg. The LHR-RD strain was the most susceptible one (RR=3.33 fold) among the field strains when compared with the Lab-RD. whereas, the rest of the field strains were less susceptible with the RR values 5.67, 7.57, 8.33 and 9.00 fold, for MTN-RD, JHG-RD, BWP-RD and SHL-RD, respectively (Table II).

Response of S. oryzae

Of five field strains of S. oryzae tested against spinosad, LHR-SO was the most susceptible strain (RR=1.73 fold), followed by MTN-SO (RR=2.00 fold), JHG-SO (RR=2.09 fold), BWP-SO (RR=2.73 fold) and SHL-SO (RR=3.45 fold) (Table II).

Discussion

Spinosad is considered as an environment friendly insecticide and is being used as a grain protectant in different countries (Sehgal et al., 2013), but it has not yet been released commercially as a grain protectant in Pakistan. The study was conducted the first time in Punjab, Pakistan, to assess the susceptibility status of spinosad in the laboratory and field strains of T. castaneum, R. dominica and S. oryzae. The baseline data regarding the spinosad susceptibility in the studied stored insect pests will therefore help to monitor resistance in the future. The findings of the study revealed that the field strains of all the stored insects were a little bit resistant to spinosad compared with their respective susceptible reference strains. However, field strains may not be assumed resistant to a particular insecticide until tenfold RR is observed (Khan et al., 2013a; Valles et al., 1997). Therefore, less than tenfold RR in all the field strains of beetles might be due to tolerance rather than resistance. Previously, a spatial variation in spinosad susceptibility has been reported in different stored insects like T. castaneum, R. dominica, and Cryptolestes ferrugineus (S.) from different states of the USA (Bajracharya et al., 2013; Huang et al., 2004; Sehgal et al., 2013; Subramanyam et al., 2007), Australia (Daglish, 2008) and Iran (Sadeghi and Ebadollahi, 2015), and T. confusum strains from Europe (Athanassiou et al., 2008). Differences in susceptibility levels against an insecticide among different field strains of the same or different insect species is a common character (Huang et al., 2004; Athanassiou et al., 2008; Sehgal et al., 2013; Khan et al., 2014, 2015c; Khan and Akram, 2017). For example, Athanassiou et al. (2008) reported the field strains of T. confusum collected from Europe showed varying levels of susceptibility to spinosad on wheat grains with up to 81% adult mortalities. Similarly, Huang et al. (2004) reported that different field strains of C. ferrugineus, T. castaneum, and Plodia interpunctella showed varying levels of resistance to spinosad compared with the corresponding laboratory strains. Such type of variations in susceptibility to a particular insecticide might be due to differences in insecticide exposure history, insects’ mobility, insecticide penetration and/or metabolism (Sehgal et al., 2013; Yasoob et al., 2017).

In this study, based on LC50 values, adult beetles of R. dominica and S. oryzae were found to be more susceptible to spinosad than those of T. castaneum. Among the different strains of R. dominica tested against spinosad, Lab-RD was the most susceptible one with the LC50 value 0.03 mg/kg. The LHR-RD strain was the most susceptible one among the field strains when compared with the Lab-RD. whereas, the rest of the field strains were less susceptible with the RR values 5.67, 7.57, 8.33 and 9.00 fold, for MTN-RD, JHG-RD, BWP-RD and SHL-RD, respectively. However, the variation in responses among different R. dominica strains might not be of practical importance since the bioassay results of the current and some previous studies has shown this species to be more susceptible than the rest of the beetle strains. These findings are consistent with those previously reported in literature (Fang et al., 2002a; Toews and Subramanyam, 2003; Huang et al., 2004; Sehgal et al., 2013; Khan et al., 2016).

The decreased susceptibility to various insecticides including spinosad has been reported in different stored insects including T. castaneum, R. dominica, and S. oryzae (Collins et al., 1993; Nayak et al., 2005). However, in the present study the decreased susceptibility to spinosad in the field strains of stored insects may not be linked with the cross-resistance, since a number of insects showing resistance to other insecticides don’t show cross-resistance to spinosad owing to the fact of a unique mode of action (Shono and Scott, 2003; Khan et al., 2014, 2015c) as it acts at the GABA receptor and nicotinic acetylcholine receptor sites of the insect nervous system (Salgado, 1998). However, there is a need to further confirm the phenomenon of cross-resistance in stored insects.

In the present study, the field strains of stored insects collected from different localities showed different responses to spinosad. The field strains of all the stored insects collected from Lahore showed more susceptibility to spinosad than the rest of the field strains. Geographic or spatio-temporal variation in susceptibility to insecticides is a common phenomenon (Huang et al., 2004). Previously, such types of geographical variations against spinosad have been reported in different insects like Musca domestica (Khan et al., 2013, 2015b), Aedes albopictus (Khan et al., 2011), Helicoverpa armigera (Ahmad et al., 2003), Frankliniella occidentalis (Bielza et al., 2007), P. interpunctella, C. ferrugineus, T. castaneum, R. dominica (Huang et al., 2004), Alphitobius diaperinus (Lambkin and Rice, 2007) and O. surinamensis (Sehgal et al., 2013).

Spinosad has been assumed to have a unique mode of action and targets nicotinic acetylcholine receptor sites in insects (Salgado, 1998). Different spinosad resistance mechanisms have been reported in different insect pests and/or even different strains of the same insect species. Geographical variations and insecticide exposure histories play a pivotal role in determining a particular resistance mechanism against a particular insecticide (Sayyed et al., 2008). A number of reports revealed that resistance to spinosad in different insects could be due to mutations in target-sites rather than metabolic detoxification (Bielza et al., 2007; Sayyed et al., 2008; Khan et al., 2014). There is a dire need to assess the type of spinosad resistance mechanism and mode of inheritance in studied stored insects by selecting individual strains under the laboratory conditions which would be helpful for effective management of stored insects.

Conclusion

In conclusion, reduced susceptibility to spinosad in stored insects necessitates the need to use this chemical cautiously in stored insect pest management programs to retain its effectiveness for a longer period of time. In this regard, regular resistance monitoring surveys and rotational use of alternate insecticides are considered the best resistance practices for resistance management, and for retaining the effectiveness of insecticides in the future (Khan et al., 2013, 2013a). Our findings provide baseline data for susceptibility to spinosad in the laboratory and field strains of T. castaneum, R. dominica, and S. oryzae. These data could be helpful for future resistance monitoring efforts in the studied strains, once spinosad is registered and released commercially as a grain protectant in Pakistan.

Statement of conflict of interest

Authors have declared no conflict of interest.

References

Ahma d, M., Arif, M.I. and Ahmad, Z., 2003. Susceptibility of Helicoverpa armigera (Lepidoptera: Noctuidae) to new chemistries in Pakistan. Crop Protect., 22: 539-544. https://doi.org/10.1016/S0261-2194(02)00219-3

Arth ur, F.H., 1996. Grain protectants: current status and prospects for the future. J. Stored Prod. Res., 32: 293-302. https://doi.org/10.1016/S0022-474X(96)00033-1

Atha nassiou, C.G., Arthur, F.H., Kavallieratos, N.G. and Throne, J.E., 2011. Efficacy of spinosad and methoprene, applied alone or in combination, against six stored-product insect species. J. Pestic. Sci., 84: 61-67. https://doi.org/10.1007/s10340-010-0326-1

Atha nassiou, C.G., Kavallieratos, N.G. and Chintzoglou, G.J., 2008. Effectiveness of spinosad dust against different European populations of the confused flour beetle, Tribolium confusum Jacquelin du Val. J. Stored Prod. Res., 44: 47-51. https://doi.org/10.1016/j.jspr.2007.05.001

Bajr acharya, N., Opit, G.P., Talley, J. and Jones, C., 2013. Efficacies of spinosad and a combination of chlorpyrifos-methyl and deltamethrin against phosphine-resistant Rhyzopertha dominica (Coleoptera: Bostrichidae) and Tribolium castaneum (Coleoptera: Tenebrionidae) on wheat. J. econ. Ent., 106: 2208-2215. https://doi.org/10.1603/EC13215

Biel za, P., Quinto, V., Contreras, J., Torne, M., Martin, A. and Espinosa, P.J., 2007. Resistance to spinosad in the western flower thrips, Frankliniella occidentalis (Pergande), in greenhouses of south-eastern Spain. Pest Manage. Sci., 63: 682-687. https://doi.org/10.1002/ps.1388

Bryma n, A. and Cramer, D., 2002. Quantitative data analysis with SPSS release 10 for Windows: A guide for social scientists. Routledge.

Colli ns, P., Lambkin, T., Bridgeman, B. and Pulvirenti, C., 1993. Resistance to grain-protectant insecticides in coleopterous pests of stored cereals in Queensland, Australia. J. econ. Ent., 86: 239-245.

Dagli sh, G.J., 2008. Impact of resistance on the efficacy of binary combinations of spinosad, chlorpyrifos-methyl and s-methoprene against five stored-grain beetles. J. Stored Prod. Res., 44: 71-76. https://doi.org/10.1016/j.jspr.2007.06.003

Fang, L., Subramanyam, B. and Arthur, F.H., 2002a. Effectiveness of spinosad on four classes of wheat against five stored-product insects. J. econ. Ent., 95: 640-650.

Fang, L., Subramanyam, B. and Arthur, F.H., 2002b. Effectiveness of spinosad on four classes of wheat against five stored-product insects. J. econ. Ent., 95: 640-650.

Huang , F. and Subramanyam, B., 2004. Responses of Corcyra cephalonica (Stainton) to pirimiphos-methyl, spinosad, and combinations of pirimiphos-methyl and synergized pyrethrins. Pest Manage. Sci., 60: 191-198. https://doi.org/10.1002/ps.815

Huang, F., Subramanyam, B. and Toews, M.D., 2004. Susceptibility of laboratory and field strains of four stored-product insect species to spinosad. J. econ. Ent., 97: 2154-2159.

llyas, A., Khan, H.A.A. and Qadir, A., 2017. Effect of leaf extracts of some indigenous plants on settling and oviposition responses of peach fruit fly, Bactrocera zonata (Diptera: Tephritidae). Pakistan J. Zool., 49: 1547-1553.

Kavalli eratos, N.G., Athanassiou, C.G., Vayias, B.J., Kotzamanidis, S. and Synodis, S.D., 2010. Efficacy and adherence ratio of diatomaceous earth and spinosad in three wheat varieties against three stored-product insect pests. J. Stored Prod. Res., 46: 73-80. https://doi.org/10.1016/j.jspr.2009.10.003

Khan, H.A.A. and Akram, W., 2017. Cyromazine resistance in a field strain of house flies, Musca domestica L.: Resistance risk assessment and bio-chemical mechanism. Chemosphere, 167: 308-313. https://doi.org/10.1016/j.chemosphere.2016.10.018

Khan, H.A.A., Akram, W. and Fatima, A., 2017. Resistance to pyrethroid insecticides in house flies, Musca domestica L., (Diptera: Muscidae) collected from urban areas in Punjab, Pakistan. Parasitol. Res., 116: 3381-3385. https://doi.org/10.1007/s00436-017-5659-8

Khan, H .A.A., Akram, W., Arshad, M. and Hafeez, F., 2015a. Toxicity and resistance of field collected Musca domestica (Diptera: Muscidae) against insect growth regulator insecticides. Parasitol. Res., 115: 1385-1390. https://doi.org/10.1007/s00436-015-4872-6

Khan, H .A.A., Akram, W. and Haider, M.S., 2015b. Genetics and mechanism of resistance to deltamethrin in the house fly, Musca domestica L., from Pakistan. Ecotoxicology, 24: 1213-1220. https://doi.org/10.1007/s10646-015-1482-0

Khan, H .A.A., Akram, W., Iqbal, J. and Naeem-Ullah, U., 2015c. Thiamethoxam resistance in the house fly, Musca domestica L.: Current status, resistance selection, cross-resistance potential and possible biochemical mechanisms. PLoS One, 10: e0125850. https://doi.org/10.1371/journal.pone.0125850

Khan, H .A.A., Akram, W. and Shad, S.A., 2013a. Resistance to conventional insecticides in Pakistani populations of Musca domestica L. (Diptera: Muscidae): A potential ectoparasite of dairy animals. Ecotoxicology, 22: 522-527. https://doi.org/10.1007/s10646-013-1044-2

Khan, H .A.A., Akram, W. and Shad, S.A., 2014. Genetics, cross-resistance and mechanism of resistance to spinosad in a field strain of Musca domestica L. (Diptera: Muscidae). Acta Trop., 130: 148-154. https://doi.org/10.1016/j.actatropica.2013.11.006

Khan, H .A.A, Shad, S.A. and Akram, W., 2013. Resistance to new chemical insecticides in the house fly, Musca domestica L., from dairies in Punjab, Pakistan. Parasitol. Res., 112: 2049-2054. https://doi.org/10.1007/s00436-013-3365-8

Khan, H .A.A., Akram, W., Shehzad, K. and Shaalan, E.A., 2011. First report of field evolved resistance to agrochemicals in dengue mosquito, Aedes albopictus (Diptera: Culicidae), from Pakistan. Parasit Vectors, 4: 146. https://doi.org/10.1186/1756-3305-4-146

Khan, T ., Shahid, A.A. and Khan, H.A.A., 2016. Could biorational insecticides be used in the management of aflatoxigenic Aspergillus parasiticus and its insect vectors in stored wheat? Peer J., 4: e1665. https://doi.org/10.7717/peerj.1665

Lambkin , T.A. and Rice, S.J., 2007. Baseline responses of Alphitobius diaperinus (Coleoptera: Tenebrionidae) to spinosad, and susceptibility of broiler populations in eastern and southern Australia. J. econ. Ent., 100: 1423-1427. https://doi.org/10.1093/jee/100.4.1423