Coexistence of plant species under harsh environmental conditions: an evaluation of niche differentiation and stochasticity along salt marsh creeks

Abstract

Background

Ecologists have achieved much progress in the study of mechanisms that maintain species coexistence and diversity. In this paper, we reviewed a wide range of past research related to these topics, focusing on five theoretical bodies: (1) coexistence by niche differentiation, (2) coexistence without niche differentiation, (3) coexistence along environmental stress gradients, (4) coexistence under non-equilibrium versus equilibrium conditions, and (5) modern perspectives.

Results

From the review, we identified that there are few models that can be generally and confidently applicable to different ecological systems. This problem arises mainly because most theories have not been substantiated by enough empirical research based on field data to test various coexistence hypotheses at different spatial scales. We also found that little is still known about the mechanisms of species coexistence under harsh environmental conditions. This is because most previous models treat disturbance as a key factor shaping community structure, but they do not explicitly deal with stressful systems with non-lethal conditions. We evaluated the mainstream ideas of niche differentiation and stochasticity for the coexistence of plant species across salt marsh creeks in southwestern Denmark. The results showed that diversity indices, such as Shannon–Wiener diversity, richness, and evenness, decreased with increasing surface elevation and increased with increasing niche overlap and niche breadth. The two niche parameters linearly decreased with increasing elevation. These findings imply a substantial influence of an equalizing mechanism that reduces differences in relative fitness among species in the highly stressful environments of the marsh. We propose that species evenness increases under very harsh conditions if the associated stress is not lethal. Finally, we present a conceptual model of patterns related to the level of environmental stress and niche characteristics along a microhabitat gradient (i.e., surface elevation).

Conclusions

The ecology of stressful systems with non-lethal conditions will be increasingly important as ongoing global-scale climate change extends the period of chronic stresses that are not necessarily fatal to inhabiting plants. We recommend that more ecologists continue this line of research.

Background

General overview

Understanding the mechanisms that maintain species coexistence has long been one of the fundamental goals in plant community ecology (Adler et al. 2007; Chase and Myers 2011; Weiher et al. 2011; Tuomisto et al. 2012; Fukami 2015). A tendency to equate (or at least approximate) coexistence to biological diversity (Chesson 2000) makes such an understanding increasingly urgent in this era of rapid environmental change, where the productivity, diversity, and stability of many ecological system is being reduced (McCann 2000; Hooper et al. 2005; Tilman et al. 2006; Ives and Carpenter 2007; Mougi and Kondoh 2012).

Here, we are broadly concerned with patterns and mechanisms of plant species coexistence under very harsh environmental conditions in a coastal salt marsh system. Harsh conditions can drive coexistence-related processes, such as biological interactions (e.g., predation, competition, and facilitation), ecological succession, dispersal, speciation, and resource partitioning. In this time of rapid climate change, many ecosystems are experiencing profound alterations to ambient physical conditions, which can increase the influences of harshness on inhabiting organisms. An example is found in coastal areas where recent sea-level anomalies have led to the increased frequency, duration, and depth of submergence by saline water (Morris et al. 2002; FitzGerald et al. 2008; Nicholls and Cazenave 2010; Sallenger et al. 2012). Similar climatic effects include extended dry periods in arid environments (Holmgren et al. 2006; Seager et al. 2007; Jiménez et al. 2011) and increased fire activities in the western US forests (Westerling et al. 2011). We propose that it will become increasingly important to understand the mechanisms of species coexistence under highly stressful conditions in order to fully understand plant community structure and function, and to develop appropriate management strategies (Callaway et al. 2002; Kelly and Goulden 2008; Anderegg et al. 2013; Walter et al. 2013).

Numerous investigations have been conducted to enhance our knowledge of the patterns and mechanisms of species coexistence, in the contexts of biological interactions among different species and environmental stress gradients (Wright 2002; Silvertown 2004; Tokeshi 2009; Siepielski and McPeek 2010; Gravel et al. 2011; Cho et al. 2019; Jung et al. 2019). As a result, there are more than 100 denominated theories developed from various ecological systems (Palmer 1994). An exhaustive review of these ideas is not the aim of this paper. Rather, we focused on a few contending but related theories that are relevant to both harshness and the particular environmental setting of the study system (see “Study area” section). For example, the intermediate disturbance hypothesis (Connell 1978) may not be a robust model because we investigated a marsh system that is very stressful but experiences little influence from disturbance events. In addition, given the absence of significant herbivores at the marsh, theories associated with predation are not explicitly considered.

Theory I: coexistence maintained by niche differentiation

The most widely accepted theory on species coexistence in traditional plant ecology is based on a simple statement, only those species differing sufficiently in resource use can coexist, a corollary of which is, two (or more) species cannot coexist in the same niche (Gause 1934). This theory appears to be adopted by many scientists as a paradigm (e.g., Silvertown 2004; Mason et al. 2011; cf. Tilman 2004), rather than merely as a hypothesis. It is even considered “perhaps the only specific principle or law of nature ever to be proposed in ecology” (Vandermeer 1972: p. 110)The core of niche theory lies in the concept of competitive exclusion. In accordance with Darwin (1859) and Gause (1934), classic ecologists, such as Hutchinson (1959), May and MacArthur (1972), Pianka (1974), Whittaker (1977), and Diamond (1975), suggested that competition plays a major role in determining community structure and diversity by impeding species coexistence. In order to coexist in a same habitat, species should differ in their utilization of resources (e.g., nutrients, water, light), so that they avoid some critical level of interaction. This difference in resource demand has been termed character displacement (Brown and Wilson 1956; Hutchinson 1959), limiting similarity (MacArthur and Levins 1967; Abrams 1983), species packing (MacArthur 1972), or more often, niche differentiation (or diversification, separation, divergence; Connell 1978). If niche differentiation is not achieved, competitive exclusion will eventually hamper species coexistence. Three conditions are required to meet such a case (Aarssen 1989):

• the demands of two species on resources sufficiently exceed the supply

• the two species make demands on a sufficiently similar set of resource units

• one of the species possesses a sufficiently greater competitive ability for these particular resource units.

Theory II: coexistence without niche differentiation

In the 1970s, some ecologists began to doubt the validity of the niche-based idea, that is, the concepts of competitive exclusion and niche differentiation. This new view was prompted by two factors. First, there was serious lack of empirical evidence that articulated whether coexistence was truly maintained only by niche differentiation, and whether competitive exclusion (and also reduced coexistence) was inevitable if niche differentiation was not sufficient (Wiens 1977; Mahdi et al. 1989; Holyoak and Loreau 2006; Leibold and McPeek 2006; Clark 2009). Specifically, few studies have provided clear-cut data showing that the degree of niche separation is compatible with the observed number of species (but see Russell et al. 1985).

Second, the coexistence of plant species in a certain habitat might be considered not in the context of niche differentiation but, paradoxically, niche overlap (i.e., coincident utilization of one or more resources by two or more species, sharing resources). Higher plants are generally immobile, and thus lack any real choice in resources. By and large, they make demands on virtually the same resources, such as light, nutrients, and water. Therefore, one can justifiably assume that there could be significant overlapping of niches among the coexisting species in many plant communities. If a group of species occur together, it is very likely that they are ecologically and functionally similar (i.e., niches overlap) because, otherwise, they would not have adapted to the habitat and coexisted in the first place. These similarities imply comparable morphology, height, expected life time, and rate of seed production, which in turn would result in similar abilities among the coexisting species to compete for resources (Ågren et al. 1984). This view of retarded competitive exclusion and continued species coexistence due to similar competitiveness has long been conceptualized as competitive combining ability (Aarssen 1983, 1989, 1992) and balanced competition (Zobel 1992).

Aarssen (1983, 1984) tried to put this issue into an evolutionary perspective. Basically, he argued that natural selection processes enhance relative competitive ability rather than niche differentiation, thereby maintaining a balance of competitive abilities for contested resources among species. By this logic, individuals of a superior species, instead of outcompeting those of an inferior counterpart, suppress themselves through intra-specific competition. In the meantime, the weaker competitors increase their fitness, which eventually leads to a reduced difference in the relative competitive abilities between the two species (see also Stoll and Prati 2001; Vellend 2006; Fridley et al. 2007; Whitlock et al. 2007; Silvertown et al. 2009). Below is a part of Aarssen’s (1983, p. 725) conclusion:

All biological attributes of a species will play a part in determining its niche requirements…, and its relative competitive ability for those requirements…niche requirements generally overlap broadly in plants,…coexistence is permitted because there are numerous possible permutations and combinations of biological attributes in plants which are roughly equivalent in the overall competitive power…these attribute complexes are continually adjusted in an ongoing process of coevolution in which local neighborhoods are constantly engaged in a fine-tuning process…Consequently, natural selection ultimately maintains a balance of overall relative competitive abilities between two species for essentially the same contested resources.

Species’ engagement in the fine-tuning process (i.e., localized competition sensu Shmida and Ellner 1984) implies that they do not necessarily avoid competition to avoid competitive exclusion. Similar doubts about the robustness and general applicability of niche differentiation and competitive exclusion were also raised by Grubb (1977); regeneration niche), Shmida and Ellner (1984); stable coexistence of trophically equivalent species), and Hubbell (2001); unified neutral theory of biodiversity). In particular, Hubbell’s (2001) neutral theory assumes all species to be demographically and ecologically identical. Hence, the structure of community assembly arises mainly from neutral stochastic processes, such as dispersal, chance colonization, random extinction, ecological drift, and priority effects (Bell 2000; Holyoak and Loreau 2006; McGill et al. 2006; Mutshinda and O’Hara 2011). Aarssen and Hubbell have long agreed that their theories amount to essentially the same thing in terms of accounting for why species are able to coexist (Aarssen personal communication via e-mail April 5, 2019).

Theory III: coexistence along environmental stress gradients

Environmental stress models (e.g., Menge and Sutherland 1987; Bruno et al. 2003; Scrosati and Heaven 2007) recognize that the level of physical stress significantly affects the behavior of individual species, modes of biological interactions, and eventually, patterns of species coexistence and diversity in a plant community. One traditional belief in this regard has been that competition should be more intense in the benign and productive side of an environmental gradient, whereas the intensity decreases toward harsh conditions (i.e., stress gradient hypothesis; Bertness and Callaway 1994; Maestre et al. 2009; Holmgren and Scheffer 2010; Lee et al. 2018a, 2018b). In the former case, a (few) strong competitor(s) will outcompete inferior ones, thereby resulting in low-level diversity. In the latter, very high stresses allow only highly tolerant species to survive (i.e., reduction of species pool), and thus, species diversity is expected to be low. Intermediate-level harshness is predicted to compromise these two extreme situations and maximize biodiversity because a certain level of stresses prevents a superior competitor from monopolizing available resources, and such stresses secure space for stable populations of inferior species. In sum, retarded competitive exclusion because of physical harshness has been considered a major factor contributing to the maintenance of species coexistence (Grime 2001; Hart and Marshall 2013). Indeed, this notion provides the basis for influential classical theories of non-equilibrium coexistence, such as patch dynamics (Pickett 1980), the intermediate disturbance hypothesis (Connell 1978), and Huston’s (1979) dynamic equilibrium model.

There are at least two theories associated with niche differentiation, indicating that one can observe high biodiversity even under highly stressful conditions. First, Tilman’s (1985) resource-ratio hypothesis suggests that high-level species coexistence can be achieved at low resource levels because the coexisting plants experience their environment as inherently spatially variable more than at high levels of resources. In a somewhat different context, Chesson and Huntly (1997) propose that stable coexistence may result easily from classical resource partitioning when strong environmental harshness makes existing tolerant species more equal in demographic rates. They conclude that, although harshness itself cannot stabilize species coexistence, it may facilitate or broaden the effects of other diversity-stabilizing mechanisms, such as resource partitioning and niche differentiation. In addition, facilitative biological interactions can be another important mechanism of species coexistence under environmental harshness (He and Berness 2014), as has been demonstrated in salt marshes (Bertness and Shumway 1993), arid ecosystems (Maestre and Cortina 2004), and alpine ecosystems (Callaway et al. 2002).

Theory IV: coexistence under non-equilibrium versus equilibrium conditions

Most of the environmental stress models discussed above exhibit one common characteristic that an important (and often imperative) component is the role of disturbance in shaping community structure and patterns of species diversity. Such a strong emphasis on disturbance implies that species coexistence is maintained under non-equilibrium states, in which disturbance events are frequent and strong enough to prevent a system from returning to its previous ecological phase before another disturbance event occurs (Connell 1978; Chesson 2000; Araújo and Pearson 2005). Consequently, a question arises concerning the general applicability of current theories of coexistence to communities that exhibit stressful conditions but limited and infrequent disturbances, that is, communities where a degree of equilibrium might be attained.

Many theorists have been optimistic that disturbance-related models will be applicable to many different types of communities (but see Fox 2013). However, for example, Locke’s (1992) classic analyses of zooplankton communities along pH gradients (i.e., acidification stress) did not support such an expectation. She argued that it is questionable whether the level of physical disturbance included in the conventional models is comparable to the actual level of stresses that plant species perceive and respond to (see also reviews by Mackey and Currie 2001; Hughes et al. 2007). In short, the traditional disturbance-based models may not be robust if species in a system experience non-lethal environmental harshness.

Theory V: modern perspectives

Although the validity of niche theory has been recognized in community ecology, it also has limitations. First, it is suggested that the niche itself is an abstract concept (Chase and Leibold 2003; Araújo and Guisan 2006; Soberón and Nakamura 2009). Ambiguities in the original formulation of fundamental and realized niche concepts often result in conflicting interpretations as to what is truly a niche in the ecological system (Araújo and Guisan 2006). Furthermore, niche theory assumes the existence of stable conditions in, for example, required resources for species, but whether such conditions actually exist or are readily detectable in reality remains elusive (Torres 1984).

Neutral theory has also faced several criticisms, arising mainly from its assumption that species are functionally equivalent and thus all individual species within a community have the same chances of reproduction and mortality regardless of their characteristics (Theory II: coexistence without niche differentiation). In real-world communities, many species exhibit large differences in traits. Therefore, if neutrality holds, these differences should be prone to fitness-equalizing tradeoffs, which is extremely unlikely (Purves and Turnbull 2010). In addition, as Hubbell (2001) acknowledged, stochastic processes associated with neutral theory do not predict coexistence patterns well at local spatial scales (Jabot and Chave 2011; Pos et al. 2019). Related to this is the question of whether perceived neutral processes might result from non-neutral mechanisms (Alonso et al. 2006). This is because neutral theory aggregates and averages the information on community assembly processes across sites, and this will blur deterministic mechanisms, making them appear as neutral dynamics (Clark 2012). However, in the last two decades, the neutral theory perspective has often been used to explain community assembly especially at regional scales (Chave 2004; Rosindell et al. 2011, 2012; Black and McKane 2012; Chesson 2013; Matthews and Whittaker 2014; D’Andrea and Ostling 2016). In addition, due to its simplicity and tractability, the theory still serves as a useful null hypothesis for complex biodiversity patterns (Rosindell et al. 2012; Matthews and Whittaker 2014; Munoz and Huneman 2016).

Despite its drawbacks in assumption, prediction, and measurement (Pocheville 2015), niche theory has also advanced in the new millennium (Adler et al. 2007; HilleRisLambers et al. 2012; Matthews and Whittaker 2014; Munoz and Huneman 2016; Letten et al. 2017; Barabás et al. 2018). Ambiguous definitions of niches have been redefined through quantitative approaches of modern coexistence theory (Chesson 2000; Letten et al. 2017). Modern coexistence theory is based on the Lotka–Volterra model and examines community structure in relation to the influence of environmental factors on species interactions (Chesson 2000, 2013; Adler et al. 2007; Ellner et al. 2019). In a theory-based model, complex interactions between species in a community are simplified into competition coefficients, which reflect intra- and inter-specific effects on per capita growth rate (Letten et al. 2017). Under modern coexistence theory, niche and fitness differences are controlling factors that allow species to coexist in the same community. The niche difference is part of a stabilizing mechanism that differentiates niches among species within a community to decrease competition and thus optimize the chance of survival. On the contrary, the equalizing mechanism balances fitness differences between species, thereby contributing to species coexistence (HilleRisLambers et al. 2012). The stabilizing niche differences can be reduced as the average fitness differences decrease. In other words, when niche differences are larger than relative fitness differences, species in a community can stably coexist (Chesson 2000).

Contemporary niche theory is developed from the consumer–resource model (Tilman 1982) and provides an explicit basis for coexistence (Chase and Leibold 2003). While Tilman’s model focuses on the dynamics of the consumer–resource system, Chase and Leibold (2003) expand the range of this model from interactions between biotic factors to interactions between biotic and abiotic factors (Letten et al. 2017). A major difference between modern coexistence theory and contemporary niche theory lies in the ways they define niche. For modern coexistence theory, boundaries of niches are defined through an interaction between two or more species. Under contemporary niche theory, in contrast, niches indicate particular traits of an individual species, and the niches of multiple species within a community can sometimes overlap.

In recent years, there has been a growing body of researchers trying to integrate the niche and neutral perspectives (Adler et al. 2007; Chisholm and Pacala 2010; Pigolotti and Cencini 2013; Kalyuzhny et al. 2015). In particular, a number of studies have demonstrated theoretically and empirically that both niche-based stabilizing mechanisms and neutral-based equalizing mechanisms appear to operate simultaneously along the continuum of environmental stress gradients in many ecological systems (Gravel et al. 2006; Mutshinda and O’Hara 2011; Jeraldo et al. 2012; Matthews and Whittaker 2014; Scheffer et al. 2018; Ellner et al. 2019; Kim 2019). Accordingly, the focus today is increasingly on understanding the relative importance of these two mechanisms for species composition and diversity under different environmental conditions (Smith and Lundholm 2010; Chase and Myers 2011; Tuomisto et al. 2012). In particular, it is recently proposed that deterministic processes gain importance and stochastic processes lose importance as a community enters a regime of more stressful conditions (e.g., Chase 2007, 2010; Guo et al. 2014; cf. Lepori and Malmqvist 2009). That is, if a system is under harshness, only a small subset of the regional species pool can pass through the associated environmental filters. Because the resulting species richness is low, the probability of ecological drift will reduce, leading to a community likely assembled by niche-driven processes (i.e., weak stochasticity) (Kim 2019).

Summary

This review of the major coexistence theories provides four insights. First, there are few general models that can be applied to many ecological systems with convincing results. Even the widely accepted niche differentiation hypothesis has faced strongly opposing viewpoints (especially from the competitive combining ability hypothesis) over decades. The contrasting theories of niche differentiation and competitive combining ability (or neutrality) are the key interests of this present work. Second, the lack of general models implies that ecologists are still far from understanding the mechanisms of species coexistence under harsh environmental conditions. Much remains to be discovered (e.g., coexistence by or without niche differentiation), beyond the hypothesis that the intensity of competitive interactions weakens near the stressful end of physical gradients. Third, due to the conventional emphasis on disturbance, stressful systems with non-lethal conditions have not received the attention they deserve. The ecology of such systems will be increasingly important as global climate change extends the period of chronic stresses that are not necessarily fatal to inhabiting plants. Finally, for decades, theories have been presented, but further empirical insights are required based on field data to test the various coexistence models at different spatial scales and in different ecosystems. We do not claim that our current research resolves these four issues. Rather, this paper provides a set of observational species composition data from a single marsh system, in which plant communities experience non-lethal stresses along an environmental gradient; it should be considered a springboard for more extensive experimental research in various ecological systems.

Hypothesis

In this research, we address a fundamental question, what are the patterns and possible mechanisms of plant species coexistence under very harsh but non-lethal environmental conditions? To rephrase, the community to be tested is not directly influenced by high-magnitude disturbing forces but shows a certain level of “well-defined, autogenic developmental trends” (Odum 1985). We broadly hypothesize that there will be an overall trend in species diversity along environmental stress gradients at the study marsh. In particular, following the general explanations of the environmental stress models explained above (Theory III: coexistence along environmental stress gradients), we expect that the highest diversity will be observed at the intermediate-level stresses along the environmental gradient we examine. However, this hypothesis might be rejected if, for example, we observe unexpectedly high species diversity even under very stressful conditions.

Pianka (1974), in his classic niche overlap hypothesis, predicted that the maximum tolerable niche overlap between two species should be an inverse function of the intensity of inter-specific competition. Therefore, both decreasing overlap (i.e., niche differentiation) and increasing species diversity will indicate highly intense diffuse competition (i.e., native species in a community compete among themselves). A corollary of this idea is that high-level species diversity implies high-level diffuse competition (MacArthur 1972), which in turn significantly lowers niche overlap (see also Russell et al. 1985; Silvertown 2004). We embody these ideas as a conceptual model along a resource gradient at different levels of species diversity (Fig. 1). In the model, we hypothesize that high diversity is maintained by niche differentiation and is associated with intense diffuse competition. In the case of low diversity, reduced competition allows niche overlap (i.e., resource sharing) to increase. Assuming that the three systems with different levels of species diversity in the model possess the same extent of resource niches, we expect that niche breadth should be high where diversity is low and low where diversity is high. Niche breadth is a measure of the evenness of resource utilization by species along given niche dimensions (Levins 1968). High and low niche breadths (i.e., high and low evenness) are, therefore, indicative of generalist and specialist species, respectively (Vandermeer 1972; Gravel et al. 2006; Finke and Snyder 2008).

Fig. 1

A conceptual model of niche characteristics along a resource gradient at different levels of species diversity

Materials and methods

Study area

The study area is a barrier spit, located on the Skallingen peninsula in southwestern Denmark (Fig. 2a). It is characterized by geomorphological zonation typical of the Wadden Sea islands. From the ocean (west) to the backbarrier lagoon (east), the depositional sequences are as follows: beach, dune, salt marsh, and tidal flat. The southwest-facing beach and sand dune of the spit are directly influenced by westerly storm surges that often cause erosion and loss of habitat. However, the backbarrier salt marsh is sheltered from the effects of such destructive events, as the surge waves pass through a relatively narrow tidal inlet between the southern end of the peninsula and an island, Fanø, so most wave energy is lost before they arrive at the backbarrier shoreline (Kim et al. 2011a). Even during storms, the wave climate at the backbarrier area of Skallingen is relatively calm (Bartholdy and Aagaard 2001; Kim et al. 2011b, 2013). The area is classified as micro-tidal: the tidal range is about 1.7 m at spring and 1.3 m at neap tides with a mean of 1.5 m.

Fig. 2

a The location of the Skallingen salt marsh in southwestern Denmark. The inset indicates b showing the distribution of 11 point bars selected across the marsh. c Sampling design at each bar. d and e are the photos of P4 and P5, respectively

The main focus of this research was on plant species coexistence under very harsh physical conditions. While salt marshes themselves are conventionally considered stressful environments (Adam 1990; Bertness et al. 2001), even these systems include high locations with relatively benign conditions (i.e., low frequency of sea water inundation), allowing less tolerant inland plants to establish. Therefore, we restricted our interest to point bars of tidal creeks that experience frequent inundation by saline water during bidirectional tidal flows. The Skallingen marsh has a natural tidal creek system across its platform (Kim et al. 2010; Kim 2018). Considering the creek’s continuous migration at Skallingen (Kim 2012, 2014), the importance of tidal creek processes to the whole marsh ecology is expected to grow. Moreover, there are a variety of point bars shaped by fluvial–geomorphic processes, and these bars show varying sizes and sinuosity. In this regard, we consider that the marsh provides a good laboratory for studying plant species coexistence and diversity in very stressful tidal channel conditions.

Previous literature has identified some potential disturbing forces around creek banks, such as wrack deposition that smothers underlying vegetation (Bertness and Ellison 1987) and cut bank erosion that may induce vegetation collapse (D’Alpaos et al. 2007; Hughes et al. 2009). However, we rarely found evidence of a wrack deposition during field work, and the effect of bank erosion is beyond the scope of this study. The point bars we investigated are characterized primarily by sedimentary processes that continuously create new potential space for the establishment of pioneer plants. Due to their very low surface elevation, the bars undergo submergence by sea water driven by semi-diurnal tidal flows along channels, as well as limited nutrient availability. These factors are perceived predominantly as chronic environmental stresses rather than abrupt, destructive disturbing forces. In short, we propose that there is little significant effect of disturbance in the creek system at the Skallingen salt marsh.

Data acquisition in the field

We selected 11 point bars across various tidal creeks on the Skallingen salt marsh (Fig. 2b). The point bars were carefully chosen based on several field reconnaissance trips across the marsh. In particular, we tried to select only those bars that looked representative of all bars at Skallingen, in terms of size, sinuosity, channel width, and depth (e.g., Fig. 2d, e). In each bar, two replicate 1 m × 1 m square quadrats were established at the lower end of vegetation (Fig. 2c). Each quadrat was subdivided into 25 small grids of 0.2 m × 0.2 m, and in each of the grids, we recorded the presence of vascular plant species. Species nomenclature followed Tind (2003). Each quadrat was vertically divided into five classes, so the 5 grids (0.2 m × 0.2 m) in a class represented the same height level. In each class, the frequency of each species varied from 0 to 5. Topographic surveys were conducted at the midpoint between the replicate quadrats using a differential Global Positioning System (Trimble R4 GPS Receiver and Trimble Recon® Controller). The device produced errors in elevation less than 0.02 m at each point sampled.

Data analysis

Before estimating diversity and niche indices, a hierarchical cluster analysis was conducted to classify the selected point bars into ecologically meaningful groups. A combination of Sørensen’s distance measure and flexible Beta (β = − 0.25) was used (McCune and Grace 2002). All samples were first relativized to make the observational units equitable in species abundance and to enhance the detection of broad compositional similarities among samples. After cluster analysis, we used indicator species analysis to quantitatively choose the optimum number of groups (Dufrene and Legendre 1997). Based on the final indicator values for each species that were calculated by multiplying its relative abundance and relative frequency by the group, the significance of the highest indicator value for a given species across groups was evaluated by 5000 runs of Monte Carlo tests. The resulting p values were utilized as an objective criterion for pruning the cluster dendrogram. The step with the smallest average p value was judged to be the most informative level in the dendrogram (McCune and Grace 2002).

In this research, we sought to address whether niche differentiation is required for species coexistence using two niche parameters: niche overlap and niche breadth. As these indices are associated with patterns of resource sharing among species, the first task should be to identify the major resource dimensions (e.g., light, water, and nutrients). However, this issue is regarded as one of the most challenging jobs in the study of niche theory and species coexistence (Mason et al. 2011). Furthermore, much of the published literature, even after reporting very high niche overlap along a certain niche axis, acknowledges that there still is a possibility that intensive competition and reduced niche overlap might be occurring along other dimensions (e.g., Colwell and Futuyma 1971; Mahdi et al. 1989; Gotelli and Entsminger 2011).

A number of factors can potentially control the spatial variability of physical conditions and vegetation in salt marshes, but the conventional paradigm suggests that tidal flooding, mediated by marsh elevation in relation to high water statistics, constitutes the fundamental factor that explains most other environmental variations (Pennings et al. 2005; Silvestri et al. 2005; Davy et al. 2011; Lee et al. 2018a, 2018b; Nam et al. 2018; Kim 2019). A corollary of this paradigm is that there is a single prominent and overriding stress gradient represented by surface elevation that governs the zonal patterns of physical variables and vegetation. This emphasis led Russell et al. (1985) to select elevation as the main niche dimension in their evaluation of the niche overlap hypothesis in two English salt marshes.

Following this example, we estimated niche overlap and niche breadth along the height dimension at each selected bar. This strategy is supported by our previous findings: Kim et al. (2012) performed intensive analyses on soil–plant–topography relationships across tidal creeks to demonstrate that soil nutrient variables (e.g., N, P, and K) had a surprisingly minor influence on the spatial pattern of vegetation, whereas surface elevation was the primary determinant of both edaphic and floristic patterns. Of course, the overriding effects of elevation might be somewhat blurred when there is micro-scale heterogeneity in sediment properties, such as texture, hydraulic conductivity, and drainage conditions (Davy et al. 2011). However, this current research was concerned only with point bars that are characterized by extremely uniform sandy substrates. These sediments have been newly deposited with little time for significant compaction or for micro-scale differentiation of physical and chemical properties. Furthermore, Schoener (1974) argued that microhabitat is one of the three most important niche axes (the other two are diet and temporal activity) and that most differentiation should occur along microhabitat gradients. The microhabitat corresponds to different height classes in this study (Fig. 2c). Thus, strictly speaking, we are measuring the overlap of microhabitats with different elevations among species, not the degree of true resource sharing per se (see also Gotelli et al. 1997). However, as justified above, it is assumed that this height dimension will successfully represent the main resource axes that are present in the marsh.

At each bar, we set the lower end of vegetation as the reference point (Fig. 2c). Although these elevations may differ from one another in terms of their absolute values in m DNN (Danish Ordnance Zero), they are still the same in that they commonly represent the threshold condition at which pioneer plants are able to colonize. Some bars are expected to show the presence of plants even at extremely low sites, while other bars exhibit pioneer species only at their highest portions. In the former case, pioneers should experience highly frequent sea water inundation (i.e., extremely harsh conditions). In the latter, they are under more moderate conditions. Accordingly, we were able to examine how niche and diversity indices varied along a gradient from the highest to lowest stresses represented by surface elevation.

We were concerned with the different diversity indices of each quadrat because diversity often indicates the level of competitive effects and is crucial for evaluating the hypothesis of this study (Fig. 1). Diversity includes at least two different aspects of community structure: richness and evenness. Richness refers to the number of different species present in a community, and evenness indicates their relative abundance. As Scrosati and Heaven (2007) pointed out, there were two related issues with the use of diversity indexes in the ecological literature: (1) the interchangeable use of diversity and richness and (2) the lack of attention to and inclusion of evenness (Mackey and Currie 2001; Kimbro and Grosholz 2006). Recently, ecologists have recognized that these two issues should be resolved to better understand patterns of biodiversity (Bruno et al. 2003; Scrosati and Heaven 2007; Kim et al. 2013). In this research, we examined trends in both richness and evenness across different bars.

Richness (S) was determined by simply counting the total number of species present in each quadrat. Then, we calculated the Shannon–Wiener (S–W) index (H′) to represent the level of species diversity in each quadrat:

$$ H^{\prime }=-\sum {p}_i\left(\mathit{\ln}\ {p}_i\right) $$

where p_i is the proportion of each species in each quadrat. We represented the evenness of each quadrat using Pielou’s (1969) evenness index (J′):

$$ J^{\prime }=H^{\prime }/H{\prime}_{max} $$

where H′_max is the maximum value of S–W diversity, equal to ln(S). We evaluated the relationship between these three diversity indices and surface elevation using a simple regression analysis to examine whether there was an overall trend in species diversity along environmental stress gradients and, in particular, whether the highest diversity was observed with the intermediate-level submergence stress. The diversity values estimated from the replicate quadrats were averaged before regression.

Finally, based on the frequency data in each height class (Fig. 2c), we estimated niche (i.e., microhabitat) overlap (O) using Pianka’s (1973) formula:

$$ {O}_{ik}={O}_{ki}=\frac{\sum_{j=1}^{j=n}{p}_{ij}{p}_{kj}}{\sqrt{\sum_{j=1}^{j=n}{p}_{ij}^2{\sum}_{j=1}^{j=n}{p}_{kj}^2}} $$

where O_ik and O_ki are mutual habitat overlaps of species i and k, respectively, p_ij is the abundance of species i in height class j, and n is the number of classes into which the height has been divided. The index ranges from zero (no habitats used in common between two species) to one (complete overlap in habitat use). Niche (i.e., microhabitat) breadth (B) was calculated for each species using Levins’ (1968) equation:

$$ {B}_{ij}=\frac{1}{\sum_{j=1}^{j=n}{p}_{ij}^2} $$

All calculations of overlaps and breadths were based on the proportional occurrences of species in each height class. Therefore, niche parameters were not influenced by differences in the absolute abundance of species among various point bars, and direct comparison of these parameters is appropriate. Species richness, evenness, and S–W diversity were regressed against niche overlap and niche breadth to evaluate the hypothesis of this research (Fig. 1).

Results

The surface elevation where plants started to grow (i.e., the lower end of vegetation) greatly differed among the 11 bars surveyed (Figs. 3 and 4). For example, the vertical difference in the lower limits of vegetation between bars 1 and 11 was 0.23 m.

Fig. 3

The elevation (in m DNN; Danish Ordnance Zero) at which the lower end of vegetation occurred at each point bar. Each dot corresponds to the line “lower limit of vegetation” presented in Fig. 2c

Fig. 4

The occurrence probability (%) of each plant species along the gradient of surface elevation (DNN, Danish Ordnance Zero)

We identified seven vascular plant species: Spartina anglica C. E. Hubb, Salicornia herbacea L., Suaeda maritima L., Puccinellia maritima (Hudson) Parl., Aster tripolium L., Limonium vulgare Mill. and L. humile Mill., and Atriplex portulacoides L. The cluster analysis classified the point bars into four groups: bars 1, 2, and 3 belonged to group A. Bar 4 comprised group B. Group C contained bars 5, 6, 7, and 8, and the other bars were classified into group D. The four groups had the same species richness (i.e., seven); however, there were substantial compositional variations across the groups (Fig. 5). In particular, Spartina showed some degree of occurrence probability (> 10%) in groups A and B, but it was extremely minor in groups with higher surface elevations (see Fig. 3).

Fig. 5

Species composition in the vegetation associations classified by hierarchical cluster analysis

S–W diversity, species richness, and evenness exhibited strikingly linear and significant relationships with bar surface elevation, niche overlap, and niche breadth (Fig. 6). The diversity indices decreased with increasing elevation, whereas they were positively correlated with the niche parameters. The niche parameters significantly decreased with increasing bar surface elevation (Fig. 7).

Fig. 6

Diversity indices predicted by bar elevation (in m DNN; Danish Ordnance Zero), niche overlap, and niche breadth

Fig. 7

Niche indices predicted by bar elevation (in m DNN; Danish Ordnance Zero)

Discussion

Stress and diversity

It is noteworthy that, although the pioneer zones of all selected bars were undoubtedly considered very stressful, there were substantial variations in the level of harshness among the zones. These bars, despite having undergone the same hydrogeomorphic processes (i.e., deposition of coarse sediments; Kim et al. 2010, 2013) and possessing similarly low surface elevations (Kim et al. 2012), were characterized by greatly different elevations at which pioneer plants start to grow (Fig. 3). The elevation difference in the lower ends of vegetation between bars 1 and 11 (i.e., 0.23 m) is considered remarkably large in ordinary salt marshes, given that soil properties and species composition respond significantly to even a few centimeters of topographic and consequently hydrologic changes (Adam 1990; Kim et al. 2012). The results of this study indicate that the pioneer zones of the selected bars are located at different surface elevations, encompassing extremely harsh and slightly less harsh conditions. Therefore, the creek bars of the Skallingen salt marsh are excellent laboratories for evaluating the original research questions of this study, concerning mechanisms of species coexistence along the gradient of environmental harshness (see “Hypothesis” section).

We consider that a definable diversity pattern exists along the stress gradient represented by surface elevation, and that high-level biodiversity is related to physical harshness in the study area. The linear increases of diversity indices toward the lower end of elevation (left column of Fig. 6) imply that strong harshness might be a driving force of species coexistence and high diversity. Therefore, it is plausible that the ecology of the Skallingen bar system does not fit into the traditional notion of environmental stress models which posit that an intermediate-level stress can promote species coexistence and diversity (see “Theory III: coexistence along environmental stress gradients” section). In particular, the high evenness under harsh conditions needs to be examined to fully understand the possible mechanisms of species coexistence in low-elevation bars.

It has long been thought that environmental harshness drives low species evenness. From diverse stream systems in a California watershed, Bendix (1997) found that large flood events removed most of a given species from a site, thereby reducing evenness. In a similar context, Scrosati and Heaven (2007) considered that, under the high physical stress of rocky intertidal seashores, evenness should become low because only a small fraction of all the species present would tolerate the harsh conditions, with the remaining few species being scarce. However, the patterns we identified at the point bars at Skallingen are in apparent disagreement with these previous findings.

It should be noted that, in previous research, the harsh conditions often refer to, or at least include, disturbances that cause sudden mortality in a given system, be it partial or complete. However, the stressful factors in the sheltered backbarrier marsh at Skallingen are explained predominantly by regular submergence (Study area), and these stresses are not necessarily lethal to pioneer plant species surveyed. In Skallingen, we found that high-level stress caused evenness to increase. This accords well with the ideas of Tilman (1985) and Chesson and Huntly (1997) introduced in the “Theory III: coexistence along environmental stress gradients” section.

The seven species surveyed in this research are survivors of stressful physical filters that are primarily imposed by seawater inundation. These species are generally adapted to the ambient environmental stresses (Sánchez et al. 1996; Costa et al. 2003), and therefore, they are potentially capable of colonizing any site within a bar. At the lowest elevation near 0.6 m DNN, these plants seem to facilitate each other, resulting in high species richness and evenness (Fig. 6), in accordance with the stress gradient hypothesis (see “Theory III: coexistence along environmental stress gradients” section). However, species richness clearly decreased as the bar elevation increased, a pattern most likely induced by competitive exclusion (Kim 2019). Due to environmental harshness, there was no opportunity for a dominant species that monopolized available resources to reduce both evenness and diversity. In a similar context, chronic stresses are known to suppress the dominance of a particular (set of) species, and retard competitive exclusion, thereby maintaining high species richness (Peet et al. 1983; Shmida and Ellner 1984). At Skallingen, we empirically identified a more even distribution of different species in more stressful bars than in slightly higher-elevation bars. Of course, the results might vary if mid to late-successional species had been considered. For example, as surface elevation increases toward channel levees, plant species having greater competitive abilities will survive, reducing diversity and niche parameters (Kim et al. 2013).

Diversity and niche parameters

The basic question of this study was whether high species diversity was maintained by niche differentiation that indicated low niche overlap. At the Skallingen salt marsh bar system, we found a surprising pattern where both niche overlap and breadth were high even in the case of high species diversity (middle and right columns of Fig. 6). This positive relationship conflicts with the traditional niche theory (Theory I: coexistence maintained by niche differentiation). The current results seem to support the theory of competitive combining ability or balanced competition (Aarssen 1983, 1984; Zobel 1992; Theory II: coexistence without niche differentiation). The finding that diversity indices, niche overlap, and niche breadth were all high at highly stressful sites advocates a possibility that different species tend to possess similar competitive abilities, rather than avoiding competition through niche differentiation. Moreover, species at lower elevations possibly enhance the relative fitness of other species by limiting their own fitness. This implies a strong effect from the equalizing mechanism that reduces differences in relative fitness among species (Chesson 2013; Theory V: coexistence under non-equilibrium versus equilibrium conditions). In short, balanced competitive ability and equalizing fitness differences appear to be key to the success of each pioneer species, and hence, the observed high evenness. Finally, based on the decreasing niche indices with increasing elevation (Fig. 7), we posit that the relative importance of niche-based deterministic and stochastic mechanisms varies along the stress gradient in this study: At low-lying, more stressful sites, stochastic processes seem to be dominant, whereas deterministic niche differentiation likely operates at higher, less stressful sites (Kim 2019).

These observed ecological patterns and discussions led us to revise the original hypothesis and develop an alternative conceptual model for the study marsh (Fig. 8). In the past, one limitation of the concepts of competitive combining ability and balanced competition has been a lack of empirical data acquired by observational approaches in the field. While the proposed new model has been developed from a single system, we believe that it still has the potential to support these underrepresented classical theories (Aarssen 1983, 1989; Zobel 1992). More number of such studies should be conducted in different ecological systems.

Fig. 8

Observed patterns in the level of environmental stress and niche characteristics along a gradient of microhabitat (i.e., surface elevation) at different levels of species diversity, Skallingen, Denmark

Conclusions

In this paper, we have shown that different diversity indices consistently had negative relationships with surface elevation and positive relationships with niche overlap and niche breadth. These two niche metrics linearly decreased with increasing elevation. These results were considered to reflect the importance of an equalizing mechanism that reduced differences in relative fitness among species in the study marsh.

In coastal environments, the potential impacts of recent climate change on ecological patterns and processes have been a central issue, not only for scientists but also for coastal zone managers, policy makers, and the general public (Feagin et al. 2010; Kirwan et al. 2010). The impacts are often perceived as profound alterations of surrounding physical conditions, which can be interpreted as increasing the effects of harshness on inhabiting species. For example, increased frequency, duration, and magnitude of waterlogging driven by recent sea-level anomalies are the main foci of a number of initiatives that the USA and European Union (e.g., Western Climate Initiative; Natura2000 Network) seek to include in their integrated coastal zone management plans (EEA 2006). We believe that understanding the mechanisms of species coexistence and the resultant biodiversity under highly stressful physical conditions will be increasingly required as a springboard for fully understanding community structure and function and for developing appropriate conservation strategies. However, because of the limited current knowledge on the ecology of stressful systems, any prediction as to the mode of species interactions and patterns of coexistence under future climate change and altered stress regimes would be difficult. We propose that more researchers investigate the ecology of very harsh systems in various types of ecosystems.