Corn nitrogen rate recommendation tools’ performance across eight US midwest corn belt states

1Univ. of MO, 269 Agric. Eng. Bldg., Columbia, MO 65211 2USDA-ARS Cropping Systems and Water Quality Research Unit, 243 Agric. Eng. Bldg, Columbia, MO 65211 3Purdue Univ., Lilly 3-365, West Lafayette, IN 47907 4Corteva Agrisciences, 7100 NW 62nd Ave., P.O. BOX 1000, Johnston, IA 50131 5Univ. of Nebraska, Keim 367, Lincoln NE 68583 6Univ. of Minnesota, 1991 Upper Buford Circle, St. Paul, MN 55108 7North Dakota State Univ., PO Box 6050, Fargo, ND 58108 8Univ. of Wisconsin-Madison, 1525 Observatory Dr., Madison, WI 53706 9Univ. of Illinois, W-301 Turner Hall, 1102 S. Goodwin, Urbana, IL 61801 10Iowa State Univ., 3208 Agronomy Hall, Ames, IA 50011 11Univ. of MO, 210 Waters Hall, Columbia, MO 65211 12Fortigen, 6807 Ridge Rd, Lincoln, NE 68512


INTRODUCTION
Nitrogen fertilizer inputs are generally necessary for optimizing corn (Zea mays L.) yields, but N is the most challenging plant nutrient to manage optimally. The difficulty arises from biophysical complexity driving soil N mineralization, crop uptake, and N loss (Lory and Scharf, 2003;Meisinger, 1984). The complexity is magnified as N transport and transformation processes vary considerably within and between fields because of spatially variable soil properties and temporally variable weather (Tremblay et al., 2012). Soil variability affecting the N cycle arises from both short-and long-range spatial differences in properties such as texture, organic matter, plant-available water, topography, with a major effect on water redistribution within the landscape, and microbial populations (Dinnes et al., 2002;Parkin, 1987;Scharf et al., 2005;Sørensen & Jensen, 1995;Zhu, Schmidt, Lin, & Sripada, 2009). This complexity challenges farmers to make accurate N fertilizer rate decisions, both between and within fields. Since N fertilizer is typically inexpensive relative to the magnitude of crop N response, farmers most easily deal with this complexity and uncertainty by erring on the side of overapplication (Vanotti & Bundy, 1994). Over applying decreases profitability and increases the potential for N loss that contributes to environmental degradation (Maharjan, Venterea, & Rosen, 2014;van Es, Kay, Melkonian, & Sogbedji, 2007).
Multiple N fertilizer rate decision tools have been developed in an attempt to help farmers make better N management decisions. An extensive review of the history, pros and cons, and current use of many corn N recommendation tools used within the United States has recently been published by Morris et al. (2017). Many of those tools are also included in our investigation (Table 1): (i) mass balance calculations based on an expected yield or yield goal (YG), (ii) preplant soil nitrate test (PPNT), (iii) pre-sidedress soil nitrate test (PSNT) and late-spring soil nitrate test (LSNT), (iv) maximum return to N (MRTN) calculation, (v) Maize-N crop growth model, and (vi) canopy reflectance sensing.
One of the first methods developed in the early 1970s for estimating corn N fertilizer rates was the mass balance approach. Based on information about the N cycle and plant uptake, a value of 0.55 kg ha −1 of added N was estimated to produce 25 kg ha −1 of corn grain (1.2 lbs N bu −1 ; Stanford, 1973; Table 2). This value multiplied by a multi-year expected yield or YG produced the rate recommendation. Limitations of this method have been documented showing that the YG and actual yield do not correlate well with economically optimal N rate (EONR) or optimal N rates (Blackmer, Voss, & Mallarino, 1997;Fox & Piekielek, 1995;Kachanoski & Fairchild, 1996;Lory & Scharf, 2003;Vanotti & Bundy, 1994). Because of these limitations, the YG approach has been discontinued in humid areas where year-to-year weather variations make it difficult Core Ideas • A comparison of 31 corn nitrogen recommendation tools across the US Midwest. • No one nitrogen recommendation tool was universally reliable across the US Midwest. • Across all metrics of success, several of the soil nitrate tests performed the best. • Relative to economically optimal N rate, all tools but Maize-N had similar profitable returns. • Yield-goal based tools resulting in the highest environmental costs.
to predict N availability (i.e., amount of N supplied to the plants through mineralization of organic N and N loss in the environment; Lory & Scharf, 2003). The weakness of YG recommendations has been attributed to the variability of N use efficiency arising from different hybrid or fertilizer types, variable soil N supply, and poor estimation of YG (Lory & Scharf, 2003;Vanotti & Bundy, 1994). To account for these limitations, many state N fertilizer recommendations were modified merely by adjusting the coefficients within the YG equation. Even with these modifications, year-to-year soil and weather variability produced an inconsistent performance for making N fertilizer recommendation with this tool. Therefore, most land-grant universities within the US Corn Belt region discontinued YG N rate recommendations in the 1990s and early 2000s (Morris et al., 2017). Regardless, this tool is still widely used by growers due to simplicity and perception.
Other tools have emerged specifically to address soil N contributions. The PPNT tool measures soil NO 3 -N prior to planting as a credit to the N recommendation (Table 2). This test effectively reduces over-application of N fertilizer in fields that have large residual NO 3 -N concentrations, such as excessively manured fields (Bundy & Andraski, 1995), or fields following drought-like conditions with significant amounts of unused N carried over from previous crops (Meisinger, Schepers, & Raun, 2008). Summarizing, the PPNT tool performs best in medium-to fine-textured soils where the previous year's precipitation was at or below average and when excessive N was applied (Gelderman & Beegle, 1998;Schröder, Neeteson, Oenema, & Struik, 2000). In contrast, this tool is less useful when excessive rainfall after sampling causes either extended periods of ponding (notably on fine-textured soils) or leaching (notably on coarse-textured soils) promoting environmental N loss (van Es et al., 2007). Since sampling occurs prior to planting, PPNT does not account for N mineralization during the growing season, which could result in overfertilization if in-season mineralization is high (Schröder et al., 2000). T A B L E 1 Strengths and weaknesses of N fertilizer recommendation tools included in this investigation (YG, yield goal; EONR, economically optimal nitrogen rate; PPNT, pre-plant nitrate test; PSNT, pre-sidedress nitrate test; LSNT, late spring nitrate test; MRTN, Maximum Return to N)

Tools Pros Cons Citations
Yield goal A mass balance approach that is easily calculated. Nitrogen recommendations can be adjusted to account for soil N using credits .
Poor relationships were observed between YG calculations and EONR due to the uncertainty of final yields, management, previous crop effects, soil N supply, corn and fertilizer prices, and fertilizer use efficiency. Additionally, this method does not account for within-field variability due to soil and water properties. Stanford, 1973;Lory and Scharf, 2003;Sawyer et al., 2006 PPNT Soil NO 3 -N levels can be assessed for residual N and N supplied by manure that could be available for plant use. Can be used as an adjustment to other N recommendations. Sampling can be taken during a lull in seasonal work.
Not a useful tool in more humid regions due to N loss during wet springs. Inaccurate test results due to varying weather affecting N mineralization rates. Additional cost and labor is required. Requires deep sampling, down to 0.60 m or deeper. Magdoff, Ross, & Amadon, 1984;Bundy and Andraski, 1995;Schröder et al., 2000;Lory and Scharf, 2003;van Es et al., 2007 PSNT and LSNT Has potential for better accounting than PPNT of N loss from leaching or denitrification and N inputs from mineralization. Successful at identifying N-sufficient sites.
Additional in-season sampling required and limited by wet conditions and short laboratory turn around. Limited by N loss due to temperature and rainfall immediately before and after sampling. Does not account for within-field spatial variability that results from variable soil and water interactions. Magdoff et al., 1984;Fox et al., 1989;Magdoff, 1991;Meisinger et al., 1992;Andraski and Bundy, 2002;Sawyer and Mallarino, 2017 MRTN Nitrogen response trials are used to determine N rates. Data are easily updated with additional experimental N-rate trials. Calculations reflect current economic status by including the price of fertilizer and corn. Provides a range within $1.00 that farmers can use, depending on their risk level.
Does not address the issue of the year-to-year temperature or rainfall variability. Cannot predict site-specific N requirements and unlikely to accurately estimate EONR for each specific environment. Does not account for within-field spatial variability due to soil and water properties. Nafziger, Sawyer, & Hoeft, 2004;Sawyer et al., 2006;van Es et al., 2007 Crop growth models Estimates possible weather scenarios during a growing season to minimize N loss and predict N supplied by the soil. Non-static N recommendation based on the genetic, environmental, and management conditions. Initial inputs require time and money. Models may need to be calibrated to specific climate and soil conditions. Many parameters are estimated or generalized.
van Es et al., 2007;Setiyono et al., 2011;Sawyer, 2013 Shanahan et al., 2008;Holland and Schepers, 2010;Kitchen et al., 2010;Franzen et al., 2016 T A B L E 2 Methods and implementation costs associated with corn N recommendation tools included in this investigation. The implementation cost and required soil analysis are reported in parenthesis. LSNT Calculated using measured soil NO 3 -N (0-0.30 m) concentration and a critical limit of 25 mg kg −1 . To determine the N recommendation when NO 3 -N (0-0.30 m) is below the critical limit, the difference between the critical limit and the measured NO 3 -N (0-0.30 m) concentration is multiplied by 8. The critical limit is reduced by 3 to 5 mg kg −1 when spring precipitation is 20% above normal amounts. Costs were based on the average reported wages ($15.25 h −1 ) for operating machinery from the Iowa Farm Rate Survey (Plastina et al., 2017) and assuming a sampling rate of 8, 6, and 4 ha −1 for shallow, medium, and deep soil samples, respectively. e Sample analysis costs: The cost associated with analyzing samples was determined by taking the average of five soil-testing laboratories throughout the US Midwest that were either land grant or commercially operated (Agvise Laboratories, Midwest Labs, North Dakota State University, University of Missouri, and University of Wisconsin-Madison). The cost increased with each additional depth analyzed. f The custom application cost was estimated using the reported average sidedress liquid fertilizer application rate ($28.40 ha −1 ) from the Iowa Farm Rate Survey (Plastina et al., 2017). It was assumed that 50% of the sidedress application cost comes from machinery upkeep and acquisition, and 50% from labor and fuel (R. Massey, personal communication, 2017). The cost of using canopy reflectance sensors was calculated as 10% ($1.40 ha −1 ) of the base machinery upkeep and acquisition costs resulting in a total sidedress application cost of $29.90 ha −1 .
A tool similar to the PPNT, but also incorporates early season mineralization, is the PSNT or LSNT (Table 1). Soil sampling for the PSNT or LSNT tool is delayed 4 to 6 wk after planting, around the V5 corn developmental stage. Effectiveness of this tool has been well documented on corn fields following alfalfa or with manure applications; under these scenarios, soil test results for PSNT often showed an increase in NO 3 -N compared to the PPNT (Bundy, Walters, & Olness, 1999). However, issues related to soil sampling when fields often are wet from spring rainfall, nitrate movement below the sample depth, as well as the time required for sampling and laboratory analysis have hindered PSNT adoption (Schmidt, Dellinger, & Beegle, 2009; Table 1).
The MRTN tool relies on an extensive database of ongoing field research trials where corn response to applied N rates is measured, with regression modeling of the individual response trials (Sawyer et al., 2006). This free web-based tool (Corn Nitrogen Rate Calculator, http://cnrc.agron. iastate.edu/; accessed 5 Mar. 2017) determines an N recommendation by grouping and analyzing across sets of response trial models for user-identified regions, such as a state or sub-region within a state, and crop rotation. Trials from continuous corn are distinguished from trials where corn follows soybean (Glycine max). The economic return to N, that is, the MRTN rate and most profitable range adjusted by user-defined fertilizer and corn prices, provides the calculated N rate or range of N rates used as the N recommendation. To account for changes in climate and ongoing improved corn hybrids, the MRTN database is updated with recent years' results and older years are excluded. The MRTN recommendation can also be credited for manure applications or PPNT values (Laboski & Peters, 2012). Because the data used for the MRTN spans many years, recommendations for any given field will generally be consistent from 1 yr to the next, with some adjustment as the underlying database changes or prices fluctuate. Therefore, the MRTN tool provides rate suggestions that apply for an expected multi-year perspective, not a single year. This is a weakness if a yearly adjusted recommendation is desired since the tool does not account for yearly sitespecific weather or soil properties that are unique to the location for which the N recommendation is being made (Table 1).
With inexpensive data storage and management with cloud computing services, crop growth models that take all the major processes of the N cycle into account have been developed recently to produce N recommendations. These models use management inputs and site-specific soil and weather information to estimate soil N transformations and losses and plant physiological processes. Several crop growth models currently being used in the North American and the US Midwest include Maize-N (Setiyono et al., 2011), Adapt-N (Melkonian, van Es, DeGaetano, & Joseph, 2008), Granular Agronomy (https://granular.ag/agronomy; accessed 17 Dec. 2019), FieldView Pro (https://www.climate.com; accessed 17 Dec. 2019), and Effigis' FieldApex (https://www.fieldapex. com/; accessed 17 Dec. 2019). This approach for N recommendations allows for a continual model refinement based on additional field trails (He, Wang, Wang, & Robertson, 2017). A disadvantage of these tools is the costs required to obtain and incorporate new data into the model and software maintenance. For commercial N model services, there may also be a consultant provided with the service built into the fee. These costs generally will be passed on to farmers using it (Morris et al., 2017).
Light reflectance from crop leaves can be used to gauge crop N status and make in-season N management decisions (Moran, Inoue, & Barnes, 1997;Mulla, 2013;Lory, 2002, 2009;Shanahan et al., 2001;Schepers, Francis, Vigil, & Below, 1992;Sripada, Heiniger, White, & Meijer, 2006). Included within this diagnostic method is proximal or near-plant active-optical canopy-reflectance sensing for corn N management (Dellinger, Schmidt, & Beegle, 2008;Franzen, Kitchen, Holland, Schepers, & Raun, 2016;Holland & Schepers, 2010;Kitchen et al., 2010). Active-optical canopy-reflectance sensing using visible and near-infrared wavelengths can be used to quantify the crop's N status, as a function of the plant's biomass and color (Kitchen et al., 2010). This is accomplished using vegetation indices such as the normalized difference vegetation index (NDVI) or the normalized difference red edge index (NDRE). These indices are employed in algorithms for N recommendations (Dellinger et al., 2008;Franzen et al., 2016;Holland & Schepers, 2013;Raun et al., 2005;Solari, Shanahan, Ferguson, Schepers, & Gitelson, 2008). In contrast to the previously described tools, canopy sensing allows for a short-scale (1-5 m) plant N assessment with a resulting variable N rate recommendation (Raun et al., 2002). Drawbacks of canopy sensing include acquisition cost, the requirement for in-season N application, and the challenges of a representative N rich reference, even when obtained as a virtual N rich reference (Holland & Schepers, 2013).
Though corn N rate recommendation tools were extensively described and contrasted in the review of Morris et al. (2017), limited research has been done simultaneously to compare the performance of these tools over a wide range of soil and weather. Previous studies comparing these tools usually focused on a small geographical area (e.g., within a state) and/or included only a limited set of decisions tools (e.g., a tool compared to the farmer's typical N rate). Furthermore, these studies often compared the tool's performance relative to another tool, not to a measured EONR or optimal N rate; therefore, it was not possible to quantify the amount of N that was under-or over-recommended. Thus, there is a need for tools to be compared side-by-side with a standard optimal N rate, over a wide range of soil and weather environments. Such comparisons would provide measures of accuracy and reliability of each of these decision tools, and a better general understanding of the usefulness of N management tools in the US Corn Belt.
The objectives of this research were to evaluate the performance of publicly-available N fertilizer recommendation tools across diverse soil and weather conditions for (i) prescribing N for planting and split fertilizer applications, and (ii) evaluating their economic and environmental effects.

Experimental design
This research was conducted as a part of a public-private collaboration between Corteva Agrisciences and eight US Midwest universities (Iowa State University, University of Illinois Urbana-Champaign, University of Minnesota, University of Missouri, North Dakota State University, Purdue University, University of Nebraska-Lincoln, and University of Wisconsin-Madison). Each state conducted research on two sites each year from 2014 to 2016, with a third site in Missouri in 2016, totaling 49 site-years. About half the sites were on farmers' fields and the other half on University research stations. All states followed a similar protocol for plot research implementation including site selection, weather data collection, soil and plant sample timing and collection methodology, N application timing, N source, and N rates, with specific details described in Kitchen et al. (2017). The average plot dimension was 3-m wide and 15-m long. Treatments included ammonium nitrate fertilizer rates between 0 and 315 kg N ha −1 in 45 kg N ha −1 increments applied either all at-planting or split, where 45 kg N ha −1 was surface broadcast at-planting and the remaining fertilizer N broadcast at the V9 corn developmental stage (Abendroth, Elmore, Boyer, & Marlay, 2011).

Determining the economically optimal nitrogen rate
For this analysis, all tools were evaluated against EONR. Only a few tools (e.g., MRTN, Nebraska YG, and crop growth models) have been developed with the inclusion of fertilizer and grain prices in their N recommendation that warrants comparison with EONR. All other tools were historically developed to maximize or reach a target yield, and thus would be slightly handicapped when comparing to EONR. Still, all tools were compared against EONR rather than the optimal N rate, as EONR is more meaningful to farmers for maximizing their profits, and currently is the more common metric for tool comparison (Kachanoski & Fairchild, 1996;Lory & Scharf, 2003;Sawyer and Mallarino, 2017;Vanotti & Bundy, 1994). Grain yield in response to N fertilizer treatments was used to calculate the EONR on a site level as described in Kitchen et al. (2017), using proven quadratic or quadratic-plateau modeling methods (Cerrato & Blackmer, 1990;Scharf et al., 2005). The EONR values were calculated for all N fertilizer applied at-planting (hereafter referred to as "at-planting"), and N split applied between an at-planting and a sidedress applications (hereafter referred to as "split"). The cost of N was $0.88 kg N −1, and the price of corn was $0.158 kg grain −1 (equivalent to $0.40 lbs N −1 and $4.00 bu −1 ). The EONR was set to not exceed the maximum N rate (315 kg N ha −1 ). Five of the seven irrigated sites had additional N applied through irrigation >12 kg N ha −1 , and this was included in determining the EONR of these sites. For 19 of the 49 sites, the at-planting and split EONR values were found to be the same statistically (P ≤ .05) and within $2.50 ha −1 of each other. Thus for these sites, the EONR used was the average of the two timings. This approach was also consistent with a previous separate analysis of this dataset (Bandura, 2017).

Farmer's N rate and yield goal
The farmer's historical N rate was the rate the farmer or research station typically applied to the field site under ideal corn-growing conditions. The information the farmer or station manager used to base this N rate was not recorded, but it was assumed to be based on crop response to N of the site over multiple years, and not necessarily on any specific decision tool.
Six YG tools were included in this evaluation as outlined in Table 2. These included a generic YG tool (General YG) based on original work of Stanford (1973) Other states in the Midwest had a documented YG method that was the same or nearly identical to previously mentioned states, and therefore these were excluded as individual tools in this evaluation, but they were included as a part of the State-Specific YG tool (see Table 2 for details). An exception was Wisconsin because it had no published YG approach, so it was excluded from the State-Specific YG evaluation. All YG methods follow a similar mass balance approach established by Stanford (1973), but each was uniquely modified by adjusting coefficients within the calculation and incorporating additional soil and/or management information (Table 2). For example, the Nebraska YG was adjusted with PPNT values that were either estimated or measured to a depth of 1.20 m. Each of these six YG tools was used to determine a corn N fertilizer recommendation for all 49 sites of this investigation.
All YG tools required an expected yield. The expected yield for each site was determined using the average of the previous 5-yr county corn yields for the respective county the site was within. The 5-yr average was then adjusted based on the soil productivity of the predominantly mapped soil of each site, similar to that done by Laboski and Peters (2012). This procedure classifies soil productivity as either low, medium, or high using soil texture, drainage class, depth to bedrock, available water capacity in the upper 150 cm of soil, average growing degree days, irrigation, and artificial tile drainage. The YG of a site was then calculated by increasing the 5-yr average yield for low, medium, and high soil productivity by 10, 20, or 30%, respectively. This estimated yield value was used to represent the six different YG tools shown in Table 2.

Soil nitrogen tests
Four distinct PPNT tools were evaluated. They are as follows: (i) General PPNT, (ii) MN PPNT, (iii) North Dakota (ND) PPNT, and (iv) WI PPNT (Table 2). Kitchen et al. (2017) detailed the sampling and NO 3 -N analysis protocols for the PPNT tool. Two of the 49 sites did not complete PPNT sampling, so this tool was evaluated using 47 of the 49 sites.
Four in-season nitrate tests were evaluated, including (i) General PSNT, (ii) Iowa (IA) LSNT, (iii) IN PSNT, and (iv) WI PSNT (Table 2). These were tested under two different conditions. The first used a site average of measured NO 3 -N from plots that received 0 kg N ha −1 at-planting. The second used a site average of measured NO 3 -N from plots that received 45 kg N ha −1 at-planting. These are noted as PSNT or LSNT 0 and PSNT or LSNT 45, respectively. Soil samples were taken at the V5 ± 1 corn development stage to a depth of 0.30 m.

Maximum return to nitrogen
The MRTN recommendation rates were determined by using values obtained in 2016, as only a few states had updated the MRTN database during the 3 yr of this project. The MRTN values for IA, IL, IN, MN, and WI were obtained from the online Corn N Rate Calculator (cnrc.agron.iastate.edu; accessed 5 Mar. 2017). The MRTN values for ND were obtained from the North Dakota Corn Nitrogen Calculator (www.ndsu.edu/pubweb/soils/corn; accessed 5 Mar. 2017). The price of corn/N fertilizer ratio used was 10:1. Since neither MO nor NE currently have a compiled database supporting the MRTN approach, sites from these states were excluded from this tool's evaluation.

Maize-N crop growth model
The Maize-N crop model version 2017.1.0 (Setiyono et al., 2011) was used to generate an N fertilizer recommendation for all sites. Required in-season weather data were obtained at each site using a HOBO (model U30) weather station (Onset, Bourne, MA). Weather data were subjected to a quality check and then aggregated into a daily summary of minimum and maximum temperature, average solar radiation, and precipitation as explained in Kitchen et al. (2017). Additional historical weather data was required to generate an N recommendation. For this, 30 yr of site-specific weather data were obtained from Corteva Agrisciences using a proprietary method for interpolating between multiple weather stations around each site. These weather data mostly came from the public National Service Storms Lab (NOAA) weather stations, supplemented with data observed by Corteva Agrisciences' internal weather network (HOBO stations). The weather data were collected within the acceptable range of 50 to 100 km radius as listed in the Maize-N user guide. Explicit site information required by the Maize-N crop growth model included management records (e.g., date of planting, plant population, average historical yield, tillage operations, and previous crop) and soil information (e.g., bulk density, % organic matter, rooting zone depth, soil pH, and soil NO 3 -N).

Canopy reflectance sensing
Canopy Reflectance measurements were obtained using the RapidSCAN CS-45 (Holland Scientific, Lincoln, NE) prior to the split N application (i.e., generally within 2 d of sensing). For the majority of sites, this was done at the ∼V8 to V10 corn development stage. Measurement details are described in Kitchen et al. (2017). The Holland and Schepers algorithm (Holland & Schepers, 2010) was used to calculate an N fertilizer recommendation derived from these reflectance measurements. All reflectance measurements were taken from plots that received 45 kg N ha −1 at-planting and where a sidedress fertilizer was to be applied. A sufficiency index (SI) was determined on a site level as the ratio between minimally-fertilized corn NDRE and a virtual reference "N rich" corn NDRE: Fertilizer N recommendations were then calculated as described in Holland and Schepers (2010) as follows: where N rec is the calculated N fertilizer recommendation; MZ i is a scaling value (0 ≥ MZ i ≤ 2) used to adjust the N recommendation based on areas of high or low yield performance; N Opt is the base N rate, which is determined by the farmer; N PreFert is the amount of N already applied prior to sensing; N CRD is N credits associated with the previous crop, NO 3 -N in irrigation water, manure, or residual soil NO 3 -N; N Comp is an optional compensation factor for growth-limiting conditions; SI is the sufficiency index, and ΔSI is a value to define the response range. For this analysis, MZ i was left as the default value of 1.0, N opt was set as the recorded farmer's N rate for each site, and N PreFert = 45 kg N ha −1 . With no supportive information relative to N CRD and N Comp , these two parameters were set to zero for all sites. The recommended value of 0.30 was used for ΔSI, which provides a response range for the measured vegetative index value between 0.70 and 1.00.

Economic assessment of tools
For an economic analysis of each tool, the implementation costs (e.g., soil sampling, sample analysis, and procurement costs) and the cost of N fertilizer were subtracted from the yield revenue at each of the tool's N recommendation rates (Table 2). Then each tool's partial profit was determined relative to EONR as follows: where GY Tool and GY EONR were the estimated yields associated with the tool's N recommendation and EONR, respectively; N Tool was the N rate associated with a tool's N recommendation; IPC was the implementation costs. The price of corn grain and the cost of N fertilizer was fixed at $0.158 kg grain −1 ($4.00 bu −1 ) and $0.88 kg N −1 ($0.40 lb N −1 ), respectively. Corn grain yields were estimated using the same N response curves developed to calculate each site's EONR value (see Figure 1 for an example). Implementation costs varied for each of the N recommendation tools based on the timing of N fertilizer application and the costs associated F I G U R E 1 An example of response models for one site's EONR partial profit and environmental cost evaluation. Shown as a function of N applied, are values of grain yield and estimated total season N loss and their respective best-fit models (Table 3). Grain yield is shown as a quadratic-plateau model (squares and solid line) and N loss as a quadratic model (open circles and small dash line). The partial profit at EONR was calculated using the interpolated grain yield from the best-fit line (for this example, 13.5 Mg ha −1 times $158 Mg −1 = $2133 ha −1 ). Environmental costs at EONR was calculated by multiplying the estimated total season N loss by a prevention cost (for this example, 69 kg NO 3 -N ha −1 times $2.75 kg −1 NO 3 -N = $190 ha −1 ). The partial profit and environmental cost for each tool were based on model outcomes using their respective N recommendation. Additional implementation costs associated with utilizing tools were subtracted from the partial profit (Table 2). Each assessment was made relative to EONR. Tools that underestimated EONR (light green) resulted in decreased partial profits but provided an environmental credit. Tools that overestimated EONR (light red) resulted in decreased partial profits and greater environmental costs. This assessment was done for all 49 sites for both at-planting and split conditions with sampling and analyzing soils as needed to implement the tool. Both the cost of N fertilizer applications and soil sampling were obtained from the Iowa Custom Application Survey (Plastina, Johanns, & Wood, 2017 that are related to completing forms, inputting information, and interpreting results could be accrued. However, for this analysis, only direct costs required to obtain an N recommendation were used. Note, this partial profit metric (Eq. (4)) used to compare tools will always be negative unless a tool exactly matched EONR at all sites and had no implementation cost, thus the evaluation is relative of tools to each other.

Environmental assessment of tools
An environmental evaluation for each tool was performed by accounting for the potential N loss from the time of planting to the end of the year. This was calculated using an N balance procedure with known N inputs and removals. This procedure did not attempt to identify N loss pathways. The estimation was as follows: where N Fert was the treatment N fertilizer rate (by plot); N Irr was the inorganic N applied through irrigation (site level); N min was the potential N mineralization measured (by replication block) (Clark, 2018); PPNT was the preplant soil NO 3 -N in the profile (0-0.90 m; by replication block); N uptake was the measured above-ground grain and biomass total N at plant maturity (by plot); and N roots was an estimated N content in the roots at plant maturity (by plot). Nitrogen mineralization was measured using the surface (0-0.30 m) PPNT soil samples with a 7-d anaerobic incubation procedure (Bundy & Meisinger, 1994;Clark, 2018;Keeney & Bremner, 1966). This procedure provides a potential mineralization rate under optimal conditions. While a full season N mineralization was not measured, N min was used as an approximation for this and allows for comparisons of potential mineralization between and across sites. Nitrogen mineralization and NO 3 -N concentrations were converted to an area basis (kg N ha −1 ) using a four-core averaged bulk density for each site, determined for each soil depth increment. Since no soil samples were preserved for N min from the Nebraska 2015 and 2016 and North Dakota 2016 sites, mineralization values from samples of nearby fields from other years of this study were substituted. The N uptake was calculated as the product of the R6 developmental growth stage dry-matter mass and the N concentration for grain and stover samples (details described in Kitchen et al., 2017). To account for N immobilized by roots, N content was estimated using the measured shoot N content at plant maturity and using a root N/shoot N ratio of 0.20:1 (Crozier & King, 1993;Merbach et al., 1999). Equation (5) was calculated for each site plot giving a total of >3000 experimental units (i.e., 49 sites with 16 N treat-ments and 4 replications). A linear, quadratic, plateau-linear, plateau-quadratic, or exponential model was used to fit N loss relative to N fertilizer rates for each site, with both at-planting and split N application treatments. A model for each site was selected based on the assessed goodness-of-fit, the significance of the model, and the lowest root-mean-square error (RMSE; Table 3). The best-fit models for each site were then used to interpolate the N loss associated with each N recommendation tool. A similar interpolated N loss value was determined at each site's EONR value (see Figure 1 for an example).
To calculate an environmental cost, the difference between the tool N loss and the EONR N loss were multiplied by a prevention cost of $2.75 kg −1 NO 3 -N. This value was based on the average of previously reported implementations costs associated with reducing soil and water NO 3 -N through various practices, such as drainage water management (Cooke, Sands, & Brown, 2008), buffers and vegetative strips (Helmers, Dosskey, Dabney, & Strock, 2008), erosion control (Czapar, Laflen, Mclsaac, & McKenna, 2008), and cover crops (Kaspar, Kladivko, Singer, Morse, & Mutch, 2008). These costs were adjusted for inflation from their reported values to a 2015-dollar amount using an average inflation rate of 1.95%, calculated using the FinanceRef inflation Calculator (www.in2013dollars.com; accessed 15 Dec. 2017).

Statistical analysis
Tools that could provide N fertilizer recommendations for both at-planting and split applications were initially assessed with both timings and treated as two different tools. Even though N recommendations averaged over all sites did not change drastically between the at-planting to the split application timing, the EONR values varied between the two N application times within sites (30 of the 49 sites; Bandura, 2017).
Two different metrics were used to evaluate the performance of each tool for predicting and matching EONR. To determine how well a tool predicted EONR, a tool's N recommendation was compared to the EONR across all sites using a simple linear regression model. Only if this relationship was positive and significant (P ≤ .10) was a tool considered successful at predicting EONR. To determine how well a tool matched EONR, the average and the RMSE were evaluated based on the difference between a tool's N recommendation and EONR. Using this approach, tools were compared within a family of tools, between at-planting and split N applications (when applicable), and across all tools and N application timings (average only). An additional performance metric examined the percentage of sites when a tool's recommendation came close to EONR. Sites within ± 30 kg N ha −1 of EONR were considered reasonably close to EONR (cEONR T A B L E 3 The best-fit models (Linear, Quadratic, Plateau-Linear, Plateau-Quadratic, and Exponential) relating N loss from the soil profile (0-0.90 m) in-season and residual soil NO 3 -N (measured post-harvest) as a function of N fertilizer rate for each site and N application timing. The goodness of fit (R 2 ) values of each model are also reported for each significant (P ≤ .05) model or 'Good'). This value around EONR was chosen as it was the average range of values calculated using ± $2.00 from EONR which also aligns with what others have suggested as both reasonable and practicable for evaluating a tool's successful performance (Laboski, Camberato, & Sawyer, 2014;Sawyer, 2013;Sela et al., 2017). The percentages of sites cEONR not classified as 'Good' were classified either as "Mediocre" (within ± 60 kg N ha −1 of EONR) or poor (> ± 60 kg N ha −1 of EONR). An optimal tool performance would consist of an average difference between that tool's N recommendation and EONR being close to zero (accurate), having a low RMSE (precise), and a high percentage of sites cEONR. For additional comparisons of all tools, the difference between the N recommendation, partial profit, and environmental costs were made relative to EONR. For each of these three analyses, an ANOVA model was examined using the response variable (N recommendation, partial profits, or environmental cost) as a function of the interaction between N application timing and tool type. The Tukey's honest significant test was used for any post hoc pairwise comparisons using a significance threshold of .05. All calculations and analyses were conducted using the R Statistical Program (R Development Core Team, 2016).

Corn nitrogen response and EONR
Growing season precipitation at these sites ranged from 245-1000 mm. Based on visual observations, investigators noted only a few days of crop stress from water deficiency at any site. Given the varied soil environments represented across the 49 sites, and excessive precipitation at some sites (Kitchen et al., 2017), a wide range of corn response to N fertilizer rates occurred. The EONR values across both application timings ranged from 0-315 kg N ha −1 . Of the 49 sites, three were nonresponsive to added N fertilizer, and another had an EONR less than 40 kg N ha −1 . In contrast, five sites resulted in high EONR values (>300 kg N ha −1 ), assumed to be the result of excessive precipitation likely resulting in conditions producing denitrification at sites with fine-textured soils and leaching at sites with coarse-textured soils. A summary of the yield response to added N in this study has been previously published (Kitchen et al., 2017). Average EONR across all sites was 169 kg N ha −1 (SD = 83) and 159 kg N ha −1 (SD = 70) for at-planting and split N applications, respectively.

Which tools gave recommendations related to EONR?
The first metric for evaluating a tool was determining when the variation in a specific tool N recommendation across sites and years exhibited a positive linear relationship with variation in EONR at a P value ≤ .10. Evaluating tools with this metric determines which tools were best able to predict EONR at a site-year level. Only 10 of 31 tools resulted in having a significant positive and linear relationship with EONR (see tools bolded in Table 4), which included 3 of 13 at-planting application tools and 7 of 18 split application tools. Of these, no tool produced a recommendation rate that predicted EONR well, with the best tool (LSNT 0) giving an r 2 = .20 (P ≤ .01). Furthermore, 6 of these 10 tools examined by individual year were only successful for one of the 3 yr, three tools were successful in two of the 3 yr, and one tool only successful when combined across all years (Table 5). This lack of success across years suggests the dominating effect that weather has on tool performance since many sites within a state were close to each other and similar in soil type and management from 1 yr to the next. Since the primary objective here was to test tools across diverse soil and weather environments, the performance was evaluated as an aggregate of all years (see Table 5 for significant linear positive relationships of each tool by individual year).
Of the 31 tools, 21 were not positive and linearly related to EONR across all 3 yr (see tools not bolded in Table 4). These included nearly all YG methods (which were negatively related with EONR), Farmer's N rate, ND PPNT, MRTN, Maize-N crop growth model, IN PSNT 0, WI PSNT 45, and T A B L E 4 Significant linear regression relationships between each N recommendation tool and the economically optimal N rate (EONR). Both at-planting and split N application tools are reported. Tools include yield goal (YG), preplant nitrate test (PPNT), pre-sidedress nitrate test (PSNT) and late-spring nitrate test (LSNT) with 0 and 45 kg N ha −1 applied at-planting, Maximum Return to N (MRTN), Maize-N crop growth model, and canopy reflectance sensing using the Holland and Schepers algorithm. Bolded tool names indicate a significant and positive relationship between recommendations and EONR were found (P ≤ .10). canopy reflectance sensing. A lack of relationship was not surprising with some of these tools. For example, MRTN was established as a long-term N recommendation system developed from an aggregation of N rate response trials over multiple site years. As such, MRTN does not specifically account for local soil or weather conditions of the growing season for which the recommendation is being made, but known to greatly affect crop N response (Morris et al., 2017;Tremblay et al., 2012). A comparable conclusion could also be made for the Farmer's N rate and YG, as they would generally be the result of an average of past years' experiences of corn N response. Yet other tools that do account for site-specific soil (e.g., PPNT, PSNT) and weather (e.g., Maize-N) were also not related to EONR. These tools are unique in that they attempt to account for soil NO 3 -N and N mineralization, which could improve their ability to identify sites with no response to N fertilizer. Tools that can predict the extremes of a site's response to N fertilizer (e.g., no response, or high fertilizer need as a result of conditions that promote N loss to the environment) would have a better chance of being related with EONR.

Which tools gave recommendations close to EONR?
The second metric of evaluating these tools included using the average difference between a tool's N recommendation and EONR, RMSE, and the percentage of sites cEONR. Under these conditions, there was a wide range of responses for each of these three metrics (  Table 6; Figure 2).

F I G U R E 2
The percentage and number of sites for the at-planting and sidedress tools' recommendations that came within: ± 30 kg N ha −1 of economically optimal N rate (EONR; "Good"), ± 60 kg N ha −1 of EONR ("Mediocre"), and > 60 or < −60 kg N ha −1 of EONR ("Bad" Using an ANOVA to compare the average differences of the tools' N recommendations relative to EONR resulted in a significant main effect for unique tools (P ≤ .001) with no significant results for N application timing (P = .33) or the two-way interaction between tool and application timing (P = .97). After averaging across application timing, there were significant differences between tools (Figure 3). Using these above-described metrics, a discussion of performance by general tool type is provided below.

Farmer's N rate
The Farmer's N rate (NR) did not have a significant relationship with EONR (Table 4), and on average, this tool overestimated EONR (Figure 3, 4). However, regarding the RMSE calculated using the difference between a tool's N recommendation rate and EONR, few tools performed better than the Farmer's NR with 88 and 84 kg N ha −1 RMSE values for planting and split applications, respectively (Table 6). Of those tools that did perform better the MRTN, WI PPNT, General PSNT 0, LSNT 0, WI PSNT 0, and canopy reflectance sensing were the only ones that showed a strong improvement with decreased RMSE ≥ 10 kg N ha −1 and/or an improved average percentage of sites cEONR > 5% (Figure 2). Even with the improved metrics, no significant difference between these tools and the Farmer's NR were found (Figure 3).

Yield goal
Three of the six YG approaches (General YG, IN YG, and MO YG) were poor performing tools. All of these tools were significant but had a negative linear relationship with EONR; Table 4). On average, they all overestimated EONR ≥ 58 kg N ha −1 , had RMSE ≥ 113 kg N ha −1 , and had a T A B L E 6 The precision and accuracy of each N recommendation tool was evaluated using the average difference [N recommendation tooleconomically optimal N rate (EONR)], root mean square error (RMSE) of the difference between a tool's N recommendation and EONR, and the percentage of sites ± 30 kg N ha −1 of the EONR or "close to EONR" (cEONR). Tools were evaluated across a maximum of 49 sites from 2014 to 2016, however, the number of sites (n) included in the evaluation differed among tools based on the availability of information needed to test the tool. Tools included yield goal (YG), preplant nitrate test (PPNT), pre-sidedress nitrate test (PSNT) and late-spring nitrate test (LSNT) with 0 and 45 kg N ha −1 applied at-planting, maximum return to N (MRTN), Maize-N crop growth model, and canopy reflectance sensing using the Holland and Schepers algorithm. The best performing tools for each metric evaluated at-planting and split application timings are bolded. percentage of sites cEONR ≤ 20% (Table 6; Figure 2). In contrast, the NE YG and MN YG on average underestimated EONR ≤ 2 kg N ha −1 , had RMSE values ≤ 90 kg N ha −1 , and had a percentage of sites cEONR between 24-41%. The NE YG was unique in that it required the most inputs, such as a measure of soil NO 3 -N, estimated amount of N in irrigation water, the price of N fertilizer and corn grain, and adjustment for application timing ( Table 2). All of these adjustments caused the NE YG to underestimate EONR on average. In contrast, the MN YG is distinctive from all other YG approaches as it categorizes sites as either being low or high in organic matter. Due to the high organic matter (>30 g kg −1 ) measured at most of the sites (Kitchen et al., 2017), this resulted in similar recommendations across all sites that averaged close to 168 kg N ha −1 , whereas the average EONR values were 171 and 162 for planting and split applications, respectively.
Conversely, using the State-Specific YG for a split N application was positively related with EONR, though this relationship was weak (r 2 = .10; P = .04). Others have also shown YG N recommendation tools were weakly related to EONR (r 2 ≤ .21; Blackmer, Morris, & Binford, 1992;Fox & Piekielek, 1995;Vanotti & Bundy, 1994). Regarding other metrics of performance, this YG approach on average overestimated EONR ≥ 22 kg N ha −1 , had a lower RMSE than the NE YG and MN YG at 72 kg N ha −1 , and a similar percentage of sites cEONR of 37% (Table 6). Sidedress N recommendation rates utilizing the State-Specific YG were less for some states than all at-planting, helping to align recommendations with EONR. For example, all NE YG-based N recommendations for the split application were reduced by 5%, which gave results slightly closer to EONR (Figure 4).

F I G U R E 3
Graph shows the mean difference (in kg N ha −1 ) between each N recommendation tool and the economically optimal N rate (EONR). Tools used for both planting and split N application timing were not different (P = 0.97), and therefore recommendations shown are averaged across timings. Tools include farmer's nitrogen rate (NR), yield goal (YG), preplant nitrate test (PPNT), pre-sidedress nitrate test (PSNT) and late-spring nitrate test (LSNT) with 0 and 45 kg N ha −1 applied at-planting, Maximum Return to N (MRTN), Maize-N crop growth model, and canopy reflectance sensing using the Holland and Schepers algorithm. Significance means separation was determined using Tukey's honest significant test with a significance threshold of 0.05. Tools with the same letter are not significantly different from each other. Tools with the same letter, indicate that means are not significantly different from each other

Preplant soil nitrate tests
Three PPNT tools produced N recommendations that were related to EONR (General, MN, and WI), but explained no better than 16% of the variability in EONR (P ≤ .01; Table 4). These tools work by adjusting a base N recommendation (State-Specific YG or MRTN) with a preplant assessment of soil NO 3 -N, applying a soil measurement into the mass balance. As such, these tools helped adjust sites that overestimated EONR. By themselves, the base N recommendations from YG or MRTN overestimated EONR, but after adjustment, these PPNT tools underestimated EONR by an average of 40, 26, and 5 kg N ha −1 for General, MN, and WI, respectively (Table 6 and Figure 5). Whereas the ND PPNT, was not significantly related to EONR, but on average overestimated EONR by 9 kg N ha −1 .
Of these four PPNT tools, the WI PPNT on average gave a recommendation closest to EONR. While this tool was not statistically different from the other PPNT tools (Figure 3), the RMSE was 14, 22, and 9 kg N ha −1 lower than the General PPNT, ND PPNT, and MN PPNT, respectively (Table 6).

F I G U R E 4 Box and whisker plots showing the difference (in kg
N ha −1 ) between the farmer's nitrogen rate (NR) and each yield goal (YG) based N recommendation and the economically optimal N rate (EONR) for both at planting and split N application timings. The median is reported by the value in the middle of the box. Notches on the side of each box indicate the 95% confidence interval around the median. Limits of the box indicate the first and third quartile, whiskers indicate 1.5 times interquartile range, and small circles indicate outliers F I G U R E 5 Box and whisker plots showing the difference (in kg N ha −1 ) between each preplant soil nitrate test (PPNT) and the economically optimal N rate (EONR). The median is reported by the value in the middle of the box. Notches on the side of each box indicate the 95% confidence interval around the median. Limits of the box indicate the first and third quartile, whiskers indicate 1.5 times interquartile range Furthermore, the WI PPNT tool had 34% sites cEONR compared to 21, 13, and 32% of sites for the General PPNT, ND PPNT, and MN PPNT, respectively (Table 6; Figure 2). The improved performance of this tool was attributed to two features. First, it does not recommend adjustments if NO 3 -N levels are below 56 kg N ha −1 ( Table 2). As such, no adjustments to the base N recommendation were made for 22 of the 44 sites evaluated (Figure 2a, 2b). However, for eight of those 22 sites, an adjustment would have been beneficial as the base N recommendation overestimated EONR by as much as 30 kg N ha −1 . Second, the WI PPNT adjustments were more substantial as it accounted for NO 3 -N levels down to 0.90 m rather than 0.60 m. This improved the final WI PPNT recommendation for those nonresponsive sites over the other two PPNT tools. Of note, we used measured NO 3 -N data down to 0.90 m. However, the WI PPNT allows for samples to be taken to a depth of 0.60 m and estimates the remaining NO 3 -N amounts in the bottom 0.60-0.90 m depth.
One factor of this study that may have reduced the predictability of PPNT N recommendations was that most of the study sites were corn following soybean. Soybean have been shown to be an excellent scavenger of soil NO 3 -N, resulting in a minimal amount of NO 3 -N remaining in the soil the following spring (Kaiser, Fernandez, Lamb, Coulter, & Barber, 2016;Shapiro, Ferguson, Hergert, Wortmann, & Walters, 2008). The PPNT may be better suited for conditions where residual soil NO 3 -N would accumulate, such as with manured fields or when precipitation was lower than average in the previous growing season.

Pre-sidedress soil nitrate test and late spring nitrate test
The PSNT and LSNT tools generally performed slightly better when evaluated under the conditions of 0 kg N ha −1 applied at-planting compared to when 45 kg N ha −1 was applied at-planting (Table 4; Figure 6). Of the PSNT methods evaluated with 0 kg N ha −1 applied at-planting, the General PSNT, LSNT, and WI PSNT tools were found to be significant and positively related to EONR. These three tools performed similarly when comparing average recommendations relative to EONR, RMSE, and the percentage of sites cEONR (Table 6; Figure 2). While the LSNT 0 tool on average underestimated EONR by ∼20 kg N ha −1 more than the General PSNT 0 and WI PSNT 0, its predicted N rate had the best linear relationship with EONR (r 2 = .20, P < .001) of all N recommendation tools evaluated (Table 4). Nevertheless, this relationship was not particularly strong and substantially less than what other researchers have reported for other PSNT tools. Schmidt et al. (2009) reported the Pennsylvania PSNT to have an r 2 = .48 with EONR. The weak relationship found in our work compared to other studies could be the result of diverse environmental conditions represented by the extensive geographic region of this study relative to the area from which the tool was developed and calibrated for N recommendations. A similar finding was reported by Scharf, Brouder, and Hoeft (2006) where pre-sidedress NO 3 -N F I G U R E 6 Box and whisker plots showing the difference (in kg N ha −1 ) between each pre-sidedress soil nitrate test (PSNT) and late spring nitrate test (LSNT) N recommendation and the economically optimal N rate (EONR). The PSNT and LSNT tools evaluated for both 0 and 45 kg N ha −1 applied at-planting. The median is reported by the value in the middle of the box. Notches on the side of each box indicate the 95% confidence interval around the median. Limits of the box indicate the first and third quartile, whiskers indicate 1.5 times interquartile range, and small circles indicate outliers concentrations from 66 sites across seven Midwest states had weak linear relationships with EONR (r 2 ≤ .16).
Of the four in-season soil nitrate tools evaluated with 45 kg N ha −1 applied at-planting, the General PSNT, LSNT, and IN PSNT tools were found to be successful (Table 4). Of these, the IN PSNT 45 had one of the lowest RMSE and on average came closest to EONR (Table 6). The IN PSNT differs from the other PSNT methods, as the N recommendation is categorized into six groups of N rates based on expected yield (Brouder & Mengel, 2003). While this method had a significant relationship when 45 kg N ha −1 was applied at-planting, no significant relationship was observed with EONR when evaluated with no N applied at-planting. The reason for this difference is unknown.
A possible explanation for why the PSNT 45 tools underestimated N recommendations relative to EONR, was that the added 45 kg N ha −1 masked the N-supplying capacity of the soil. Others have found limits as to how much N could be applied at-planting before the PSNT becomes ineffective in predicting N requirements. Fernández, Nafziger, Ebelhar, and Hoeft (2009) stated that the PSNT tool should not be used if >22-30 kg N ha −1 was applied at-planting, while Blackmer et al. (1997) reported limiting N up to 84 kg N ha −1 with corn following soybean. Additionally, Ketterings, Albrecht, Czymmek, and Stockin (2012) documented the limit to be no more than 45 kg N ha −1 when fertilizer was banded. Our research supports the conclusions of others that applying N at-planting can reduce the effectiveness of PSNT tools.
The PSNT is not currently recommended under certain situations, such as sandy soils or soils with low organic matter (Fox, Roth, Iversen, & Piekielek, 1989;Meisinger, Bandel, Angle, O'Keefe, & Reynolds, 1992;Sawyer & Mallarino, 2017). Nevertheless, removing the three sites with sand >80% from the analysis resulted in little or no improvement for all of the PSNT and LSNT tools (reduced RMSE < 5 kg N ha −1 ; data not shown). As such, all sites were included in this analysis regardless of soil texture.

MRTN
The MRTN tool was a poor predictor of EONR with no significant linear relationship across all site-years (Table 4, 5). This is consistent with how this tool functions, as MRTN recommendations are based on a regional aggregation of numerous site-years of N response trials, and does not currently allow for making site-specific recommendations (based on variation in temporal and spatial N response). Thus, MRTN will tend to perform poorly when N response is abnormal to what was used to develop the tool. As such, this approach failed on 3 of the 36 sites used to evaluate MRTN, where no or minimal N response was observed (EONR ≤ 50 kg N ha −1 ), and for one coarse-textured site with a high propensity toward N leaching (EONR ≥ 270 kg N ha −1 ). However, evaluated MRTN using other metrics resulted in this tool as one of the top five top-performing tools for both at-planting and split applications (Table 6; Figure 7). On average it overestimated EONR by 16 and 19 kg N ha −1 , had an RMSE value of 77 and 72 kg N ha −1 , and the percentage of sites cEONR were 39 and 42% for at-planting and split applications, respectively ( Figure 2).
In contrast to the majority of the YG based tools that are also based on multiple years of information (5+ years of yield data), MRTN had recommendations much closer to EONR, resulting in lower RMSE values and a higher percentage of sites cEONR. These results are consistent with previous research that showed MRTN recommended less N but had greater profitability compared to YG based methods (Sawyer & Nafziger, 2010

Maize-N
The Maize-N crop growth model was one of the poorest performing tools, as it greatly underestimated EONR (Figure 3), F I G U R E 7 Box and whisker plots showing the difference (in kg N ha −1 ) between each of the tools' N recommendation and the economically optimal N rate (EONR) for both at-planting and split N application timings. Tools include Maximum Return to N (MRTN), Maize-N crop growth model, and canopy reflectance sensing using the Holland and Schepers algorithm. The median is reported by the value in the middle of the box. Notches on the side of each the box indicate the 95% confidence interval around the median. Limits of the box indicate the first and third quartile, whiskers indicate 1.5 times interquartile range, and small circles indicate outliers had the largest RMSE ≥ 122 kg N ha −1 (Table 6), and the lowest percentage of sites cEONR (Figure 2). Crop growth models show promise as they attempt to predict if sites will be responsive or nonresponsive to N fertilizer applications. They do this by incorporating mechanistic modeling routines that estimate crop N need and soil N, management inputs, and inseason and long-term (≥10 yr) weather data. With this mechanistic approach, Maize-N correctly identified two of the four nonresponsive sites for both at-planting and split N applications, but falsely identified five at-planting and three split sites as nonresponsive. One might assume with actual in-season weather information, the Maize-N split N recommendation would better match EONR than when used for an at-planting application. However, for about half the sites (23 of 49) the split N recommendations from Maize-N were weaker predictors of EONR than at-planting N recommendations (Figure 7).
These results suggest improvements are needed for the Maize-N model to better account for the year-to-year and location-to-location soil and weather variability represented by the US Corn Belt. Currently, many of the model coefficients used in Maize-N are simplified estimates of management, soil, and genetic parameters. These parameters may have worked well for the western Corn Belt where the model was developed (Setiyono et al., 2011), but perhaps need altering in other regions of the Corn Belt.

Canopy reflectance sensing
Nitrogen recommendations using canopy reflectance sensing and the Holland-Schepers algorithm was not linearly related with EONR (P = .89). On average canopy reflectance sensing underestimated the amount of N required by 7 kg N ha −1 , which was one of the top five tools for this metric of performance (Table 6; Figure 7). Using this tool resulted in 44% of sites cEONR, the highest percentage of any tool. This tool did not perform well at sites where corn had no or a limited response to N fertilizer. Using an SI based on the 95th percentile of data on plot research will always result in the Holland-Schepers algorithm recommending N rates >45 kg N ha −1 . Removing these sites from the analysis results in the Holland-Schepers algorithm underestimating EONR by 25 kg N ha −1 .
An evaluation of the Holland-Schepers algorithm was previously performed on this same dataset (Bean et al., 2018). However, their findings resulted in a poorer performing Holland-Schepers algorithm-based recommendation, as they used N rich values derived from the mean NDRE measurements taken from plots that received 225 and 270 kg N ha −1 at-planting. This approach caused the Holland and Schepers algorithm recommendations to decrease by an average of about 40 kg ha −1 compared to the virtual N rich reference based recommendation reported here. The reason for this large difference in recommendations is that the virtualbased N rich reference had higher NDRE values (an average of 11% higher) compared with using an established high N rich reference, thus resulting in smaller SI values, which indicate more N stress and result in higher N recommendations.

Economic and environmental assessment of tools
Separate from how well tools performed making an N rate recommendation relative to EONR, each tool was also assessed on an economic and environmental basis.

Economic assessment
An analysis of variance (ANOVA) model comparing mean differences of the tools' partial profit relative to the EONR's partial profit showed a significant main effect for tool type (P ≤ .001) with no significant results for N application timing F I G U R E 8 Mean partial profit (in $ ha −1 ) for N recommendation tools relative to the economically optimal N rate (EONR). Tools used for both planting and split N application timing were not different (P = 0.99), and therefore recommendations shown are averaged across timings. Tools include farmer's nitrogen rate (NR), yield goal (YG), preplant nitrate test (PPNT), pre-sidedress nitrate test (PSNT) and late-spring nitrate test (LSNT) with 0 and 45 kg N ha −1 applied at-planting, Maximum Return to N (MRTN), Maize-N crop growth model, and canopy reflectance sensing using the Holland and Schepers algorithm. Significance means separation was determined using Tukey's honest significant test with a significance threshold of 0.05. Tools with the same letter, indicate that means are not significantly different from each other (P = .44) or the two-way interaction between tool type and application timing (P = .99). As a result, partial profit was averaged across timings for tools that gave recommendations for both ( Figure 8). Statistically, there is little difference among tools in partial profits, despite profit ranging from −$50-−154 ha −1 (excluding Maize-N; Figure 8).
The Maize-N crop growth model underestimated EONR and had more implementation costs compared to all other tools, which lead to an average loss in partial profits of −$269 ha −1 (Table 2). While unrealistic to think any tool could generate an N recommendation equivalent to EONR all the time, this analysis shows that most any tool would be profitable.
For farmers to adopt N recommendation tools, tools need to be affordable, simple to use, and profitable (Stuart, Schewe, & Mcdermott, 2014). Much of corn N for the US Midwest is currently applied in the fall or early spring before planting for convenience. Tools requiring soil or plant information and/or F I G U R E 9 Mean environmental cost (in $ ha −1 ) for N recommendation tools relative to the economically optimal N rate (EONR). Tools used for both planting and split N application timing were not different (P = 0.98), and therefore recommendations shown are averaged across timings. Tools include farmer's nitrogen rate (NR), yield goal (YG), preplant nitrate test (PPNT), pre-sidedress nitrate test (PSNT) and late-spring nitrate test (LSNT) with 0 and 45 kg N ha −1 applied at-planting, Maximum Return to N (MRTN), Maize-N crop growth model, and canopy reflectance sensing using the Holland and Schepers algorithm. Significance means separation was determined using Tukey honest significant difference test with a significance threshold of 0.05. Tools with the same letter, indicate that means are not significantly different from each other implemented in-season need to perform as well or better at determining N recommendations than tools that can be utilized any time of the year without specialized information to compensate for the inconvenience. This performance comparison also needs to consider specialized and sometimes expensive application equipment and the need to cover many acres within a narrow time frame.

Environmental assessment
An ANOVA model comparing mean differences of the tools' environmental cost and the EONR's environmental cost showed a significant main effect for tool type (P ≤ .001), but no significant difference for N application timing (P = .12) or the two-way interaction between tool type and application timing (P = .93). As such, tools that gave recommendations at both timings have environmental costs averaged across timings (Figure 9). The tools' average environmental costs ranged from -$185-54 ha −1 relative to EONR, and statistically, these values were significantly different among tools. A positive cost indicates that a tool's recommendation resulted in less N loss compared to EONR, giving an environmental credit. The General PPNT and General PSNT 45 (highest values) were significantly different from the IN YG, MO YG, General YG, IN PSNT 0, and Farmer NR (Figure 9). The majority of other N recommendation tools did not have environmental costs that were significantly different from each other. In general, the environmental costs are inversely related to how well the tool's N recommendation comes close to EONR. Tools that overestimate EONR have negative environmental costs, while tools that underestimate EONR have positive costs.
The lack of significant difference between the majority of the tools observed in our work is consistent with the results of Hong, Scharf, Davis, Kitchen, & Sudduth (2007) and Bandura (2017) who found no significant increase in residual soil NO 3 -N (i.e., N loss) until N rates exceeded EONR by about 30 kg N ha −1 . As only four tools recommended an N rate in excess of EONR by more than an average of 30 kg N ha −1 (Figure 3), minimal differences in total N loss between the majority of tools (i.e., tools close to EONR such as IN PSNT 45) and those with the largest negative costs (i.e., IN YG) were observed.

CONCLUSIONS
Many N recommendation tools are available to help farmers make N management decisions. An analysis was conducted using six metrics of performance for each of the tools based on their ability to: (i) predict EONR using a simple linear regression; (ii) match EONR based on the average difference between the N recommendation and EONR being close to zero, lowest RMSE, and the percentage of site cEONR; (iii) partial profits relative to EONR; and (iv) environmental costs relative to EONR. No N recommendation tool was a good predictor of EONR for all growing conditions of this study. Only 10 of the 31 tools evaluated had a significant positive (but weak) linear relationship with EONR (r 2 ≤ .20; P ≤ .07). This poor relationship could be the result of diverse soil and environmental conditions represented by the extensive geographic region of this study relative to the area from which the tool was developed and calibrated for N recommendations. Given this observation, successful tools were those based on soil sampling (e.g., PPNT and PSNT).
When trying to match EONR (cEONR), there were several tools that performed poorly (e.g., nearly all the YG approaches and the Maize-N crop growth model) while others did better (e.g., MRTN, PPNT, PSNT, and canopy reflectance sensing). None of these "better" performing tools showed any statistical difference for partial profits or environmental costs.
These findings demonstrate the difficulty of predicting EONR correctly, and that while current publicly-available N recommendation tools may be successful on individual fields or sub-regions, they were not universally reliable over the diversity of soils and weather in this study. Refinement of current tools or development of new tools that are adaptive and more responsive to soil and weather conditions have the potential for improved performance. Potentially utilizing multiple tools together to form an N recommendation may leverage the strengths of individual tools for better corn N management decisions. However, cost and implementation requirements need to be considered.