This text and reference focuses on various popular data assimilation methods, such as weak and strong constraint variational methods and ensemble filters and smoothers. Here, an additional yellow “interface routine” is used in between the model code and the PDAF library routine. Soc., 136, 1991–1999, 2010. , van Leeuwen, P. J., Künsch, H. R., Nerger, L., Potthast, R., and Reich, S.: 2020. a, Penny, S. G., Akella, S., Alves, O., Bishop, C., Buehner, M., Chevalier, M., R., Savcenko, R., Bosch, W., Skachko, S., and Danilov, S.: On the Adv. Here, the implementation of the analysis step uses the same domain decomposition as the models; hence, only the full ensemble for each process subdomain has to be collected by the DA coupling. There is a natural linkage between data assimilation and ensemble forecasting: ensemble forecasts are designed to estimate the flow-dependent uncertainty of the forecast; data assimilation techniques require accurate estimates of forecast uncertainty in order to optimally As a step in developing nearly generic ensemble filter assimilation systems, a method to estimate ‘localization’ functions is presented. The tutorials will cover many of the ensemble filter algorithms that are used today for geophysical applications. Meteor. Weather Rev., 144, 3–20, 2016. a, Gaspari, G. and Cohn, S. E.: Construction of Correlation Functions in Two and Thus, after obtaining the observations in a compartment, a cross-compartment observation vector is initialized using MPI communication. System with a Regional High-Resolution Atmosphere-Ocean Coupled Model Based The same SST observations as in Sect. 4.1 are assimilated, which are treated as in Tang et al. In the analysis step at time tk, the ESTKF transforms a forecast ensemble Xkf of Ne model states of size Nx stored in the columns of this matrix into a matrix of analysis states Xka as. (2016). This property is also important for coupled DA, where the state vector will be distributed over different compartments, such as the atmosphere and the ocean. Kalman filter for data assimilation in oceanography, J. Given that both model compartments in AWI-CM scale to larger processor numbers than we used for the DA experiment, we expect that the DA in AWI-CM with ECHAM at a resolution of T127 (i.e., about 1∘) could be run at a similar execution time as for T63 given that a higher number of processors would be used. Kalman Filter Technique, Mon. The code modifications for online coupling are described in Sect. 3.2, and the modifications of the parallelization are described in Sect. 3.3. Etna Explosive Eruption, Atmosphere, 11, 359, Penny, S. G., Akella, S., Alves, O., Bishop, C., Buehner, M., Chevalier, M., With this configuration, the assimilation can be performed independently for both compartments. Further, it provides functionality to adapt a model parallelization for parallel ensemble forecasts as well as routines for parallel communication linking the model and filters. Further, Karspeck et al. OASIS3-MCT computes the fluxes between the ocean and the atmosphere and performs the interpolation between both model grids. The most widely used class of ensemble DA methods are ensemble-based Kalman filters (EnKFs) like the local ensemble transform Kalman filter (LETKF; Hunt et al., 2007), the deterministic ensemble Kalman filter (DEnKF; Sakov and Oke, 2008), or the local error-subspace transform Kalman filter (LESTKF; Nerger et al., 2012b). The overall execution time is dominated by the time to compute the forecasts. uncertainty estimates, J. Geophys. The time for the DA coupling (blue line) varies by a factor of 2.5. Practically, one computes an error-subspace matrix by Further, the ensemble is initialized and the analysis step of the data assimilation can be executed at any time without restarting the model. Further, each observation is weighted according to its distance from the water column to down-weight observations at larger distances (Hunt et al., 2007). A conventional observation dataset and bias-corrected satellite temperature data are 對assimilated. In this way the coupler will also be initialized for an ensemble of model states. Weather Rev., While the RMSE of the salinity first increases during the first month, it is reduced from day 60, but until day 140 it is sometimes larger than at the initial time. However, we do not expect that a single atmospheric analysis step would require significantly more time than the ocean DA, so due to the parallelization, the overall run time should not increase by more than 10 %–20 %. A global coupled ensemble data assimilation system using the Community (2019) discussed the strongly coupled DA for a coastal ocean–biogeochemical model assimilating real observations of sea surface temperature. However, the LESTKF also performs MPI communication for gathering the observational information from different process domains. SEIK filter, Ocean Dynam., 56, 634–649, 2006. a, Nerger, L., Janjić, T., Schröter, J., and Hiller, W.: A regulated The maximum ensemble size was here limited by the batch job size of the used computer. Figure 8 shows the RMSE of the experiment DA-SST relative to the RMSE for the free run. Apart from the additional subroutine calls, a few changes were required in the source codes of ECHAM, FESOM, and OASIS3-MCT, which are related to the parallelization. For the coupled model, the routine is called in both ECHAM and FESOM. With this strategy, a wrapper that combines the compartment model into a single executable as used by Kurtz et al. parallelization”). Physica D, 230, 112–126, 2007. . Valcke, S.: The OASIS3 coupler: a European climate modelling community software, Geosci. assessment of oceanic variability for 1960–2010 from the GFDL ensemble The ocean model uses an unstructured triangular grid with 46 vertical layers. The routine includes calls to the routine “PDAF_print_info”, which print out information about execution times of different parts of the assimilation program as measured by PDAF as well as information about the memory allocated by PDAF. Data Assimilation comprehensively covers data assimilation and inverse methods, including both traditional state estimation and parameter estimation. For more information, please send email to which is monitored by all of the instructors. The framework devel-oped in AA allowed a synthesis of the data assimilation and ensemble generation problem. effect of topography representation, J. Geophys. Meteor. Remote Sensing, 11, 234, Burgers, G., van Leeuwen, P. J., and Evensen, G.: On the Analysis Scheme in the Principles and evaluation, Ocean Model., 6, 125–150, 2004. , Evensen, G.: Sequential data assimilation with a nonlinear quasi-geostrophic Shown are the relative RMSEs for temperature (blue) and salinity (green). First an overview of PDAF is given (Sect. 3.1). T., Zampieri, L., Losch, M., and Goessling, H. F.: Toward a data assimilation The combined time for the analysis and the pre- and post-step operations is only between 4 % and 7 % of the forecast time. The execution time of the coupled ensemble data assimilation program was dominated by the time to compute the ensemble integrations in between the time instances at which the observations are assimilated. Each of the two coupled compartment models were augmented in this way. coupled ocean–atmosphere data assimilation in the ECMWF NWP system, OpenMP: OpenMP Application Program Interface Version 3.0, available at: Pardini, F., Corradini, S., Costa, A., Ongari, T. E., Merucci, L., Neri, A., The module runs the ensemble of model runs and executes the experiment as defined by . Analogous to many large-scale geoscientific simulation models, PDAF is implemented in Fortran and is parallelized using the Message Passing Interface standard (MPI; Gropp et al., 1994) as well as OpenMP (OpenMP, 2008). For updating a column, only observations within a horizontal influence radius l are taken into account. Soc., 125, 723–757, 1999. . For localization (yellow), the localized analysis described in Sect. 2.1.1 requires several operations, which are provided by call-back routines. Weather Rev., 145, 565–581, 2017. ., 2019b. a, OpenMP: OpenMP Application Program Interface Version 3.0, available at: Lett., 43, 752–759, 2016. a, Snyder, C., Bengtsson, T., Bickel, P., and Anderson, J.: Obstacles to Meteorol. Here, a configuration is used that computes the filter analysis step on the first coupled model task using the same domain decomposition as the coupled model. OASIS3-MCT is linked into each program as a library. with the Message-Passing Interface, The MIT Press, Cambridge, Massachusetts, As such, the system is significantly faster than the coupled ensemble DA application by Karspeck et al. The major change for strongly coupled DA is to communicate the observations in between the compartments as mentioned above. The parallelization for the DA is configured by the routine init_parallel_pdaf. The blue and black lines show the maximum execution times. efficient particle filter, Q. J. Roy. Q. J. Roy. Further, information for the coupling will be initialized like the grid configuration, which is required by the coupler to interpolate data in between the different model grids. Thus, using the same computer, one could run a larger ensemble, with less processes per model and accordingly a larger run time, or a smaller ensemble with less run time. In offline-coupled DA one uses separate programs for the model and the assimilation and performs the data transfer between both through disk files. For the salinity, the effect of the DA is lower. However, some parameters are distinct: the time step size of ECHAM is 450 s, while it is 900 s for FESOM. B. Instead of integrating the state of a single model instance (“model task”), the model is modified to run an ensemble of model tasks. For the coupled model, “PDAF_init” and “PDAF_get_state” are called in each compartment. The ensemble allows the calculation of the uncertainty of its atmospheric variables at the time of the analysis. For example, a routine provides PDAF with the number of observations, which is obtained by reading the available observations and counting them. The resolution of the observations is 0.1∘ and hence higher than the resolution of the model in most regions. The analysis step will then be executed in each compartment according to the configuration of the assimilation. A global coupled ensemble data assimilation system using the Community Kurtz, W., He, G., Kollet, S. J., Maxwell, R. M., Vereecken, H., and Hendricks Franssen, H.-J. Charron, Martin, Spacek, Lubos & Hansen, Bjarne 2005 Atmospheric data assimilation with an ensemble kalman lter: Results with real observations. The distribution of the processes is exemplified in Fig. 3a for the case of six processes in MPI_COMM_WORLD. These routines are executed as call-back routines and can be implemented like routines of the numerical model, which should simplify their implementation. Further influences are the parallel communication within each compartment at each time step and the communication for the model coupling by OASIS3-MCT that is performed at each model hour. By now the weakly coupled assimilation is the common choice for assimilation into coupled models and recent studies assess the effect of this assimilation approach. Calls to interface routines (yellow) are inserted into the model code (blue). Ensemble filter data assimilation algorithms use a set (ensemble) of model state estimates to enable the assimilation process. Geosci., 55, 110–118, 2013. , Nerger, L., Hiller, W., and Schröter, J.: PDAF - The Parallel These variations are due to the fact that the large compute application is widely spread over processors of the computer. The output files containing the timing information, the outputs from the 1-year experiments, and plotting scripts are available at Zenodo (, Nerger et al., 2019b). Commonly, DA is applied to separate models simulating, for example, the atmospheric dynamics or the ocean circulation. Thus, through Eq. (1), the algorithm can directly update both compartments, xA and xO, using observations of just one compartment. Climate, 26, 10218–10231, 2013. , Harlim, J. and Hunt, B. R.: Four-dimensional local ensemble transform Kalmn The difference is that in the online coupling, model information like the model grid is initialized by the model code and usually stored in, for example, Fortran modules. (2012b), one can distinguish between offline and online DA coupling. Given that the atmospheric analysis step would typically be applied after each sixth hour, the time for the DA coupling and the analysis steps would increase. For coupled models consisting of multiple executables, this call structure is used for each compartment model. These routines are executed by all processes that participate in the model integrations, and each routine acts on its process subdomain. Thus, only a single-line call to each interface routine is added to the model code, which keeps the changes to the model code to a minimum. Hence values below one indicate improvements. As the computer is also used by other applications, it is likely that the application is widely spread over the computer so that even different compute racks are used. R., Wang, Y., and Wu, X.: Coupled data assimilation for integrated Earth 2012b. , Nerger, L., Schulte, S., and Bunse-Gerstner, A.: On the influence of model Figure 5 shows the execution times per model day for different parts of the assimilation program. This text and reference focuses on various popular data assimilation methods, such as weak and strong constraint variational methods and ensemble filters and smoothers. However, the core part of the filter, which computes the corrected state vector (the so-called “analysis state”) taking into account the observational information, does not need to know how the state vector is constructed. The modification of the model parallelization for ensemble DA is a core element of the DA online coupling. SMOS sea ice thickness data simultaneously, Q. J. Roy. However, the strategy can be easily used for other model systems consisting of a single or multiple executables. Here, the routine is provided with a state vector x from the ensemble and has to return the observed state vector, i.e., H(x). This study explains the required modifications to the programs with the example of the coupled atmosphere–sea-ice–ocean model AWI-CM (AWI Climate Model). Part I: model formulation and mean climate, Clim. Meteor. ocean-biogeochemical model: Assessment of weakly- and strongly-coupled data Afterwards, the RMSE remains nearly constant, which is a typical behavior. The method of hierarchical modelling allows us to calculate these probabilities. This resulted in an overhead which was, depending on the ensemble size, only up to 15 % in computing time compared to the model without assimilation functionality. A common approach is to use an ensemble of models. Abstract. This text and reference focuses on various popular data assimilation methods, such as weak and strong constraint variational methods and … (AWI-CM-PDAF version 1.0), Zenodo,, Below, the time index k is omitted, as all computations in the analysis refer to time tk. Karspeck et al. After this step, all involved processes of the program are active (for the parallelization aspects see Sect. 3.3). A common approach for data assimilation is ensemble-based methods which utilize an ensemble of state realizations to estimate the state and its uncertainty. temperature and salinity profiles and monthly objective analyses with Given that each model task writes its own restart files in a separate directory, a model restart is possible from these files without any adaptions to the model code. Open questions for strongly coupled DA are, for example, how to account for the different temporal and spatial scales in the atmosphere and the ocean. Since the model grid is unstructured with varying resolution, super-observations are generated by averaging onto the model grid. Strongly coupled DA is a much younger approach, which is not yet well established. In ensemble data assimilation, b T is approximated by using the sample covariance estimated from an ensemble of model forecasts. Partly, the mentioned studies used twin experiments assimilating synthetic observations to assess the DA behavior. When the time for the DA coupling is subtracted from the forecast time, the variability is much reduced as the black dashed line shows. There are four types of routines: transfers between model fields and state vector (cyan), observation handling (orange), treatment of localization (yellow), and pre- and postprocessing (blue). 142, 65–78, 2016. . into an ensemble data assimilation system using MPI, Environ. Lorkowski, I., and Nerger, L.: Temperature assimilation into a coastal Here, the communicator is split into four processes for COMM_FESOM (green) and two for COMM_ECHAM (blue). In a weakly coupled application of DA, COMM_FILTER is initialized so that two separate communicators are created: one for all subdomains of FESOM and another one for all subdomains of ECHAM as shown in Fig. 3c. The ensemble data assimilation employs a configuration of 40+1, i.e. Given that the SST observations are assimilated, it is a necessary condition for the DA to reduce the deviation from these observations. Data Assimilation comprehensively covers data assimilation and inverse methods, including both traditional state estimation and parameter estimation. individual phytoplankton fields, J. Geophys. model using Monte Carlo methods to forecast error statistics, J. Geophys. In addition, the length of the initial forecast phase, i.e., the number of time steps until the first analysis step, is initialized.
2020 ensemble data assimilation