Model Based Policy Analysis (Subject) / Everything but Manski (Lesson)
There are 12 cards in this lesson
Comprehensive policy model, uncertainty, modeling policy choices, PIF, GGF, CAADP, Bayes, meta modeling
This lesson was created by Moadscha.
- What belongs to a comprehensive policy analysis? To achieve future sustainable economic growth different public policies (e.g. SDGs) are considered as the key determinants à participatory policy Policy analysis: “The term policy analysis describes the scientific evaluations of the impacts of past public policies and predictions of the outcomes of potential future policies.” (Manski,2018) When modeling policy (doing policy analysis) we need to look at: 1. Goals --> incentives 2. Statistical data and what influence it has 3. Assumptions 4. Institutions, law, rules, decisions 5. Uncertainty in data 6. Potential effects à side effects 7. Technology 8. Transformation process We need: - Economic system AND political system - Understanding the choice of polices (political process of formulating policies) is crucial - We need to understand the influence of the political system on the economy: policies can impact the economy for example via taxes (directly) , investment incentives (indirectly) - How to get the economic system (theory) and the political system (theory) together? - Empirical evidence? - How would a policy change beliefs? How to model this change? - Policy impact --> policy drives factor changes à policy influences outcomes à How to model policy into that process of economy?
- What challenges are there concerning comprehensive policy analysis? Challenges concerning comprehensive policy analysis: 1. Integrating policy instruments into scientific models 2. Integarting models of different scientific disciplines into a concictent model framework 3. Empirical specification of modules 4. Including model uncertainty into policy analysis 5. Incorporating a positive model of real-life political decision making 6. Specifying individual political belliefs or relevant political agents and stakeholders 7. Modeling policy learning and belief updating
- What is a CGPE? What doe sit consit of? CGPE= Computable General Political Equilibrium CGPE is a CGE designed to analyze policies A CGPE is divided in 4 modules: 1. Political system: a) Political belief formation module b) Political decision making module 2. Economic system: a) Policy impact function module b) Economic equilibrium module What needs to be modeled? - The drerivation (Herkunft) of policy makers incentives from electoral competition and lobbying --> modeling voter behavior and interest group activities - Modeling legislative bargaining (Verhandeln) --> the derivation of a collective policy decision by a set of heterogenous legislators based on constitutional rules - Economic modeling of policy impacts --> the transformation of policies into outcomes - Modeling of policy learning --> the formulation and updating of policy beliefs via observational and communication level
- How can we model policy choice? What needs to be modeled (integrated into the model)? CGPE: Political decision-making module: - Choose second best alternative instead of first best - Political decisions ≠ science - Electoral competition and lobbying influence the decision-making process - Problem: political decision process is complex, partly because many people and opinions are involved Policy --> Policy outcome Z --> outcome evaluation --> Indiividual policy preferences --> policy What needs to be modeled? Policy choice = political power + political incentives + political knowledge Political knowledge = knowledge of the transformation process. For CAADP: policy intervention --> economic growth --> Growth in policy outcome PIF= policy impact function + CGE --> modeling political knowledge Political Power: How much influence have decision makers? Political incentives: utility function “social welfare function” (measured by surveys) à modeling political incentives constitutional rules + bargaining game (network study) à modeling political power
- Model uncertainty: What kinds of uncertainty are there? 1. strutural uncertainty: - model structure - assumptions - variables used in regression (depends on which variables you choose) - parameter values (elasticities in a CGE) - model selection and model averaging overfitting and underfitting overfitting In statistics, overfitting is "the production of an analysis that corresponds too closely or exactly to a particular set of data, and may therefore fail to fit additional data or predict future observations reliably". An overfitted model is a statistical model that contains more parameters than can be justified by the data.As an extreme example, if the number of parameters is the same as or greater than the number of observations, then a model can perfectly predict the training data simply by memorizing the data in its entirety. Such a model, though, will typically fail severely when making predictions. underfitting Underfitting occurs when a statistical model cannot adequately capture the underlying structure of the data. An underfitted model is a model where some parameters or terms that would appear in a correctly specified model are missing. 2. Parameter uncertainty: - estimated values - distribution of estimates à distribution of results This comes from the model parameters that are inputs to the computer model (mathematical model) but whose exact values are unknown to experimentalists and cannot be controlled in physical experiments, or whose values cannot be exactly inferred by statistical methods. Some examples of this are the local free-fall acceleration in a falling object experiment, various material properties in a finite element analysis for engineering, and multiplier uncertainty in the context of macroeconomic policy optimization. (Wikipedia) 3. Data uncertainty: - limited data, few observations,… - measurement errors of data (sensor data, indicators,…) Policy choice: chose policy that maximizes the objective function Intervention logic: logic how a policy affects the desired aims of the objective function Not just one but many models, true model m is part of the set M. Decision-maker is uncertain about which model is the true one, only partial knowledge (subjective beliefs). To get closer use of the expected utility à maximization of all possible models; ignoring uncertainty would lead to the use of only one model Agents g preselect model m out of M, might be based on subjective beliefs or be random Scientists suggest only one model, with a probability of 1 as the correct model Derived policy choices in both cases will lead to a loss of expected utility and are therefore inefficient
- What is CAADP? Comprehensive Africa Agriculture Programme Africa’s policy framework for agricultural transformation, wealth creation, food security and nutrition, economic growth and prosperity for all. First declaration on CAADP was 2003. Over 41 AU member states singned CAADP contracts. A great share of those countries has developed formal national agriculture and food security investment plans. These have become their medium-term expenditure frameworks for agriculture. - economic policy to reach goals like poverty reduction by pro poor growth - 10% of national budget should be contributed to agriculture to achieve a 6& annual growth rate and improveme food and nutrition security - policy instruments: Natural resource management (NR), farm management (FM), Market access (MA) and human resource management (HR)
- Explain the intervention logic of CAADP. CAADP intervention logic Policy intervention --> Economic Growth --> Growth in Policy Outcome (Z) Pro-Poor Growth (PPG), dh der strategische Ansatz, mit dem Wirtschaftswachstum speziell zur Armutsbekämpfung eingesetzt wird, steht ganz oben auf der entwicklungspolitischen Agenda Example: Allocate budget to farm management à pay professionals to teach farmers how to manage their farm more efficiently à more/better crops = economic growth à higher farm income Policy intervention --> Economic growth: PIF, e.g. how does it affect economic growth when a government spends a certain amount of budget on HR? Economic growth --> Growth in policy outcome: GGF - PIF: Bayesian estimation combining statistical data with expert information - GGF: Meta-modeling approach based on a micro-macro linked CGE Policy intervention (agr): Natural Resources, Human Resources, Farm Management, nonagr Policy --> Technical Progress Economic Growth --> Growth in Policy Outcome Technical Progress --> Agricultural sector: Crop, Lifestock, Fish, Export Crop, Agribusiness --> Economic Growth --> Policy Outcomes Policy intervention (nonagr): Nonagr Policy, Infrastrutructure Roads, Infrastruture Storages --> Market Access Economic Growth --> Growth in Policy Outcome Market Access --> Nonagr Sector: Industry, Trade, Services, Public Services --> Economic Growth --> Policy Outcomes Policy Outcomes: Farm income, Poverty, Public Goods, Export Crop, Urban Income, Industry, Sustainability
- What is a PIF? PIF= policy impact function Policies like taxes or tariffs can be easily simulated in a CGE model, but policies like policy programs aiming to increase technological progress must be translated into CGE parameters. PIF is defined as a transformation of policy instruments into relevant CGE parameters that correspond to sector-specific technological progress or transaction costs. Policy intervention --> Economic growth: PIF, e.g. how does it affect economic growth when a government spends a certain amount of budget on HR? intervention logic: Z= GGF(PIF(γ)) β(γ) = PIF (γ) β: changed factors, tp shock in CGE γ: policy intervention, budget allocation to different policy instruments PIF function needs to be estimated for each CAADP policy: NR, FM, MA, HR two staged approach 1. lower stage: the budget is transformed into effective budget for each sector (agr: e.g. FM-Food, NR-Water etc. and nonagr) 2. upper stage: The effective budget is transformed into technical progress
- How is it possible to estimate PIF? How to apply Bayesian estimation? Problem: 1. About 45 sectors and 10 policy instruments --> about 600 parameters needed to be estimated for the policy impact function (PIF) --> That would require large amounts of data in order to estimate using classical estimation approaches. Even if there were sufficient data, there would still be estimation problems. 2. How well does the past performance explain potential future behavior? 3. PIF: Parameter uncertainty (limited data, expert opinions) and data uncertainty Possible solution: limited data but prior knowledge/assumptions about parameter values --> Bayesian estimation approach. Politician probably have some knowledge (experience)--> that knowledge can be integrated into Bayesian estimation. Bayesian estimation approach: Classical view: parameters are fixed, data are random quantities Bayesian view: data fixed, parameters are random quantities Focus on maximum posterior (MAP) technique Two steps 1. Use historical data to find a first good parameter fit --> empirical PIF: The model replicates past observations. 2. Find parameters, that match expected future developments using expert knowledge --> expert PIF (The PIF parameters could be estimated econometrically by using the observed optimal policy positions and the preferred policy outcomes of a set of political actors.) Bayesian parameter estimation assumption: There is an unknown but objectively fixed parameter θ (teta). It chooses the value of θ which maximizes the likelihood of observed data. In other words: making the available data as likely as possible. A common example is the maximum likelihood estimator. Prior and Posterior knowledge A prior probability is the probability available beforehand, and before making any additional observations. A posterior probability is the probability obtained from the prior probability after making additional observations (survey on expert knowledge) to the prior knowledge available. Estimate the paramter: We compute the probability of a parameter given the likelihood of data. --> Bayesian parameter estimation specifies how we should update our beliefs in the light of newly introduced evidence. Prior: Values taken from empirical PIF Posterior: expert knowledge The Bayesian approach to parameter estimation works as folows: 1. Formulate our knowledge about a simulation 2. Gather data (e.g. ask governmental and NGOS to subdivide the CAADP budget across four pillars and 9 CAADP programs) 3. Obtain posterior knowledge to update our beliefs The posterior is obtained by 1. Multiplying the prior probability of each possible parameter value by its likelihood. 2. Obtaining the marginal likelihood adding the values obtained in 1. 3. Dividing the values obtained in 1. by the value obtained in 2.
- What is meta modeling? Approach? How to? Meta modeling is needed for GGW, in order to combine GGF and CGE and to use the Bayesian model selection. A metamodel models the behavior of another model. underlying problems: 1. computational time and expenses of CGE model 2. Discussion of elasticity parameter estimation --> causes uncertainty 3. Closure rules --> causes uncertainty 4. difficulty of combining results with other analysis (CGE is too complex) Ad 1) meta models are simple to construct and understand, less time and computational expenses. They are mathematical functions, e.g. z=ß0+ß1x1+ß2x2 Ad 4) meta model can be integrated into decision-making model CGE-models cannot be used for Bayesian model selection, but meta model can be used Byesian model selection: Choose the "correct" model --> backward approach: assumption: We have a complete database containing all possible models --> the "correct" is one of them and only needs to be selected The meta modeling process: 1. Determining input(s) and output(s) 2. Selecting metamodel types LOPM: low-order polynomial metamodel or Kriging) and DOE (=design of experiments) 3. Running simulation and collecting data 4. Estimating and validating the metamodel 5. Using metamodel for intended purposes Methodology Inputs and Outputs - elasticity parameters are generated, not estimated - elasticity parameters are treated as independent variables due to the assumption that they affect the output Sampling (DOE) Conventionally, elasticity parameter combinations are assumed or estimated, here they are generated using DOE. Problem: If we run all possible scenarios that could be a number that is not feasable to handle --> DOE --> select type of DOE: Central composite design or Latin Hypercube Fitting and validation - collect simulation inputs: DOE - collect simulation outputs: by running the simulation scenarios - indicators (e.g. R2 adj.) determine whether model fitting is satisfactory
- What is a Growth Goal Function? Grwoth Goal Fuction Economic growth --> Growth in policy outcome: GGF GGF structural uncertainty (assumptions) and data uncertainty (Results driven by assumptions • functional forms • closure rules • trade/production elasticities) Different sectors will most likely have different effects on various goals. In order to analyzethis relationship a growth goal function (GGF) (w = GGF(f)) will be used tomodel this relationship. We want to find key sectors and key policies that are needed to best improve a specific goal The growth goal function (GGF) captures the relationship between economic growth and policy goals. This relationship is modeled in the CGE model. The applied target measure for the growth will be the growth rates w of the policy goals. Since the CGE model is a rather complex model that can not easily be included in further estimations, we will derive the GGF as a metamodel of the CGE
- DOE: Latin Hyper Cube sample design. LHS goal: choose data points to fit the metamodel chracteristics: space filling design, that arranges sample points as spread out as possible across the design space. given: n variables xi --> e.g. n=2 --> x1, x2 needed: p sample points to fit the meta model - intervals of every variable are divided into p subintervals --> one value is choosen out of every subinterval (based on probability density) --> p values of x1 are randomly paired with p-values of x2 --> pairing with p-values of x3 --> ... --> until pn-tuplets are formed = exactely the p sample points needed for simulation.