Inference and Summarization
Methods for summarization and inference of estimators in the CausalELM package
CausalELM.Inference
— ModuleMethods for summarization and inference from estimators and metalearners.
CausalELM.summarize
— Functionsummarize(study, mean_effect)
Return a summary from an event study.
Examples
julia> X₀, Y₀, X₁, Y₁ = rand(100, 5), rand(100), rand(10, 5), rand(10)
julia> m1 = EventStudy(X₀, Y₀, X₁, Y₁)
julia> estimatetreatmenteffect!(m1)
[0.25714308]
julia> summarize(m1)
{"Task" => "Regression", "Regularized" => true, "Activation Function" => relu,
"Validation Metric" => "mse","Number of Neurons" => 2,
"Number of Neurons in Approximator" => 10, "β" => [0.25714308],
"Causal Effect" => -3.9101138, "Standard Error" => 1.903434356, "p-value" = 0.00123356}
summarize(g)
Return a summary from a G-Computation estimator.
Examples
julia> X, Y, T = rand(100, 5), rand(100), [rand()<0.4 for i in 1:100]
julia> m1 = GComputation(X, Y, T)
julia> estimatetreatmenteffect!(m1)
[0.3100468253]
julia> summarize(m1)
{"Task" => "Regression", "Quantity of Interest" => "ATE", Regularized" => "true",
"Activation Function" => "relu", "Time Series/Panel Data" => "false",
"Validation Metric" => "mse","Number of Neurons" => "5",
"Number of Neurons in Approximator" => "10", "β" => "[0.3100468253]",
"Causal Effect: 0.00589761, "Standard Error" => 5.12900734, "p-value" => 0.479011245}
summarize(dre, n)
Return a summary from a doubly robust estimator.
Examples
julia> X, Y, T = rand(100, 5), rand(100), [rand()<0.4 for i in 1:100]
julia> m1 = DoublyRobust(X, X, Y, T)
julia> estimatetreatmenteffect!(m1)
[0.5804032956]
julia> summarize(m1)
{"Task" => "Regression", "Quantity of Interest" => "ATE", Regularized" => "true",
"Activation Function" => "relu", "Validation Metric" => "mse", "Number of Neurons" => "5",
"Number of Neurons in Approximator" => "10", "Causal Effect" = 0.5804032956,
"Standard Error" => 2.129400324, "p-value" => 0.0008342356}
summarize(m, n)
Return a summary from a metalearner.
Examples
julia> X, Y, T = rand(100, 5), rand(100), [rand()<0.4 for i in 1:100]
julia> m1 = SLearner(X, Y, T)
julia> estimatecate!(m1)
[0.20729633391630697, 0.20729633391630697, 0.20729633391630692, 0.20729633391630697,
0.20729633391630697, 0.20729633391630697, 0.20729633391630697, 0.20729633391630703,
0.20729633391630697, 0.20729633391630697 … 0.20729633391630703, 0.20729633391630697,
0.20729633391630692, 0.20729633391630703, 0.20729633391630697, 0.20729633391630697,
0.20729633391630692, 0.20729633391630697, 0.20729633391630697, 0.20729633391630697]
julia> summarise(m1)
{"Task" => "Regression", Regularized" => "true", "Activation Function" => "relu",
"Time Series/Panel Data" => "false", "Validation Metric" => "mse",
"Number of Neurons" => "5", "Number of Neurons in Approximator" => "10",
"β" => "[0.3100468253]", "Causal Effect: [0.20729633391630697, 0.20729633391630697,
0.20729633391630692, 0.20729633391630697, 0.20729633391630697, 0.20729633391630697,
0.20729633391630697, 0.20729633391630703, 0.20729633391630697, 0.20729633391630697 …
0.20729633391630703, 0.20729633391630697, 0.20729633391630692, 0.20729633391630703,
0.20729633391630697, 0.20729633391630697, 0.20729633391630692, 0.20729633391630697,
0.20729633391630697, 0.20729633391630697], "Standard Error" => 5.3121435085,
"p-value" => 0.0632454855}
CausalELM.Inference.quantitiesofinterest
— Functionquantitiesofinterest(model, n)
Generate a p-value and standard error through randomization inference
This method generates a null distribution of treatment effects by reestimating treatment effects from permutations of the treatment vector and estimates a p-value and standard from the generated distribution.
Note that lowering the number of iterations increases the probability of failing to reject the null hypothesis.
For a primer on randomization inference see: https://www.mattblackwell.org/files/teaching/s05-fisher.pdf
Examples
julia> x, y, t = rand(100, 5), rand(1:100, 100, 1), [rand()<0.4 for i in 1:100]
julia> g_computer = GComputation(x, y, t)
julia> estimatecausaleffect!(g_computer)
julia> quantitiesofinterest(g_computer, 1000)
(0.114, 6.953133617011371)
quantitiesofinterest(model, nsplits)
Generate a p-value and standard error through randomization inference
This method generates a null distribution of treatment effects by reestimating treatment effects from permutations of the treatment vector and estimates a p-value and standard from the generated distribution. Randomization for event studies is done by creating time splits at even intervals and reestimating the causal effect.
Note that lowering the number of iterations increases the probability of failing to reject the null hypothesis.
For a primer on randomization inference see: https://www.mattblackwell.org/files/teaching/s05-fisher.pdf
Examples
julia> x₀, y₀, x₁, y₁ = rand(1:100, 100, 5), rand(100), rand(10, 5), rand(10)
julia> event_study = EventStudy(x₀, y₀, x₁, y₁)
julia> estimatecausaleffect!(event_study)
julia> quantitiesofinterest(event_study, 10)
(0.0, 0.07703275541001667)
CausalELM.Inference.generatenulldistribution
— Functiongeneratenulldistribution(e, n)
Generate a null distribution for the treatment effect of G-computation, doubly robust estimation, or metalearning.
This method estimates the same model that is provided using random permutations of the treatment assignment to generate a vector of estimated effects under different treatment regimes. When e is a metalearner the null statistic is the difference is the ATE.
Note that lowering the number of iterations increases the probability of failing to reject the null hypothesis.
Examples
julia> x, y, t = rand(100, 5), rand(1:100, 100, 1), [rand()<0.4 for i in 1:100]
julia> g_computer = GComputation(x, y, t)
julia> estimatecausaleffect!(g_computer)
julia> generatenulldistribution(g_computer, 500)
[0.016297180690693656, 0.0635928694685571, 0.20004144093635673, 0.505893866040335,
0.5130594630907543, 0.5432486130493388, 0.6181727442724846, 0.61838399963459,
0.7038981488009489, 0.7043407710415689 … 21.909186142780246, 21.960498059428854,
21.988553083790023, 22.285403459215363, 22.613625375395973, 23.382102081355548,
23.52056245175936, 24.739658523175912, 25.30523686137909, 28.07474553316176]
generatenulldistribution(e, n, mean_effect)
Generate a null distribution for the treatment effect in an event study design. By default, this method generates a null distribution of mean differences. To generate a null distribution of cummulative differences, set the mean_effect argument to false.
Instead of randomizing the assignment of units to the treamtent or control group, this method generates the null distribution by reestimating the event study with the intervention set to n splits at even intervals within the total study duration.
Note that lowering the number of iterations increases the probability of failing to reject the null hypothesis.
For a primer on randomization inference see: https://www.mattblackwell.org/files/teaching/s05-fisher.pdf
Examples
julia> x₀, y₀, x₁, y₁ = rand(1:100, 100, 5), rand(100), rand(10, 5), rand(10)
julia> event_study = EventStudy(x₀, y₀, x₁, y₁)
julia> estimatecausaleffect!(event_study)
julia> generatenulldistribution(event_study, 10)
[-0.5012456678829079, -0.33790650529972194, -0.2534340182760628, -0.21030239864895905,
-0.11672915615117885, -0.08149441936166794, -0.0685134758182695, -0.06217013151235991,
-0.05905529159312335, -0.04927743270606937]