- Siddharth Arora(@sarora)
- Yinghua Guan(@vinverguan)
- Abishek Murali(@abimur-123)
To install please execute the following in R:
devtools::load_all()
devtools::install_github("UBC-MDS/BlackBoxR",build_vignettes = TRUE)
The Bayesian vs Frequentist approach is more of a philosophical debate which this package will not delve into. This package attempts at breaking down the understanding and the underlying assumptions of the 2 approaches and how they compare. The package will run a significance analysis using both approaches based on data provided by the user, compare credible and confidence intervals and finally debunks the understanding of MAP and MLE for parameter estimation.
This package is aimed at users who are attempting to familiarize themselves with the Bayesian/Frequentist approach(although I'm guessing it will be more Bayesian). This package can elucidate the difference in approaches and will attempt to help the user get a basic high-level understanding of both approaches and how they should proceed to carry out further analysis.
getCredibleInterval(x,prior\_dis,sample\_dis)
Purpose: obtain credible intervals using Bayesian approach(we now just accept normal distribution data, may accept more distribution in future )
- x, a numeric vector
- prior_dis, a numeric vector
- sample_dis, a numeric vector
numeric vector with length 2
Example
sample<-rnorm(5,mean=3,sd=1)
getCredibleInterval(sample,c(2,1),c(3,1))
getConfidenceInterval(x)
Purpose: Obtain confidence interval for the result(we now just accept normal distribution data, may accept more distribution in future)
- x
- a numeric vector
numeric vector with length 2
Example
sample<-rnorm(5,mean=2, sd =0.98)
getConfidenceInterval(sample)
Purpose:
A/B testing is an experiment with 2 versions - A and B. It is a two sample hypothesis testing which compares the subject's response to 2 versions of an entity(like a website).
performABTest(data,alpha = 0.05)
- data: input dataframe with 2 columns: name and event. Name consists of the A and B values(2 website pages) and event consists of the outcome of the event(0 or 1). This dataframe should have at least 20 rows to be able to perform AB tests.
- alpha: This defines the false positive rate while testing. Default value is 0.05
Example
df<-data.frame(name=rep(c('A','B'),100),events=rbinom(200,1,0.5))
op <- performABTest(df,0.05)
Purpose:
Maximum likelihood estimation (MLE) is a method of estimating the parameters of a statistical model, given observations. It attempts to find the parameter values that maximize the likelihood function, given the observations.
getMLE(distribution,column): Get maximum likelihood value of the parameter for a given distribution.
- distribution: type of distribution of the data: Supporting bernoulli and poisson as of now
- column: the column is a vector of numeric data over which likelihood is performed
log likelihood of the data. For example, mean for Poisson, probability for Bernoulli.
Example
bernoulli_column <- c(0,1,1,0,1,0,1,1,1,1,1)
getMLE("bernoulli",bernoulli_column)
poisson_column <- c(0,1,2,3,1,2,3,9,6,10,11)
getMLE("poisson",poisson_column)
This is work in progress.
After completion one would be able to perform
- AB testing using the Bayesian approach
- Get Maximum A Priori estimate(MAP)
The BayesAB package does AB testing using Bayesian approach with different distributions. The prop.test function in R performs proportion tests in R.