Locally Interpretable Model-agnostic

Explanations (LIME)
SolFinder Research
LIME
• Machine learning models are mostly black boxes.
• The purpose of lime is to explain the predictions of black box
classifiers.
• What this means is that for any given prediction and any given
classifier it is able to determine a small set of features in the original
data that are responsible for the outcome of the prediction.
• LIME can be used for machine learning models developed in Python
or R.
LIME
• Locally: Every complex model is linear on a local scale
• Interpretable: Representation that can be interpreted by humans
• Model-agnostic: Applied to any black box machine learning model
• Explanations: A statement that explains individual predictions.

LIME is a python/R library which tries provide model interpretability by

producing locally faithful explanations for predictions.
LIME
Algorithm
• Given an observation, permute it to create replicated feature data with
slight value modifications.
• Compute similarity distance measure between original observation and
permuted observations.
• Apply selected machine learning model to predict outcomes of permuted
data.
• Select m number of features to best describe predicted outcomes.
• Fit a simple model to the permuted data, explaining the complex model
outcome with m features from the permuted data weighted by its
similarity to the original observation .
• Use the resulting feature weights to explain local behavior.
LIME Flow
LIME Explainer
explainer = lime.lime_tabular.LimeTabularExplainer(<parameters>)
Parameters
• X_train = Training set
• feature_names = Concatenated list of all feature names
• class_names = Target values
• categorical_features = List of categorical columns in the dataset
• categorical_names = List of the categorical column names
• Kernel width = Parameter to control the linearity of the induced
model; the larger the width more linear is the model
LIME Explainer
exp=explainer.explain_instance(<observation>,<model_prediction_fn>,<
no_of_features>)
exp.show_in_notebook()
LIME Explainer ( R )
sentence_to_explain <- head(test_sentences[test_labels,]$text, 5)

explainer <- lime(sentence_to_explain, model = xgb_model,

preprocess = get_matrix)

explanation <- explain(sentence_to_explain, explainer, n_labels = 1,

n_features = 2)
LIME Feature Selection
Lime supports the following algorithms for feature selection:

none: Use all features for the explanation. Not advised unless you have very few
features.
forward selection: Features are added one by one based on their improvements to
a ridge regression fit of the complex model outcome.
highest weights: The m features with highest absolute weight in a ridge regression
fit of the complex model outcome are chosen.
lasso: The m features that are least prone to shrinkage based on the regularization
path of a lasso fit of the complex model outcome is chosen.
tree: A tree is fitted with log2(m) splits, to use at max m features. It may possibly
select less.
auto: Uses forward selection if m <= 6 and otherwise highest weights.
Summary
• For any given prediction and any given classifier it is able to
determine a small set of features in the original data that has driven
the outcome of the prediction.

• Basic assumption is that it is much easier to approximate a black-box

model by a simple model locally in the neighborhood of the
prediction to be explained.

• In this way LIME also enables us to extract bad features and create
new features to improve the model.
Further Information
• https://arxiv.org/abs/1602.04938
• https://cran.r-
project.org/web/packages/lime/vignettes/Understanding_lime.html
• https://github.com/marcotcr/lime
• https://github.com/thomasp85/lime