Understanding Black-box Predictions via Influence Functions In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Biggio, B., Nelson, B., and Laskov, P. Poisoning attacks against support vector machines. Time permitting, we'll also consider the limit of infinite depth. All information about attending virtual lectures, tutorials, and office hours will be sent to enrolled students through Quercus. (b) 7 , 7 . 2172: 2017: . To run the tests, further requirements are: You can either install this package directly through pip: Calculating the influence of the individual samples of your training dataset Pearlmutter, B. We have a reproducible, executable, and Dockerized version of these scripts on Codalab. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby . Understanding Black-box Predictions via Influence Functions This could be because we explicitly build optimization into the architecture, as in MAML or Deep Equilibrium Models. Bilevel optimization refers to optimization problems where the cost function is defined in terms of the optimal solution to another optimization problem. your individual test dataset. He, M. Narayanan, S. Gershman, B. Kim, and F. Doshi-Velez. Stochastic gradient descent as approximate Bayesian inference. Depending what you're trying to do, you have several options: You are welcome to use whatever language and framework you like for the final project. Fortunately, influence functions give us an efficient approximation. Things get more complicated when there are multiple networks being trained simultaneously to different cost functions. We'll consider bilevel optimization in the context of the ideas covered thus far in the course. In, Martens, J. If Influence Functions are the Answer, Then What is the Question? we develop a simple, efficient implementation that requires only oracle access to gradients Some of the ideas have been established decades ago (and perhaps forgotten by much of the community), and others are just beginning to be understood today. Understanding Blackbox Prediction via Influence Functions - SlideShare >> For details and examples, look here. ( , ?) We would like to show you a description here but the site won't allow us. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. Optimizing neural networks with Kronecker-factored approximate curvature. ICML 2017 best paperStanfordPang Wei KohPercy liang, x_{test} y_{test} label x_{test} , n z_1z_n z_i=(x_i,y_i) L(z,\theta) z \theta , \hat{\theta}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta), z z \epsilon ERM, \hat{\theta}_{\epsilon,z}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta)+\epsilon L(z,\theta), influence function, \mathcal{I}_{up,params}(z)={\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0}=-H_{\hat{\theta}}^{-1}\nabla_{\theta}L(z,\hat{\theta}), H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta) Hessien, \begin{equation} \begin{aligned} \mathcal{I}_{up,loss}(z,z_{test})&=\frac{dL(z_{test},\hat\theta_{\epsilon,z})}{d\epsilon}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T {\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T\mathcal{I}_{up,params}(z)\\&=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta) \end{aligned} \end{equation}, lossNLPer, influence function, logistic regression p(y|x)=\sigma (y \theta^Tx) \sigma sigmoid z_{test} loss z \mathcal{I}_{up,loss}(z,z_{test}) , -y_{test}y \cdot \sigma(-y_{test}\theta^Tx_{test}) \cdot \sigma(-y\theta^Tx) \cdot x^{T}_{test} H^{-1}_{\hat\theta}x, \sigma(-y\theta^Tx) outlieroutlier, x^{T}_{test} x H^{-1}_{\hat\theta} Hessian \mathcal{I}_{up,loss}(z,z_{test}) resistencevariation, \mathcal{I}_{up,loss}(z,z_{test})=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta), Hessian H_{\hat\theta} O(np^2+p^3) n p z_i , conjugate gradientstochastic estimationHessian-vector productsHVP H_{\hat\theta} s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta) \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta) , H_{\hat\theta}^{-1}v=argmin_{t}\frac{1}{2}t^TH_{\hat\theta}t-v^Tt, HVPCG O(np) , H^{-1} , (I-H)^i,i=1,2,\dots,n H 1 j , S_j=\frac{I-(I-H)^j}{I-(I-H)}=\frac{I-(I-H)^j}{H}, \lim_{j \to \infty}S_j z_i \nabla_\theta^{2} L(z_i,\hat\theta) H , HVP S_i S_i \cdot \nabla_\theta L(z_{test},\hat\theta) , NMIST H loss , ImageNetInceptionRBF SVM, RBF SVMRBF SVM, InceptionInception, Inception, , Inception591/60059133557%, check \mathcal{I}_{up,loss}(z_i,z_i) z_i , 10% \mathcal{I}_{up,loss}(z_i,z_i) , H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta), s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta), \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta), S_i \cdot \nabla_\theta L(z_{test},\hat\theta). Abstract. The infinitesimal jackknife. This leads to an important optimization tool called the natural gradient. ": Explaining the predictions of any classifier. . Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. On linear models and convolutional neural networks, The power of interpolation: Understanding the effectiveness of SGD in modern over-parameterized learning. logistic regression p (y|x)=\sigma (y \theta^Tx) \sigma . Understanding Black-box Predictions via Influence Functions Cook, R. D. Detection of influential observation in linear regression. ICML 2017 best paperStanfordPang Wei KohCourseraStanfordNIPS 2019influence functionPercy Liang11Michael Jordan, , \hat{\theta}_{\epsilon, z} \stackrel{\text { def }}{=} \arg \min _{\theta \in \Theta} \frac{1}{n} \sum_{i=1}^{n} L\left(z_{i}, \theta\right)+\epsilon L(z, \theta), \left.\mathcal{I}_{\text {up, params }}(z) \stackrel{\text { def }}{=} \frac{d \hat{\theta}_{\epsilon, z}}{d \epsilon}\right|_{\epsilon=0}=-H_{\tilde{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta}), , loss, \begin{aligned} \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) &\left.\stackrel{\text { def }}{=} \frac{d L\left(z_{\text {test }}, \hat{\theta}_{\epsilon, z}\right)}{d \epsilon}\right|_{\epsilon=0} \\ &=\left.\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} \frac{d \hat{\theta}_{\epsilon, z}}{d \epsilon}\right|_{\epsilon=0} \\ &=-\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} H_{\hat{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta}) \end{aligned}, \varepsilon=-1/n , z=(x,y) \\ z_{\delta} \stackrel{\text { def }}{=}(x+\delta, y), \hat{\theta}_{\epsilon, z_{\delta},-z} \stackrel{\text { def }}{=}\arg \min _{\theta \in \Theta} \frac{1}{n} \sum_{i=1}^{n} L\left(z_{i}, \theta\right)+\epsilon L\left(z_{\delta}, \theta\right)-\epsilon L(z, \theta), \begin{aligned}\left.\frac{d \hat{\theta}_{\epsilon, z_{\delta},-z}}{d \epsilon}\right|_{\epsilon=0} &=\mathcal{I}_{\text {up params }}\left(z_{\delta}\right)-\mathcal{I}_{\text {up, params }}(z) \\ &=-H_{\hat{\theta}}^{-1}\left(\nabla_{\theta} L(z_{\delta}, \hat{\theta})-\nabla_{\theta} L(z, \hat{\theta})\right) \end{aligned}, \varepsilon \delta \deltaloss, \left.\frac{d \hat{\theta}_{\epsilon, z_{\delta},-z}}{d \epsilon}\right|_{\epsilon=0} \approx-H_{\hat{\theta}}^{-1}\left[\nabla_{x} \nabla_{\theta} L(z, \hat{\theta})\right] \delta, \hat{\theta}_{z_{i},-z}-\hat{\theta} \approx-\frac{1}{n} H_{\hat{\theta}}^{-1}\left[\nabla_{x} \nabla_{\theta} L(z, \hat{\theta})\right] \delta, \begin{aligned} \mathcal{I}_{\text {pert,loss }}\left(z, z_{\text {test }}\right)^{\top} &\left.\stackrel{\text { def }}{=} \nabla_{\delta} L\left(z_{\text {test }}, \hat{\theta}_{z_{\delta},-z}\right)^{\top}\right|_{\delta=0} \\ &=-\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z, \hat{\theta}) \end{aligned}, train lossH \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) , -y_{\text {test }} y \cdot \sigma\left(-y_{\text {test }} \theta^{\top} x_{\text {test }}\right) \cdot \sigma\left(-y \theta^{\top} x\right) \cdot x_{\text {test }}^{\top} H_{\hat{\theta}}^{-1} x, influence functiondebug training datatraining point \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) losstraining pointtraining point, Stochastic estimationHHHTFO(np)np, ImageNetdogfish900Inception v3SVM with RBF kernel, poisoning attackinfluence function59157%77%10590/591, attackRelated worktraining set attackadversarial example, influence functionbad case debug, labelinfluence function, \mathcal{I}_{\text {up,loss }}\left(z_{i}, z_{i}\right) , 10%labelinfluence functiontrain lossrandom, \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right), \mathcal{I}_{\text {up,loss }}\left(z_{i}, z_{i}\right), \mathcal{I}_{\text {pert,loss }}\left(z, z_{\text {test }}\right)^{\top}, H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z, \hat{\theta}), Less Is Better: Unweighted Data Subsampling via Influence Function, influence functionleave-one-out retraining, 0.86H, SVMhinge loss0.95, straightforwardbest paper, influence functionloss. The list Kelvin Wong, Siva Manivasagam, and Amanjit Singh Kainth. Besides just getting your networks to train better, another important reason to study neural net training dynamics is that many of our modern architectures are themselves powerful enough to do optimization. Measuring the effects of data parallelism on neural network training. The answers boil down to an observation that neural net training seems to have two distinct phases: a small-batch, noise-dominated phase, and a large-batch, curvature-dominated one. Github We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. The canonical example in machine learning is hyperparameter optimization. How can we explain the predictions of a black-box model? The previous lecture treated stochasticity as a curse; this one treats it as a blessing. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. ordered by helpfulness. % Automatically creates outdir folder to prevent runtime error, Merge branch 'expectopatronum-update-readme', Understanding Black-box Predictions via Influence Functions, import it as a package after it's in your, Combined, the original paper suggests that. This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited. Influence functions are a classic technique from robust statistics to identify the training points most responsible for a given prediction. International Conference on Machine Learning (ICML), 2017. We are given training points z 1;:::;z n, where z i= (x i;y i) 2 XY . We'll use linear regression to understand two neural net training phenomena: why it's a good idea to normalize the inputs, and the double descent phenomenon whereby increasing dimensionality can reduce overfitting. Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. Are you sure you want to create this branch? Components of inuence. Overwhelmed? when calculating the influence of that single image. The first mode is called calc_img_wise, during which the two However, in a lower Data-trained predictive models see widespread use, but for the most part they are used as black boxes which output a prediction or score. This class is about developing the conceptual tools to understand what happens when a neural net trains. Data poisoning attacks on factorization-based collaborative filtering. below is divided into parameters affecting the calculation and parameters A. M. Saxe, J. L. McClelland, and S. Ganguli. If the influence function is calculated for multiple This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. A spherical analysis of Adam with batch normalization. reading both values from disk and calculating the influence base on them. Interacting with predictions: Visual inspection of black-box machine learning models. No description, website, or topics provided. In this lecture, we consider the behavior of neural nets in the infinite width limit. With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Class will be held synchronously online every week, including lectures and occasionally tutorials. In. Cook, R. D. and Weisberg, S. Characterizations of an empirical influence function for detecting influential cases in regression. ( , , ). In, Cadamuro, G., Gilad-Bachrach, R., and Zhu, X. Debugging machine learning models. In contrast with TensorFlow and PyTorch, JAX has a clean NumPy-like interface which makes it easy to use things like directional derivatives, higher-order derivatives, and differentiating through an optimization procedure. We have 3 hours scheduled for lecture and/or tutorial. Simonyan, K., Vedaldi, A., and Zisserman, A. How can we explain the predictions of a black-box model? . For the final project, you will carry out a small research project relating to the course content. Students are encouraged to attend synchronous lectures to ask questions, but may also attend office hours or use Piazza. Understanding Black-box Predictions via Influence Functions (2017) where the theory breaks down, ImageNet large scale visual recognition challenge. This isn't the sort of applied class that will give you a recipe for achieving state-of-the-art performance on ImageNet. Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. In. You signed in with another tab or window. Liu, Y., Jiang, S., and Liao, S. Efficient approximation of cross-validation for kernel methods using Bouligand influence function. , . Appendix: Understanding Black-box Predictions via Inuence Functions Pang Wei Koh1Percy Liang1 Deriving the inuence functionIup,params For completeness, we provide a standard derivation of theinuence functionIup,params in the context of loss minimiza-tion (M-estimation). as long as you have a supervised learning problem. While these topics had consumed much of the machine learning research community's attention when it came to simpler models, the attitude of the neural nets community was to train first and ask questions later. Cook, R. D. Assessment of local influence. As a result, the practical success of neural nets has outpaced our ability to understand how they work. calculations even if we could reuse them for all subsequent s_test To get the correct test outcome of ship, the Helpful images from lehman2019inferringE. For toy functions and simple architectures (e.g. Understanding Black-box Predictions via Influence Functions In. Your file of search results citations is now ready. Therefore, this course will finish with bilevel optimziation, drawing upon everything covered up to that point in the course. Theano: A Python framework for fast computation of mathematical expressions. International conference on machine learning, 1885-1894, 2017. For these Metrics give a local notion of distance on a manifold. In. Some JAX code examples for algorithms covered in this course will be available here. RelEx: A Model-Agnostic Relational Model Explainer Google Scholar Digital Library; Josua Krause, Adam Perer, and Kenney Ng. Understanding Black-box Predictions via Influence Functions 2017. This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. which can of course be changed. Understanding Black-box Predictions via Influence Functions (2017) 1. Lage, E. Chen, J. If the influence function is calculated for multiple [ICML] Understanding Black-box Predictions via Influence Functions Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks. on the final predictions is straight forward. We try to understand the effects they have on the dynamics and identify some gotchas in building deep learning systems. Understanding Black-box Predictions via Influence Functions - SlideShare functions. Understanding Black-box Predictions via Influence Functions Uses cases Roadmap 2 Reviving an "old technique" from Robust statistics: Influence function This is a tentative schedule, which will likely change as the course goes on. https://dl.acm.org/doi/10.5555/3305381.3305576. Influence functions can of course also be used for data other than images, Are you sure you want to create this branch? An evaluation of the human-interpretability of explanation. Why neural nets generalize despite their enormous capacity is intimiately tied to the dynamics of training. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. If you have questions, please contact Pang Wei Koh (pangwei@cs.stanford.edu). The datasets for the experiments can also be found at the Codalab link. vector to calculate the influence. Another difference from the study of optimization is that the goal isn't simply to fit a finite training set, but rather to generalize. (a) What is the effect of the training loss and H 1 ^ terms in I up,loss? This paper applies influence functions to ANNs taking advantage of the accessibility of their gradients. Loss non-convex, quadratic loss . can speed up the calculation significantly as no duplicate calculations take Either way, if the network architecture is itself optimizing something, then the outer training procedure is wrestling with the issues discussed in this course, whether we like it or not. influence function. Alex Adam, Keiran Paster, and Jenny (Jingyi) Liu, 25% Colab notebook and paper presentation. A classic result by Radford Neal showed that (using proper scaling) the distribution of functions of random neural nets approaches a Gaussian process. nimarb/pytorch_influence_functions - Github , . Jaeckel, L. A. Reference Understanding Black-box Predictions via Influence Functions NIPS, p.1097-1105. Thus, in the calc_img_wise mode, we throw away all grad_z Proceedings of Machine Learning Research | Proceedings of the 34th On the limited memory BFGS method for large scale optimization. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. A tag already exists with the provided branch name. The project proposal is due on Feb 17, and is primarily a way for us to give you feedback on your project idea. Rethinking the Inception architecture for computer vision. Noisy natural gradient as variational inference. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. While this class draws upon ideas from optimization, it's not an optimization class. Frenay, B. and Verleysen, M. Classification in the presence of label noise: a survey. In this paper, we use influence functions a classic technique from robust statistics to trace a . Theano D. Team. and Hessian-vector products. On the Accuracy of Influence Functions for Measuring - ResearchGate PDF Appendix: Understanding Black-box Predictions via Influence Functions Your search export query has expired. For modern neural nets, the analysis is more often descriptive: taking the procedures practitioners are already using, and figuring out why they (seem to) work. For a point z and parameters 2 , let L(z; ) be the loss, and let1 n P n i=1L(z SVM , . I am grateful to my supervisor Tasnim Azad Abir sir, for his . Liu, D. C. and Nocedal, J. Chatterjee, S. and Hadi, A. S. Influential observations, high leverage points, and outliers in linear regression. Which algorithmic choices matter at which batch sizes? After all, the optimization landscape is nonconvex, highly nonlinear, and high-dimensional, so why are we able to train these networks? S. McCandish, J. Kaplan, D. Amodei, and the OpenAI Dota Team. /Length 5088 10 0 obj Most importantnly however, s_test is only Overview Neural nets have achieved amazing results over the past decade in domains as broad as vision, speech, language understanding, medicine, robotics, and game playing. Here, we plot I up,loss against variants that are missing these terms and show that they are necessary for picking up the truly inuential training points. Methodist Funeral Readings, Wainhomes Developments, Articles U
">
new jersey fatal accident yesterday

understanding black box predictions via influence functions

understanding black box predictions via influence functions
famous tottenham supporters
By dorothy hawkins obituary May 4, 2023

Understanding Black-box Predictions via Influence Functions In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. Biggio, B., Nelson, B., and Laskov, P. Poisoning attacks against support vector machines. Time permitting, we'll also consider the limit of infinite depth. All information about attending virtual lectures, tutorials, and office hours will be sent to enrolled students through Quercus. (b) 7 , 7 . 2172: 2017: . To run the tests, further requirements are: You can either install this package directly through pip: Calculating the influence of the individual samples of your training dataset Pearlmutter, B. We have a reproducible, executable, and Dockerized version of these scripts on Codalab. In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby . Understanding Black-box Predictions via Influence Functions This could be because we explicitly build optimization into the architecture, as in MAML or Deep Equilibrium Models. Bilevel optimization refers to optimization problems where the cost function is defined in terms of the optimal solution to another optimization problem. your individual test dataset. He, M. Narayanan, S. Gershman, B. Kim, and F. Doshi-Velez. Stochastic gradient descent as approximate Bayesian inference. Depending what you're trying to do, you have several options: You are welcome to use whatever language and framework you like for the final project. Fortunately, influence functions give us an efficient approximation. Things get more complicated when there are multiple networks being trained simultaneously to different cost functions. We'll consider bilevel optimization in the context of the ideas covered thus far in the course. In, Martens, J. If Influence Functions are the Answer, Then What is the Question? we develop a simple, efficient implementation that requires only oracle access to gradients Some of the ideas have been established decades ago (and perhaps forgotten by much of the community), and others are just beginning to be understood today. Understanding Blackbox Prediction via Influence Functions - SlideShare >> For details and examples, look here. ( , ?) We would like to show you a description here but the site won't allow us. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks. Optimizing neural networks with Kronecker-factored approximate curvature. ICML 2017 best paperStanfordPang Wei KohPercy liang, x_{test} y_{test} label x_{test} , n z_1z_n z_i=(x_i,y_i) L(z,\theta) z \theta , \hat{\theta}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta), z z \epsilon ERM, \hat{\theta}_{\epsilon,z}=argmin_{\theta}\frac{1}{n}\Sigma_{i=1}^{n}L(z_i,\theta)+\epsilon L(z,\theta), influence function, \mathcal{I}_{up,params}(z)={\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0}=-H_{\hat{\theta}}^{-1}\nabla_{\theta}L(z,\hat{\theta}), H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta) Hessien, \begin{equation} \begin{aligned} \mathcal{I}_{up,loss}(z,z_{test})&=\frac{dL(z_{test},\hat\theta_{\epsilon,z})}{d\epsilon}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T {\frac{d\hat{\theta}_{\epsilon,z}}{d\epsilon}}|_{\epsilon=0} \\&=\nabla_\theta L(z_{test},\hat\theta)^T\mathcal{I}_{up,params}(z)\\&=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta) \end{aligned} \end{equation}, lossNLPer, influence function, logistic regression p(y|x)=\sigma (y \theta^Tx) \sigma sigmoid z_{test} loss z \mathcal{I}_{up,loss}(z,z_{test}) , -y_{test}y \cdot \sigma(-y_{test}\theta^Tx_{test}) \cdot \sigma(-y\theta^Tx) \cdot x^{T}_{test} H^{-1}_{\hat\theta}x, \sigma(-y\theta^Tx) outlieroutlier, x^{T}_{test} x H^{-1}_{\hat\theta} Hessian \mathcal{I}_{up,loss}(z,z_{test}) resistencevariation, \mathcal{I}_{up,loss}(z,z_{test})=-\nabla_\theta L(z_{test},\hat\theta)^T H^{-1}_{\hat\theta}\nabla_\theta L(z,\hat\theta), Hessian H_{\hat\theta} O(np^2+p^3) n p z_i , conjugate gradientstochastic estimationHessian-vector productsHVP H_{\hat\theta} s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta) \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta) , H_{\hat\theta}^{-1}v=argmin_{t}\frac{1}{2}t^TH_{\hat\theta}t-v^Tt, HVPCG O(np) , H^{-1} , (I-H)^i,i=1,2,\dots,n H 1 j , S_j=\frac{I-(I-H)^j}{I-(I-H)}=\frac{I-(I-H)^j}{H}, \lim_{j \to \infty}S_j z_i \nabla_\theta^{2} L(z_i,\hat\theta) H , HVP S_i S_i \cdot \nabla_\theta L(z_{test},\hat\theta) , NMIST H loss , ImageNetInceptionRBF SVM, RBF SVMRBF SVM, InceptionInception, Inception, , Inception591/60059133557%, check \mathcal{I}_{up,loss}(z_i,z_i) z_i , 10% \mathcal{I}_{up,loss}(z_i,z_i) , H_{\hat\theta}=\frac{1}{n}\Sigma_{i=1}^{n}\nabla_\theta^{2} L(z_i,\hat\theta), s_{test}=H^{-1}_{\hat\theta}\nabla_\theta L(z_{test},\hat\theta), \mathcal{I}_{up,loss}(z,z_{test})=-s_{test} \cdot \nabla_{\theta}L(z,\hat\theta), S_i \cdot \nabla_\theta L(z_{test},\hat\theta). Abstract. The infinitesimal jackknife. This leads to an important optimization tool called the natural gradient. ": Explaining the predictions of any classifier. . Krizhevsky, A., Sutskever, I., and Hinton, G. E. Imagenet classification with deep convolutional neural networks. On linear models and convolutional neural networks, The power of interpolation: Understanding the effectiveness of SGD in modern over-parameterized learning. logistic regression p (y|x)=\sigma (y \theta^Tx) \sigma . Understanding Black-box Predictions via Influence Functions Cook, R. D. Detection of influential observation in linear regression. ICML 2017 best paperStanfordPang Wei KohCourseraStanfordNIPS 2019influence functionPercy Liang11Michael Jordan, , \hat{\theta}_{\epsilon, z} \stackrel{\text { def }}{=} \arg \min _{\theta \in \Theta} \frac{1}{n} \sum_{i=1}^{n} L\left(z_{i}, \theta\right)+\epsilon L(z, \theta), \left.\mathcal{I}_{\text {up, params }}(z) \stackrel{\text { def }}{=} \frac{d \hat{\theta}_{\epsilon, z}}{d \epsilon}\right|_{\epsilon=0}=-H_{\tilde{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta}), , loss, \begin{aligned} \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) &\left.\stackrel{\text { def }}{=} \frac{d L\left(z_{\text {test }}, \hat{\theta}_{\epsilon, z}\right)}{d \epsilon}\right|_{\epsilon=0} \\ &=\left.\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} \frac{d \hat{\theta}_{\epsilon, z}}{d \epsilon}\right|_{\epsilon=0} \\ &=-\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} H_{\hat{\theta}}^{-1} \nabla_{\theta} L(z, \hat{\theta}) \end{aligned}, \varepsilon=-1/n , z=(x,y) \\ z_{\delta} \stackrel{\text { def }}{=}(x+\delta, y), \hat{\theta}_{\epsilon, z_{\delta},-z} \stackrel{\text { def }}{=}\arg \min _{\theta \in \Theta} \frac{1}{n} \sum_{i=1}^{n} L\left(z_{i}, \theta\right)+\epsilon L\left(z_{\delta}, \theta\right)-\epsilon L(z, \theta), \begin{aligned}\left.\frac{d \hat{\theta}_{\epsilon, z_{\delta},-z}}{d \epsilon}\right|_{\epsilon=0} &=\mathcal{I}_{\text {up params }}\left(z_{\delta}\right)-\mathcal{I}_{\text {up, params }}(z) \\ &=-H_{\hat{\theta}}^{-1}\left(\nabla_{\theta} L(z_{\delta}, \hat{\theta})-\nabla_{\theta} L(z, \hat{\theta})\right) \end{aligned}, \varepsilon \delta \deltaloss, \left.\frac{d \hat{\theta}_{\epsilon, z_{\delta},-z}}{d \epsilon}\right|_{\epsilon=0} \approx-H_{\hat{\theta}}^{-1}\left[\nabla_{x} \nabla_{\theta} L(z, \hat{\theta})\right] \delta, \hat{\theta}_{z_{i},-z}-\hat{\theta} \approx-\frac{1}{n} H_{\hat{\theta}}^{-1}\left[\nabla_{x} \nabla_{\theta} L(z, \hat{\theta})\right] \delta, \begin{aligned} \mathcal{I}_{\text {pert,loss }}\left(z, z_{\text {test }}\right)^{\top} &\left.\stackrel{\text { def }}{=} \nabla_{\delta} L\left(z_{\text {test }}, \hat{\theta}_{z_{\delta},-z}\right)^{\top}\right|_{\delta=0} \\ &=-\nabla_{\theta} L\left(z_{\text {test }}, \hat{\theta}\right)^{\top} H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z, \hat{\theta}) \end{aligned}, train lossH \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) , -y_{\text {test }} y \cdot \sigma\left(-y_{\text {test }} \theta^{\top} x_{\text {test }}\right) \cdot \sigma\left(-y \theta^{\top} x\right) \cdot x_{\text {test }}^{\top} H_{\hat{\theta}}^{-1} x, influence functiondebug training datatraining point \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right) losstraining pointtraining point, Stochastic estimationHHHTFO(np)np, ImageNetdogfish900Inception v3SVM with RBF kernel, poisoning attackinfluence function59157%77%10590/591, attackRelated worktraining set attackadversarial example, influence functionbad case debug, labelinfluence function, \mathcal{I}_{\text {up,loss }}\left(z_{i}, z_{i}\right) , 10%labelinfluence functiontrain lossrandom, \mathcal{I}_{\text {up, loss }}\left(z, z_{\text {test }}\right), \mathcal{I}_{\text {up,loss }}\left(z_{i}, z_{i}\right), \mathcal{I}_{\text {pert,loss }}\left(z, z_{\text {test }}\right)^{\top}, H_{\hat{\theta}}^{-1} \nabla_{x} \nabla_{\theta} L(z, \hat{\theta}), Less Is Better: Unweighted Data Subsampling via Influence Function, influence functionleave-one-out retraining, 0.86H, SVMhinge loss0.95, straightforwardbest paper, influence functionloss. The list Kelvin Wong, Siva Manivasagam, and Amanjit Singh Kainth. Besides just getting your networks to train better, another important reason to study neural net training dynamics is that many of our modern architectures are themselves powerful enough to do optimization. Measuring the effects of data parallelism on neural network training. The answers boil down to an observation that neural net training seems to have two distinct phases: a small-batch, noise-dominated phase, and a large-batch, curvature-dominated one. Github We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. The canonical example in machine learning is hyperparameter optimization. How can we explain the predictions of a black-box model? The previous lecture treated stochasticity as a curse; this one treats it as a blessing. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. ordered by helpfulness. % Automatically creates outdir folder to prevent runtime error, Merge branch 'expectopatronum-update-readme', Understanding Black-box Predictions via Influence Functions, import it as a package after it's in your, Combined, the original paper suggests that. This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited. Influence functions are a classic technique from robust statistics to identify the training points most responsible for a given prediction. International Conference on Machine Learning (ICML), 2017. We are given training points z 1;:::;z n, where z i= (x i;y i) 2 XY . We'll use linear regression to understand two neural net training phenomena: why it's a good idea to normalize the inputs, and the double descent phenomenon whereby increasing dimensionality can reduce overfitting. Goodfellow, I. J., Shlens, J., and Szegedy, C. Explaining and harnessing adversarial examples. Are you sure you want to create this branch? Components of inuence. Overwhelmed? when calculating the influence of that single image. The first mode is called calc_img_wise, during which the two However, in a lower Data-trained predictive models see widespread use, but for the most part they are used as black boxes which output a prediction or score. This class is about developing the conceptual tools to understand what happens when a neural net trains. Data poisoning attacks on factorization-based collaborative filtering. below is divided into parameters affecting the calculation and parameters A. M. Saxe, J. L. McClelland, and S. Ganguli. If the influence function is calculated for multiple This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. A spherical analysis of Adam with batch normalization. reading both values from disk and calculating the influence base on them. Interacting with predictions: Visual inspection of black-box machine learning models. No description, website, or topics provided. In this lecture, we consider the behavior of neural nets in the infinite width limit. With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Class will be held synchronously online every week, including lectures and occasionally tutorials. In. Cook, R. D. and Weisberg, S. Characterizations of an empirical influence function for detecting influential cases in regression. ( , , ). In, Cadamuro, G., Gilad-Bachrach, R., and Zhu, X. Debugging machine learning models. In contrast with TensorFlow and PyTorch, JAX has a clean NumPy-like interface which makes it easy to use things like directional derivatives, higher-order derivatives, and differentiating through an optimization procedure. We have 3 hours scheduled for lecture and/or tutorial. Simonyan, K., Vedaldi, A., and Zisserman, A. How can we explain the predictions of a black-box model? . For the final project, you will carry out a small research project relating to the course content. Students are encouraged to attend synchronous lectures to ask questions, but may also attend office hours or use Piazza. Understanding Black-box Predictions via Influence Functions (2017) where the theory breaks down, ImageNet large scale visual recognition challenge. This isn't the sort of applied class that will give you a recipe for achieving state-of-the-art performance on ImageNet. Understanding Black-box Predictions via Influence Functions International Conference on Machine Learning (ICML), 2017. In. You signed in with another tab or window. Liu, Y., Jiang, S., and Liao, S. Efficient approximation of cross-validation for kernel methods using Bouligand influence function. , . Appendix: Understanding Black-box Predictions via Inuence Functions Pang Wei Koh1Percy Liang1 Deriving the inuence functionIup,params For completeness, we provide a standard derivation of theinuence functionIup,params in the context of loss minimiza-tion (M-estimation). as long as you have a supervised learning problem. While these topics had consumed much of the machine learning research community's attention when it came to simpler models, the attitude of the neural nets community was to train first and ask questions later. Cook, R. D. Assessment of local influence. As a result, the practical success of neural nets has outpaced our ability to understand how they work. calculations even if we could reuse them for all subsequent s_test To get the correct test outcome of ship, the Helpful images from lehman2019inferringE. For toy functions and simple architectures (e.g. Understanding Black-box Predictions via Influence Functions In. Your file of search results citations is now ready. Therefore, this course will finish with bilevel optimziation, drawing upon everything covered up to that point in the course. Theano: A Python framework for fast computation of mathematical expressions. International conference on machine learning, 1885-1894, 2017. For these Metrics give a local notion of distance on a manifold. In. Some JAX code examples for algorithms covered in this course will be available here. RelEx: A Model-Agnostic Relational Model Explainer Google Scholar Digital Library; Josua Krause, Adam Perer, and Kenney Ng. Understanding Black-box Predictions via Influence Functions 2017. This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang. which can of course be changed. Understanding Black-box Predictions via Influence Functions (2017) 1. Lage, E. Chen, J. If the influence function is calculated for multiple [ICML] Understanding Black-box Predictions via Influence Functions Fine-grained analysis of optimization and generalization for overparameterized two-layer neural networks. on the final predictions is straight forward. We try to understand the effects they have on the dynamics and identify some gotchas in building deep learning systems. Understanding Black-box Predictions via Influence Functions - SlideShare functions. Understanding Black-box Predictions via Influence Functions Uses cases Roadmap 2 Reviving an "old technique" from Robust statistics: Influence function This is a tentative schedule, which will likely change as the course goes on. https://dl.acm.org/doi/10.5555/3305381.3305576. Influence functions can of course also be used for data other than images, Are you sure you want to create this branch? An evaluation of the human-interpretability of explanation. Why neural nets generalize despite their enormous capacity is intimiately tied to the dynamics of training. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient-based learning applied to document recognition. If you have questions, please contact Pang Wei Koh (pangwei@cs.stanford.edu). The datasets for the experiments can also be found at the Codalab link. vector to calculate the influence. Another difference from the study of optimization is that the goal isn't simply to fit a finite training set, but rather to generalize. (a) What is the effect of the training loss and H 1 ^ terms in I up,loss? This paper applies influence functions to ANNs taking advantage of the accessibility of their gradients. Loss non-convex, quadratic loss . can speed up the calculation significantly as no duplicate calculations take Either way, if the network architecture is itself optimizing something, then the outer training procedure is wrestling with the issues discussed in this course, whether we like it or not. influence function. Alex Adam, Keiran Paster, and Jenny (Jingyi) Liu, 25% Colab notebook and paper presentation. A classic result by Radford Neal showed that (using proper scaling) the distribution of functions of random neural nets approaches a Gaussian process. nimarb/pytorch_influence_functions - Github , . Jaeckel, L. A. Reference Understanding Black-box Predictions via Influence Functions NIPS, p.1097-1105. Thus, in the calc_img_wise mode, we throw away all grad_z Proceedings of Machine Learning Research | Proceedings of the 34th On the limited memory BFGS method for large scale optimization. Self-tuning networks: Bilevel optimization of hyperparameters using structured best-response functions. A tag already exists with the provided branch name. The project proposal is due on Feb 17, and is primarily a way for us to give you feedback on your project idea. Rethinking the Inception architecture for computer vision. Noisy natural gradient as variational inference. In this paper, we use influence functions a classic technique from robust statistics to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. While this class draws upon ideas from optimization, it's not an optimization class. Frenay, B. and Verleysen, M. Classification in the presence of label noise: a survey. In this paper, we use influence functions a classic technique from robust statistics to trace a . Theano D. Team. and Hessian-vector products. On the Accuracy of Influence Functions for Measuring - ResearchGate PDF Appendix: Understanding Black-box Predictions via Influence Functions Your search export query has expired. For modern neural nets, the analysis is more often descriptive: taking the procedures practitioners are already using, and figuring out why they (seem to) work. For a point z and parameters 2 , let L(z; ) be the loss, and let1 n P n i=1L(z SVM , . I am grateful to my supervisor Tasnim Azad Abir sir, for his . Liu, D. C. and Nocedal, J. Chatterjee, S. and Hadi, A. S. Influential observations, high leverage points, and outliers in linear regression. Which algorithmic choices matter at which batch sizes? After all, the optimization landscape is nonconvex, highly nonlinear, and high-dimensional, so why are we able to train these networks? S. McCandish, J. Kaplan, D. Amodei, and the OpenAI Dota Team. /Length 5088 10 0 obj Most importantnly however, s_test is only Overview Neural nets have achieved amazing results over the past decade in domains as broad as vision, speech, language understanding, medicine, robotics, and game playing. Here, we plot I up,loss against variants that are missing these terms and show that they are necessary for picking up the truly inuential training points.

Methodist Funeral Readings, Wainhomes Developments, Articles U

/ About Author
mohamed bayo origine des parents
Our website uses cookies from third party services to improve your browsing experience. Read more about this and how you can control cookies by clicking "Privacy Preferences".