Local Learning

By Chris Atkeson
cga@cc.gatech.edu
College of Computing
Georgia Institute of Technology

What is Local Learning?

• "Local methods assign a weight to each training observation that regulates its influence on the training process. This weight depends upon the location of the training point in the input variable space relative to that of the point to be predicted. Training observations closer to the prediction point generally receive higher weights."

  from: Jerome H. Friedman, abstract for talk on "Intelligent Local Learning For Prediction in High Dimensions", International Conference on Artificial Neural Networks (ICANN 95), October 9-13, 1995, Paris, France.

Reviews, Overviews, and Surveys

An overview of work on local learning algorithms is given by:
Atkeson, C. G., Moore, A. W., & Schaal, S. (submitted). Locally Weighted Learning. Artificial Intelligence Review.

An overview of local learning applied to robots is given by:
Atkeson, C. G., Moore, A. W., & Schaal, S. (submitted). Locally Weighted Learning for Control. Artificial Intelligence Review.

Overviews of local regression are given in:
Cleveland, W. S. and C. Loader. Smoothing by Local Regression: Principles and Methods.
and
Fan, J. Local Modeling

A book is available:
Jianqing Fan and Irene Gijbels
Local Polynomial Modeling and its Applications
Chapman and Hall, London, 1996.

Software for local regression is available:

LOCFIT

Recent ATT/Bell Labs work.

LOESS

Older ATT/Bell Labs work. Also available from ftp://ftp.netlib.org/a/loess

LOWESS

Very old ATT/Bell Labs work.

AUTON

Andrew Moore's work at CMU.

RFWR

Stefan Schaal's work at ATR, GT, and MIT.

Biostat

Local polynomial regression fitting with Epanechnikov weights or ridging, and MATLAB Smoothing Toolbox from the Department of Biostatistics at unizh.ch.

NoLoEss

Locally parametric regression estimation: DOS program by Andrzej S. Kozek.

Papers

• Locally Weighted Learning
  Atkeson, C. G., Moore, A. W., & Schaal, S.
• Locally Weighted Learning for Control
  Atkeson, C. G., Moore, A. W., & Schaal, S.
• Smoothing by Local Regression: Principles and Methods
  Cleveland, W. S. and C. Loader.
• Function Approximation with Neural Networks and Local Methods: Bias, Variance, and Smoothness
  Steve Lawrence, Ah Chung Tsoi, and Andrew D. Back. (also at http://www.elec.uq.edu.au/~lawrence)
• A New Nonparmetric Estimation Method: Local and Nonlinear
  Andrzej S. Kozek.
• Fast Computation of Auxiliary Quantities in Local Polynomial Regression
  B. A. Turlach and M. P. Wand.
• Variance properties of local polynomials
  Burkhardt Seifert and Theo Gasser, Nov. 1994.
• Finite sample variance of local polynomials
  Burkhardt Seifert and Theo Gasser, May 1994.
• Flexible Metric Nearest Neighbor Classification
  Jerome H. Friedman, Technical Report (Nov. 1994).
• Discriminant Adaptive Nearest Neighbor Classification
  Hastie, T. J. and Tibshirani, R., Technical Report (Dec. 1994).
• Handwritten Digit Recognition via Deformable Prototypes
  Hastie, T. J. and Tibshirani, R., AT&T Bell Laboratories Technical Report 1994.
• Learning Prototype Models for Tangent Distance
  NIPS proceedings, 1994.
• Is Learning the n-th Thing Any Easier Than Learning The First?
  Sebastian Thrun. Application of memory-based learning to reinforcement learning. To appear in: Advances in Neural Information Processing Systems (NIPS) 8.
• Discovering Structure in Multiple Learning Tasks: The TC Algorithm
  Sebastian Thrun and Joseph O'Sullivan. Application of memory-based learning to reinforcement learning. To appear in: International Conference on Machine Learning 1996.
• Scaling up Average Reward Reinforcement Learning by Approximating the Domain Models and the Value Function
  Tadepalli, P. and Ok, D. Submitted to ICML-96. Application of local linear regression to reinforcement learning.

People and Places

• Chris Atkeson cga@cc.gatech.edu
• Andrew Back back@elec.uq.edu.au
• Leon Bottou leon@sun8.lri.fr leon@bop.neuristique.fr
• William Cleveland wsc@bell-labs.com
• Jianqing Fan jfan@stat.unc.edu
• Valery V. Fedorov Department of Applied Statistics, U Minn.
• Jerome H. Friedman jhf@playfair.stanford.edu
• Theo Gasser
• Eric Grosse ehg@bell-labs.com
• Peter Hackl Dept. of Stat. U. Ec and Bus. Adm., Vienna.
• Trevor Hastie trevor@playfair.stanford.edu
• Aram Karalic aram.karalic@ijs.yu
• Andrzej Stefan Kozek akozek@zen.efs.mq.edu.au
• Steve Lawrence lawrence@elec.uq.edu.au (also http://www.elec.uq.edu.au/~lawrence)
• Clive Loader clive@research.att.com
• Andrew Moore awm@cs.cmu.edu
• Hans-Georg Muller
• Werner G. Muller Dept. of Stat. U. Ec and Bus. Adm., Vienna.
• DoKyeong OK okd@research.cs.orst.edu
• Stefan Schaal sschaal@cc.gatech.edu
• Burkhardt Seifert seifert@ifspm.unizh.ch
• Patrice Simard
• Patrick van der Smagt Patrick.van.der.Smagt@dlr.de
• Prasad Tadepalli tadepall@research.cs.orst.edu
• Sebastion Thrun thrun@cs.cmu.edu
• Ah Chung Tsoi act@elec.uq.edu.au
• Vladimir Vapnik vlad@research.att.com
• Zlatko Zografski zz@gea.etf.ukim.edu.mk

Web Stuff

• Chemometrics from A to Z (L): Locally Weighted Regression (LWR)
• Lab Notes: Loess Smoothing by Dick Mitchell

Search Keywords

• local* + learn*
• local* + regression
• local* + weight*
• local + function approximation
• local* + model*
• LWR
• LOCFIT
• LOESS
• LOWESS
• memory + learn*
• instance + learn*
• exemplar + learn*
• lazy + learn*
• least commitment learn*

If you have any comments or hotlinks to add about projects related to what we do, please let cga@cc.gatech.edu know.

Back to Chris Atkeson's home page