# Papers Using ASA

## https://www.ingber.com/asa_papers.txt

asa_papers.txt
[To-Top-of-Archive]

This file is an addendum to the NOTES file in the ASA code, containing references to some difficult problems optimized using ASA or its precursor VFSR.

ASA Code Reference:
%A L. Ingber
%R Global optimization C-code
%I Caltech Alumni Association
%D 1993
%O URL http://alumni.caltech.edu/~ingber/#ASA-CODE

### Patents

Some examples of patents issued using ASA can be found from patent searches on "adaptive simulated annealing" on https://www.google.com/patents, http://www.freepatentsonline.com, and http://www.sumobrain.com :

A search on the use in patents of its precursor VFSR can be found from patent searches on "very fast simulated re-annealing"
and "very fast simulated reannealing"
http://www.freepatentsonline.com/result.html?query_txt=%22very%20fast%20simulated%20reannealing%22

Some additional use in patents can be found from patent searches on "lester ingber"
http://www.freepatentsonline.com/result.html?query_txt=%22lester%20ingber%22
http://www.sumobrain.com/result.html?p=1&from_ss=&srch_id=&search_name=&srch=xprtsrch&query_txt=%22Lester+Ingber%22&uspat=on&date_range=all&stemming=on&sort=chron&search=Search

%A L. Ingber
%T Statistical mechanics of neocortical interactions: Large-scale EEG influences on molecular processes
%J Journal of Theoretical Biology
%V 395
%P 144-152
%D 2016
%O URL https://www.ingber.com/smni16_large-scale_molecular.pdf

Graphs for data used in this project are in smni16_large-scale_molecular_EEGgraphs.pdf

Calculations further support the premise that large-scale synchronous firings of neurons may affect molecular processes. The context is scalp electroencephalography (EEG) during short-term memory (STM) tasks. The mechanism considered is $\mathbf{\Pi} = \mathbf{p} + q \mathbf{A}$ (SI units) coupling, where $\mathbf{p}$ is the momenta of free $\mathrm{Ca}^{2+}$ waves $q$ the charge of $\mathrm{Ca}^{2+}$ in units of the electron charge, and $\mathbf{A}$ the magnetic vector potential of current $\mathbf{I}$ from neuronal minicolumnar firings considered as wires, giving rise to EEG. Data has processed using multiple graphs to identify sections of data to which spline-Laplacian transformations are applied, to fit the statistical mechanics of neocortical interactions (SMNI) model to EEG data, sensitive to synaptic interactions subject to modification by $\mathrm{Ca}^{2+}$ waves.

%A L. Ingber
%T Calculating consciousness correlates at multiple scales of neocortical interactions
%B Horizons in Neuroscience Research
%I Nova
%C Hauppauge, NY
%D 2015
%O Invited paper. URL https://www.ingber.com/smni15_calc_conscious.pdf

A lot of what we consider "mind" is conscious attention to short-term memories (STM). At least some STM memories are actively processed by highly synchronized patterns of neuronal firings, with enough synchrony to be able to be easily measured by scalp electroencephalographic recordings (EEG). Large-scale synchronous macrocolumnar EEG firings is a top-down process developed by a statistical mechanics of neocortical interactions (SMNI), depending on the associated magnetic vector potential $\mathbf{A}$. Molecular-scale $\mathrm{Ca}^{2+}$ waves are the affected bottom-up process that influence neuronal firings, depending on the wave momentum $\mathbf{p}$. $\mathbf{A}$ directly influences $\mathbf{p}$ via the canonical momentum $\mathbf{\Pi} = \mathbf{p} + q \mathbf{A}$ (SI units), where the charge of $\mathrm{Ca}^{2+}$ is $q = - 2 e$, $e$ is the magnitude of the charge of an electron. Calculations in both classical and quantum mechanics are consistent with this effect. This approach also suggests some nanosystem-pharmaceutical applications. Results give strong confirmation of the SMNI model of STM, but only weak statistical consistency of $\mathbf{\Pi} = \mathbf{p} + q \mathbf{A}$ influences on scalp EEG.

%A H. Aguiar, Jr.
%A A. Petraglia
%T Dimensional reduction in constrained global optimization on smooth manifolds
%J Information Sciences
%V 299
%N 1
%D 2015
%P 243-261
%O URL http://dx.doi.org/10.1016/j.ins.2014.12.032

This work introduces an approach aimed at reducing the dimension of search domains in constrained global optimization of real valued functions defined on smooth manifolds, and subject (also and mainly) to equality constraints. The functions expressing the cited constraints must satisfy certain smoothness conditions, and other types of restrictions are simultaneously possible, but the effect of dimensional reduction will be proportional to the number of equality constraints. The objective functions under study do not need to be differentiable or even continuous, and it is shown that the optimization task will be executed so that candidate points remain in the corresponding submanifolds, evolving there during the whole optimization process. The proposed paradigm may be employed jointly with an extensive family of already established metaheuristics. After introducing the fundamental ideas and establishing the theoretical basis, some examples will illustrate the effectiveness of the proposed method.

%A L. Ingber
%A M. Pappalepore
%A R.R. Stesiak
%J Journal of Theoretical Biology
%T Electroencephalographic field influence on calcium momentum waves
%V 343
%P 138-153
%D 2014
%O URL https://www.ingber.com/smni14_eeg_ca.pdf and http://dx.doi.org/10.1016/j.jtbi.2013.11.002

This paper uses ASA to fit statistical mechanics of neocortical interactions (SMNI) to EEG data.

Macroscopic electroencephalographic (EEG) fields can be an explicit top-down neocortical mechanism that directly drives bottom-up processes that describe memory, attention, and other neuronal processes. The top-down mechanism considered are macrocolumnar EEG firings in neocortex, as described by a statistical mechanics of neocortical interactions (SMNI), developed as a magnetic vector potential $\mathbf{A}$. The bottom-up process considered are $\mathrm{Ca}^{2+}$ waves prominent in synaptic and extracellular processes that are considered to greatly influence neuronal firings. Here, the complimentary effects are considered, i.e., the influence of $\mathbf{A}$ on $\mathrm{Ca}^{2+}$ momentum, $\mathbf{p}$. The canonical momentum of a charged particle in an electromagnetic field, $\mathbf{\Pi} = \mathbf{p} + q \mathbf{A}$ (SI units), is calculated, where the charge of $\mathrm{Ca}^{2+}$ is $q = - 2 e$, $e$ is the magnitude of the charge of an electron. Calculations demonstrate that macroscopic EEG $\mathbf{A}$ can be quite influential on the momentum $\mathbf{p}$ of $\mathrm{Ca}^{2+}$ ions, in both classical and quantum mechanics. Molecular scales of $\mathrm{Ca}^{2+}$ wave dynamics are coupled with $\mathbf{A}$ fields developed at macroscopic regional scales measured by coherent neuronal firing activity measured by scalp EEG. The project has three main aspects: fitting $\mathbf{A}$ models to EEG data as reported here, building tripartite models to develop $\mathbf{A}$ models, and studying long coherence times of $\mathrm{Ca}^{2+}$ waves in the presence of $\mathbf{A}$ due to coherent neuronal firings measured by scalp EEG. The SMNI model supports a mechanism wherein the $\mathbf{p} + q \mathbf{A}$ interaction at tripartite synapses, via a dynamic centering mechanism (DCM) to control background synaptic activity, acts to maintain short-term memory (STM) during states of selective attention.

%A L. Ingber
%T Stochastic global optimization and its applications with fuzzy adaptive simulated annealing
%E H.A. Oliveira, Jr.
%E A. Petraglia
%E L. Ingber
%E M.R. Petraglia
%I Springer
%C New York
%D 2012
%O Invited Paper. URL https://www.ingber.com/asa11_options.pdf

This paper includes contributions on applications from the other Editors of this book.

%A L. Ingber
%T Statistical mechanics of neocortical interactions: Nonlinear columnar electroencephalography
%J NeuroQuantology Journal
%V 7
%N 4
%P 500-529
%D 2009
%O URL https://www.ingber.com/smni09_nonlin_column_eeg.pdf and http://www.neuroquantology.com/journal/index.php/nq/article/view/365/385

Columnar firings of neocortex, modeled by a statistical mechanics of neocortical interactions (SMNI), are investigated for conditions of oscillatory processing at frequencies consistent with observed electroencephalography (EEG). A strong inference is drawn that physiological states of columnar activity receptive to selective attention support oscillatory processing in observed frequency ranges. Direct calculations of the Euler-Lagrange (EL) equations which are derived from functional variation of the SMNI probability distribution, giving most likely states of the system, are performed for three prototypical Cases, dominate excitatory columnar firings, dominate inhibitory columnar firings, and in-between balanced columnar firings, with and without a Centering mechanism (CM) (based on observed changes in stochastic background of presynaptic interactions) which pulls more stable states into the physical firings ranges. Only states with the CM exhibit robust support for these oscillatory states. These calculations are repeated for the visual neocortex, which has twice as many neurons/minicolumn as other neocortical regions. These calculations argue that robust columnar support for common EEG activity requires the same columnar presynaptic parameter necessary for ideal short-term memory (STM). It is demonstrated at this columnar scale, that both shifts in local columnar presynaptic background as well as local or global regional oscillatory interactions can effect or be affected by attractors that have detailed experimental support to be considered states of STM. Including the CM with other proposed mechanisms for columnar-glial interactions and for glial-presynaptic background interactions, a path for future investigations is outlined to test for quantum interactions, enhanced by magnetic fields from columnar EEG, that directly support cerebral STM and computation by controlling presynaptic noise. This interplay can provide mechanisms for information processing and computation in mammalian neocortex.

This paper demonstrates by explicit calculations that short-term memory (STM) and EEG can indeed be correlated. At least according to some reviewers, this seems not to have been demonstrated previously. This paper shows that the previous SMNI models which calculate many features measured as STM also support EEG at columnar scales. To put this into some perspective, many neuroscientists believe that global regional activity supports EEG wave-like oscillatory observations, by solving wave equations with hemisphere boundary conditions with spherical eigenfunctions that detail the frequencies of EEG. In this columnar study, wave-type equations are derived via nonlinear EL equations from SMNI probability distributions, and these are explicitly numerically solved to demonstrate that observed EEG frequencies are supported under the same SMNI conditions that support STM.

%A L. Ingber
%T Ideas by statistical mechanics (ISM)
%R Report 2006:ISM
%I Lester Ingber Research
%D 2006
%O URL https://www.ingber.com/smni06_ism.pdf

A short version appears as "AI and Ideas by Statistical Mechanics (ISM)" in Encyclopedia of Artificial Intelligence, pp. 58-64 (2008), and details in this paper appear in "Ideas by Statistical Mechanics (ISM)", Journal of Integrated Systems Design and Process Science, Vol. 11, No. 3, pp. 22-43 (2007), Special Issue: Biologically Inspired Computing.

Links from smni06_ism.pdf to asa06_ism.pdf, combat06_ism.pdf, markets06_ism.pdf, and path06_ism.pdf.

Ideas by Statistical Mechanics (ISM) is a generic program to model evolution and propagation of ideas/patterns throughout populations subjected to endogenous and exogenous interactions. The program is based on the author's work in Statistical Mechanics of Neocortical Interactions (SMNI), and uses the author's Adaptive Simulated Annealing (ASA) code for optimizations of training sets, as well as for importance-sampling to apply the author's copula financial risk-management codes, Trading in Risk Dimensions (TRD), for assessments of risk and uncertainty. This product can be used for decision support for projects ranging from diplomatic, information, military, and economic (DIME) factors of propagation/evolution of ideas, to commercial sales, trading indicators across sectors of financial markets, advertising and political campaigns, etc.

It seems appropriate to base an approach for propagation of ideas on the only system so far demonstrated to develop and nurture ideas, i.e., the neocortical brain. A statistical mechanical model of neocortical interactions, developed by the author and tested successfully in describing short-term memory and EEG indicators, is the proposed model. ISM develops subsets of macrocolumnar activity of multivariate stochastic descriptions of defined populations, with macrocolumns defined by their local parameters within specific regions and with parameterized endogenous inter-regional and exogenous external connectivities. Parameters with a given subset of macrocolumns will be fit using ASA to patterns representing ideas. Parameters of external and inter-regional interactions will be determined that promote or inhibit the spread of these ideas. Tools of financial risk management, developed by the author to process correlated multivariate systems with differing non-Gaussian distributions using modern copula analysis, importance-sampled using ASA, will enable bona fide correlations and uncertainties of success and failure to be calculated. Marginal distributions will be evolved to determine their expected duration and stability using algorithms developed by the author, i.e., PATHTREE and PATHINT codes.

%A L. Ingber
%T Statistical mechanics of neocortical interactions: Portfolio of physiological indicators
%R Report 2006:PPI
%I Lester Ingber Research
%D 2006
%O URL https://www.ingber.com/smni06_ppi.pdf

There are several kinds of non-invasive imaging methods that are used to collect data from the brain, e.g., EEG, MEG, PET, SPECT, fMRI, etc. It is difficult to get resolution of information processing using any one of these methods. Approaches to integrate data sources may help to get better resolution of data and better correlations to behavioral phenomena ranging from attention to diagnoses of disease. The approach taken here is to use algorithms developed for the author's Trading in Risk Dimensions (TRD) code using modern methods of copula portfolio risk management, with joint probability distributions derived from the author's model of statistical mechanics of neocortical interactions (SMNI). The author's Adaptive Simulated Annealing (ASA) code is for optimizations of training sets, as well as for importance-sampling. Marginal distributions will be evolved to determine their expected duration and stability using algorithms developed by the author, i.e., PATHTREE and PATHINT codes.

%A L. Ingber
%T Trading in Risk Dimensions (TRD)
%R Report 2005:TRD
%D 2005
%I Lester Ingber Research
%C Ashland, OR
%O URL https://www.ingber.com/markets05_trd.pdf

An updated shorter paper with this title is in the Handbook of Trading: Strategies for Navigating and Profiting from Currency, Bond, and Stock Markets (McGraw-Hill, 2010).

Previous work, mostly published, developed two-shell recursive trading systems. An inner-shell of Canonical Momenta Indicators (CMI) is adaptively fit to incoming market data. A parameterized trading-rule outer-shell uses the global optimization code Adaptive Simulated Annealing (ASA) to fit the trading system to historical data. A simple fitting algorithm, usually not requiring ASA, is used for the inner-shell fit. An additional risk-management middle-shell has been added to create a three-shell recursive optimization/sampling/fitting algorithm. Portfolio-level distributions of copula-transformed multivariate distributions (with constituent markets possessing different marginal distributions in returns space) are generated by Monte Carlo samplings. ASA is used to importance-sample weightings of these markets.

The core code, Trading in Risk Dimensions (TRD), processes Training and Testing trading systems on historical data, and consistently interacts with RealTime trading platforms at minute resolutions, but this scale can be modified. This approach transforms constituent probability distributions into a common space where it makes sense to develop correlations to further develop probability distributions and risk/uncertainty analyses of the full portfolio. ASA is used for importance-sampling these distributions and for optimizing system parameters.

%A J. Zhou
%A G. Livingston
%A G. Grinstein
%T Automatic Parameter Selection for Sequence Similarity Search
%R Dept. Computer Science Preprint
%I U Massachusetts
%C Lowell, MA
%D 2003
%O URL http://www.cs.uml.edu/~jzhou

This study used ASA to select parameters for sequence similar region search in their autonomous AutoSimS model of general pairwise sequence similarity analysis. Their artificial intelligence approach involves more advanced features than commonly used BLAST and FASTA tools. Contact Jianping Zhou <jzhou@cs.uml.edu> for more information.

%A H.A. Oliveira Jr.
%T Fuzzy control of stochastic global optimization algorithms and very fast simulated reannealing
%I hime@engineer.com
%C Rio de Janeiro, Brazil
%D 2001
%O URL http://www.optimization-online.org/DB_FILE/2003/11/779.pdf

This paper provides a supplementary algorithm for adaptive quenching to make ASA more efficient for many systems.

%A Hime Aguiar e Oliveira Jr.
%T Fuzzy Modeling with Adaptive Simulated Annealing
%C Rio de Janeiro, Brazil
%I hime@engineer.com
%D 2004
%O URL http://www.optimization-online.org/DB_HTML/2004/07/901.html

Abstract: A new method for data-based fuzzy system modeling is presented. The approach uses Takagi-Sugeno models and Adaptive Simulated Annealing (ASA) to achieve its goal . The problem to solve is well defined - given a training set containing a finite number of input-output pairs, construct a fuzzy system that approximates the behavior of the real system that originated that set , within a pre-established precision .

%A H.A. Oliveira Jr.
%A H.R. Petraglia.
%A A. Petraglia
%T Frequency domain FIR filter design using fuzzy adaptive simulated annealing
%B 7th International Symposium on Signal Processing and Information Technology, 2007, vol. 1
%I Proceedings of ISSPIT
%C Cairo
%D 2007
%P 899-903
%A H.A Oliveira Jr.
%A A. Petraglia
%A M.R. Petraglia
%T Design of FIR quadrature mirror-image filter banks using fuzzy adaptive simulated annealing
%B Proceedings of the IEEE 13th DSP Workshop, 2009. vol. 1
%I IEEE 13th DSP Workshop, 2009
%C Marco Island, FL
%D 2009
%P 485-489
%A H.A. Oliveira, Jr.
%A A. Petraglia
%A M.R. Petraglia
%T Frequency domain FIR filter design using fuzzy adaptive simulated annealing
%J Circuits, Systems, and Signal Processiing
%v 28
%N 6
%D 2009
%P 899-911
%O DOI: 10.1007/s00034-009-9128-1

Abstract An alternative approach to digital filter design is presented. The overall technique is as follows: Starting from frequency domain constraints and a parameterized expression of the filter family under adaptation, a corresponding training set is created, an error function is synthesized and a global minimization process is executed. At the end, the point that minimizes globally the particular cost function at hand determines the optimal filter. The adopted numerical optimization algorithm is based upon the well-known simulated annealing paradigm and its implementation is known as fuzzy adaptive simulated annealing. Although it is used in this paper to fit FIR filters to frequency domain specifications, the method is suitable to application in other problems of digital filter design, where the matter under study can be stated as finding the global minimum of a numerical function of filter parameters. Design examples are shown to verify the effectiveness of the proposed approach.

%A L. Ingber
%T Adaptive Simulated Annealing (ASA) and Path-Integral (PATHINT) Algorithms: Generic Tools for Complex Systems
%R ASA-PATHINT Lecture Plates
%I Lester Ingber Research
%C Chicago, IL
%D 2001
%O URL https://www.ingber.com/asa01_lecture.pdf

%A L. Ingber
%A R.P. Mondescu
%T Optimization of Trading Physics Models of Markets
%D 2001
%V 12
%N 4
%P 776-790
%J IEEE Trans. Neural Networks
%O Invited paper for special issue on Neural Networks in Financial Engineering. URL markets01_optim_trading.pdf

ABSTRACT: We describe an end-to-end real-time S&P futures trading system. Inner-shell stochastic nonlinear dynamic models are developed, and Canonical Momenta Indicators (CMI) are derived from a fitted Lagrangian used by outer-shell trading models dependent on these indicators. Recursive and adaptive optimization using Adaptive Simulated Annealing (ASA) is used for fitting parameters shared across these shells of dynamic and trading models.

%A M. C. Forman
%A A. Aggoun
%A M. McCormick
%T Simulated Annealing for Optimisation and Characterisation of Quantisation Parameters in Integral 3D Image Compression
%B Image Processing II: Mathematical Methods, Algorithms and Applications
%E J. M. Blackledge
%E M. J. Turner
%I The Institute of Mathematics and its Applications / Horwood Publishing
%D 2000
%P 399-413
%O URL http://www.imtech.cse.dmu.ac.uk/3d-med/pubs/ima98.pdf

%A M. C. Forman
%T Compression of Integral Three Dimensional Television Pictures
%R Ph.D. Thesis
%I De Montfort University
%C Leicester, United Kingdom
%D 2000

These papers use ASA in quantisation scheme optimisation in three dimensional image compression.

%A L. Ingber
%A J.K. Wilson
%T Statistical mechanics of financial markets: Exponential modifications to Black-Scholes
%J Mathematical Computer Modelling
%V 31
%N 8/9
%P 167-192
%D 2000
%O URL https://www.ingber.com/markets00_exp.pdf

ABSTRACT: The Black-Scholes theory of option pricing has been considered for many years as an important but very approximate zeroth-order description of actual market behavior. We generalize the functional form of the diffusion of these systems and also consider multi-factor models including stochastic volatility. We use a previous development of a statistical mechanics of financial markets to model these issues. Daily Eurodollar futures prices and implied volatilities are fit to determine exponents of functional behavior of diffusions using methods of global optimization, Adaptive Simulated Annealing (ASA), to generate tight fits across moving time windows of Eurodollar contracts. These short-time fitted distributions are then developed into long-time distributions using a robust non-Monte Carlo path-integral algorithm, PATHINT, to generate prices and derivatives commonly used by option traders. The results of our study show that there is only a very small change in at-the money option prices for different probability distributions, both for the one-factor and two-factor models. There still are significant differences in risk parameters, partial derivatives, using more sophisticated models, especially for out-of-the-money options.

%A S. Sakata
%A H. White
%T High breakdown point conditional dispersion estimation with application to S&P 500 daily returns volatility
%J Econometrica
%V 66
%P 529-567
%D 1998

This paper continues the thesis of Shinichi Sakata, referenced below, studying estimation methods for conditional location and dispersion models, with an application to estimating the conditional volatility of the S&P 500 cash index. Contact Shinichi Sakata <shinichi.sakata@gmail.com> for more information.

%A L. Ingber
%T Data mining and knowledge discovery via statistical mechanics in nonlinear stochastic systems
%J Mathematical Computer Modelling
%V 27
%N 3
%P 9-31
%D 1998
%O URL https://www.ingber.com/path98_datamining.pdf

This paper emphasizes the generic use of ASA in a wide class of multivariate nonlinear stochastic systems, together with other codes and a stochastic calculus, to datamine large data sets and to extract knowledge from fitted patterns of information.

ABSTRACT: A modern calculus of multivariate nonlinear multiplicative Gaussian-Markovian systems provides models of many complex systems faithful to their nature, e.g., by not prematurely applying quasi-linear approximations for the sole purpose of easing analysis. To handle these complex algebraic constructs, sophisticated numerical tools have been developed, e.g., methods of adaptive simulated annealing (ASA) global optimization and of path integration (PATHINT). In-depth application to three quite different complex systems have yielded some insights into the benefits to be obtained by application of these algorithms and tools, in statistical mechanical descriptions of neocortex (short-term memory and electroencephalography), financial markets (interest-rate and trading models), and combat analysis (baselining simulations to exercise data).

%A K. Wu
%A M.D. Levine
%T 3-D shape approximation using parametric geons
%J Intl J Image and Vision Computing
%V 15
%N 2
%P 143-158
%D 1997

This paper follows their 1994 paper below, using ASA to solve some very difficult imaging problems that did not yield to other global optimization techniques. Contact Kenong Wu <wu@cim.mcgill.ca> for further information.

%A L. Ingber
%T Statistical mechanics of neocortical interactions: Canonical momenta indicators of electroencephalography
%J Physical Review E
%V 55
%N 4
%P 4578-4593
%D 1997
%O URL https://www.ingber.com/smni97_cmi.pdf
%A L. Ingber
%T Statistical mechanics of neocortical interactions (SMNI)
%R SMNI Lecture Plates
%I Lester Ingber Research
%C Chicago, IL
%D 1997
%O URL https://www.ingber.com/smni97_lecture.pdf

%A M. Bowman
%A L. Ingber
%T Canonical momenta of nonlinear combat
%B Proceedings of the 1997 Simulation Multi-Conference, 6-10 April 1997, Atlanta, GA
%I Society for Computer Simulation
%C San Diego, CA
%D 1997
%O URL https://www.ingber.com/combat97_cmi.pdf

This paper uses ASA to fit combat simulation data, illustrating how to develop measures of effectiveness of systems as they synergistically contribute to nonlinear combat contexts. Canonical Momenta Indicators (CMI) offer graphical decision aids faithful to the underlying multivariate nonlinear stochastic model.

%A D.G. Mayer
%A J.A. Belward
%A K. Burrage
%T Use of advanced techniques to optimize a multi-dimensional dairy model
%J Agricultural Systems
%V 50
%P 239-253
%D 1996

Available methods for the optimization of agricultural systems vary widely, in terms of derivation, applicability and performance. A whole-farm dairying model with 16 separate, interacting managerial options was subjected to optimisation by the hill-climbing (quasi-Newton), direct search (simplex), genetic algorithm (GENESIS) and simulated annealing (VFSR) techniques. The latter two clearly out-performed the former, with simulated annealing always identifying the global optimum.

%A D.G. Mayer
%A P.M. Pepper
%A J.A. Belward
%A K. Burrage
%A A.J. Swain
%T Simulated annealing - A robust optimization technique for fitting nonlinear regression models
%B Proceedings 'Modelling, Simulation and Optimization' Conference, International Association of Science and Technology for Development (IASTED), 6-9 May 1996 Gold Coast
%D 1996

Optimization procedures are required for the fitting of nonlinear regression models to data. Whilst generally smooth in nature, the objective function can contain multiple local optima or sub-optimal plateaus, creating difficulties for the optimization routine. Hill-climbing techniques are used in the majority of statistical packages. We investigate their performance on difficult parameterizations of a climbing techniques are used in the majority of statistical packages. We investigate their performance on difficult parameterizations of a conception rates model, for two data sets. We also evaluate a simulated annealing algorithm, and demonstrate its superior performance on this type of problem.

%A D.G. Mayer
%A J.A. Belward
%A K. Burrage
%A M.A Stuart
%P 1995
%T Optimization of a dairy farm model - Comparison of simulated annealing, simulated quenching and genetic algorithms
%B Proceedings 1995 International Congress on Modelling and Simulation, 27-30 November 1995, University of Newcastle
%P 33-38
%D 1995

Available optimization techniques vary widely in terms of derivation, application, and efficiency. A complex dairy farm model was used to benchmark those which have been used previously in the model optimisation field. The more traditional methods, including random search, hill-climbing and direct search, were notably inferior in identifying the economic optimum of this agricultural system. Genetic algorithms proved quite efficient, but overall results were marginally down on those from the simulated annealing methods. Initially, these proved to be quite slow, but a retuned simulated annealing algorithm was found to be more efficient, thorough and safe. It's extension to simulated quenching proved best for this problem, safely identifying the optimum at a good rate of convergence. As this program is freely available and relatively easy to use, it is strongly recommended. Also, initial investigations with the tabu search strategy are reported, which show it to have potential.

Contact David G. Mayer <davidm@ysun73.ari.dpi.qld.gov.au> for further information.

%A A. Su
%A S. Mager
%A S.L. Mayo
%A H.A. Lester
%T A multi-substrate single-file model for ion-coupled transporters
%J Biophysical Journal
%V 70
%D 1996
%P 762-777

This paper develops a parametrized model of ion-coupled transporters fit to GABA transporter GAT1, which is then extrapolated to describe other experimental data. Contact Henry Lester <lester@caltech.edu> for more information.

%A R. Desai
%A R. Patil
%T SALO: Combining simulated annealing and local optimization for efficient global optimization
%R 9th Florida AI Research Symposium
%D 1996

While I disagree with the authors that SALO is a true annealing algorithm, even as a new quenching algorithm it represents a useful contribution to an analyst's toolbox. Contact Rutvick Desai <rudesai@indiana.edu> for more information.

%A L. Ingber
%T Canonical momenta indicators of financial markets and neocortical EEG
%B Progress in Neural Information Processing
%E S.-I. Amari, L. Xu, I. King, and K.-S. Leung
%I Springer
%C New York
%P 777-784
%D 1996
%O Invited paper to the 1996 International Conference on Neural Information Processing (ICONIP'96), Hong Kong, 24-27 September 1996. ISBN 981 3083-05-0. URL https://www.ingber.com/markets96_momenta.pdf

A brief introduction to canonical momenta is included in ingber_projects.html.

%A L. Ingber
%T Statistical mechanics of nonlinear nonequilibrium financial markets: Applications to optimized trading
%J Mathematical Computer Modelling
%V 23
%N 7
%P 101-121
%D 1996

%A L. Ingber
%T Adaptive simulated annealing (ASA): Lessons learned
%J Control and Cybernetics
%V 25
%N 1
%P 33-54
%D 1996
%O Invited paper to a special issue of Control and Cybernetics on "Simulated Annealing Applied to Combinatorial Optimization." URL https://www.ingber.com/asa96_lessons.pdf

This is a special issue on Simulated Annealing Applied to Combinatorial Optimization, prepared by the Polish Academy of Sciences Systems Research Institute. The listing of other contributors to this special issue is in asa96_vidal_nahorski.txt

%A L. Ingber
%R Report No. TMWCMASA-96
%I Lester Ingber Research
%C McLean, VA
%D 1996
%O URL https://www.ingber.com/markets96_brief.pdf

This paper gives relatively non-technical descriptions of ASA and canonical momenta, and their applications to markets and EEG. The paper was solicited by AI in Finance prior to cessation of publication.

%A R.A. Cozzio-Bu\*:eler
%T The design of neural networks using a priori knowledge
%R Ph.D. Thesis
%I Swiss Fed. Inst. Tech.
%C Zurich, Switzerland
%D 1995

This thesis used ASA to minimize the one-step-ahead forecasting error of a neural network, incorporating differential equations as a priori knowledge. Contact Rico Cozzio <cozzio@ln.active.ch> for more information.

%A S. Sakata
%T High breakdown point estimation in econometrics
%R Ph.D. Thesis
%I University of California at San Diego
%C La Jolla, CA
%D 1995

This paper studied estimation methods for conditional location and dispersion models, with an application to estimating the conditional volatility of the S&P 500 cash index. Contact Shinichi Sakata <shinichi.sakata@gmail.com> for more information.

%A M.K. Sen
%A P.L. Stoffa
%T Global Optimization Methods in Geophysical Inversion
%D 1995
%I Elsevier
%C The NetherLands
%O ISBN 0-444-81767-0

One of the major goals of geophysical inversion is to find earth models that explain the geophysical observations. Both local and global optimization methods are used in the estimation of material properties from geophysical data. Contact Mrinal Sen <mrinal@utig2.ig.utexas.edu> for more information.

%A M. Buszko
%A D.C. Wang
%A M.F. Kempka
%A E. Szczesniak
%A E.R. Andrew
%T Application of Adaptive Simulated Annealing to Optimization of Gradient Coils with Concentric Return Paths
%R NMR Poster Session '95 http://micro.ifas.ufl.edu/
%I U. Florida
%C Gainesville, FL
%D 1995

This paper addresses the optimization of magnetic-field gradient (MFG) coils, one of the fundamental problems in designing magnetic resonance imaging systems. Contact Marian Buszko <marian@micro.ifas.ufl.edu> for more information.

%A X.gz. Tang
%A E.R. Tracy
%A A.D. Boozer
%A A. deBrauw
%A R. Brown
%T Symbol sequence statistics in noisy chaotic signal reconstruction
%J Physical Review E
%V 51
%N 4
%D 1995
%P (in press)

This paper presents an investigation of fitting dynamical systems models to observed data, by comparing the resultant symbolic dynamics transition probabilities of iterated model and observed data. Contact Xianzhu Tang <tang@perceval.apne.columbia.edu> for further information.

%A R. Brown
%A N.F. Rulkov
%A N.B. Tufillaro
%T The effects of additive noise and drift in the dynamics of the driving on chaotic synchronization
%J Phys. Lett.
%N
%V
%D 1994

%A R. Brown
%A N.F. Rulkov
%A N.B. Tufillaro
%T Synchronization of chaotic systems: the effects of additive noise and drift in the dynamics and driving
%J Physical Review E
%N
%V
%D 1994

These two papers used ASA to fit nonlinear forms to data as part of a project examining the effects of additive noise and drift on chaotic synchronization. Contact Nicholes Tufillaro <nbt@reed.edu> for further information.

%A B. Cohen
%R Training synaptic delays in a recurrent neural network, M.S. Thesis
%I Tel-Aviv University
%C Tel-Aviv, Israel
%D 1994

This thesis used ASA to optimize both synaptic-delay parameters and weights of neural networks. The addition of (realistic) nonlinear complexity of delayed recurrent activity permits using fewer parameters in several tasks. Contact Barak Cohen <goman@netvision.net.il> for further information.

%A K. Wu
%A M. D. Levine
%T Recovering parametric geons from multiview range data
%B IEEE Conference on Computer Vision & Pattern Recognition
%P 159-166
%D 1994
%I IEEE Computer Society
%C Seattle

This paper used ASA to solve some very difficult imaging problems that did not yield to other global optimization techniques. Contact Kenong Wu <wu@cim.mcgill.ca> for further information.

%A A.F. Atiya
%A A.G. Parlos
%A L. Ingber
%R A reinforcement learning method based on adaptive simulated annealing
%D 1993
%O URL https://www.ingber.com/asa03_reinforce.pdf

Reinforcement learning is a hard problem and the majority of the existing algorithms suffer from poor convergence properties for difficult problems. In this paper we propose a new reinforcement learning method, that utilizes the power of global optimization methods such as simulated annealing. Specifically, we use a particularly powerful version of simulated annealing called Adaptive Simulated Annealing (ASA). Towards this end we consider a batch formulation for the reinforcement learning problem, unlike the online formulation almost always used. The advantage of the batch formulation is that it allows state-of-the-art optimization procedures to be employed, and thus can lead to further improvements in algorithmic convergence properties. The proposed algorithm is applied to a decision making test problem, and it is shown to obtain better results than the conventional Q-learning algorithm.

%A L. Ingber
%T Simulated annealing: Practice versus theory
%J Mathematical Computer Modelling
%V 18
%N 11
%D 1993
%P 29-57
%O URL https://www.ingber.com/asa93_sapvt.pdf

This paper illustrates the use of the QUENCHing and REANNEALing OPTIONS in ASA, and contains an expanded section describing the use of simulated annealing across many disciplines.

%A G. Blais
%A M.D. Levine
%T Registering multiview range data to create 3D computer objects
%R TR-CIM-93-16
%I Center for Intelligent Machines, McGilo University
%D 1993

This paper used ASA to solve some very difficult imaging problems that did not yield to other global optimization techniques. Contact Gerard Blais <gblais@bnr.ca> for further information.

%A M. Wofsey
%T Technology: Shortcut Tests Validity of Complicated Formulas
%J The Wall Street Journal
%V CCXXII
%N 60
%P B1
%D 24 September 1993

%A G. Indiveri
%A G. Nateri
%A L. Raffo
%A D. Caviglia
%R Report
%T A neural network architecture for defect detection through magnetic inspection
%I University of Genova
%C Genova, Italy
%D 1993

This paper is an application of ASA to neural networks. You can contact Giacomo Indiveri <giacomo@dibe.unige.it> for further information.

%A L. Ingber
%T Statistical mechanics of combat and extensions
%B Toward a Science of Command, Control, and Communications
%E C. Jones
%I American Institute of Aeronautics and Astronautics
%C Washington, D.C.
%D 1993t93.ps.gz
%P 117-149
%O ISBN 1-56347-068-3. URL https://www.ingber.com/combat93_c3sci.pdf

This is the most recent of a series of papers using VFSR on a 1988 project baselining the JANUS(T) combat simulation to exercise data from the National Training Center (NTC).

%A B. Rosen
%T Function optimization based on advanced simulated annealing
%J IEEE Workshop on Physics and Computation - Phys Comp '92
%I IEEE Press
%C Dallas, Texas
%P 289-293
%D 1992

This paper compared standard Boltzmann annealing with "fast" Cauchy annealing with VFSR, and concluded that VFSR was superior in both efficiency and accuracy.

%A L. Ingber
%T Generic mesoscopic neural networks based on statistical mechanics of neocortical interactions
%J Physical Review A
%V 45
%N 4
%P R2183-R2186
%D 1992
%O URL https://www.ingber.com/smni92_mnn.pdf

This Rapid Communications presented an algorithm generalizing ASA by a confluence of features from ASA, modern calculus of multivariate nonlinear stochastic systems, statistical mechanics of neocortical interactions (SMNI), and parallel processing.

%A L. Ingber
%A B. Rosen
%T Genetic algorithms and very fast simulated reannealing: A comparison
%J Mathematical Computer Modelling
%V 16
%P 87-100
%D 1992
%O URL https://www.ingber.com/asa92_saga.pdf

This paper showed VFSR to be superior to a standard genetic algorithm (GA) simulation on a suite of standard GA test problems.

%A L. Ingber
%T Statistical mechanical aids to calculating term structure models
%J Physical Review A
%V 42
%N 12
%D 1990
%P 7057-7064
%O URL https://www.ingber.com/markets90_interest.pdf

%A L. Ingber
%A M.F. Wehner
%A G.M. Jabbour
%A T.M. Barnhill
%T Application of statistical mechanics methodology to term-structure bond-pricing models
%J Mathematical Computer Modelling
%V 15
%N 11
%D 1991
%P 77-98
%O URL https://www.ingber.com/markets91_interest.pdf

These two papers used VFSR to fit a current economic model of coupled short-term and long-term interest rates to bond data.

%A L. Ingber
%T Statistical mechanics of neocortical interactions: A scaling paradigm applied to electroencephalography
%J Physical Review A
%N 6
%V 44
%P 4017-4060
%D 1991
%O URL https://www.ingber.com/smni91_eeg.pdf

This paper fit EEG data from a clinical study to a model of large-scale neuronal activity in the human brain.

%A L. Ingber
%T Very fast simulated re-annealing
%J Mathematical Computer Modelling
%V 12
%N 8
%P 967-973
%D 1989
%O URL https://www.ingber.com/asa89_vfsr.pdf

This was the first VFSR paper.

Lester Ingber <ingber@ingber.com>
$Id: asa_papers.html,v 10.158 2016/12/19 16:49:17 ingber Exp ingber$