Tutorial 8: Self-Organizing AI: From Cybernetics to Multi-Stage Selection

Speakers

 

Abstract

This tutorial introduces a principled, historically informed, and hands-on path to building AI systems that cope with real-world complexity via self-organization and adaptivity. We start from the cybernetic roots – Wiener’s feedback, Ashby’s law of requisite variety, and Shannon’s information theory – then trace forward through neural computation and evolutionary methods to contemporary Multi-Stage Selection Procedures (MSSP) for autonomous model synthesis. Participants will learn how directed selection, inconclusive decision principles, and emergent modeling translate into practical workflows for AutoML-like pipelines, time-series modeling, portfolio construction, and encrypted-signal prediction (e.g., Numerai). The session blends conceptual foundations with concrete case studies and code-level guidance (Matlab library), equipping attendees to design, analyze, and explain self-organizing AI solutions in data-scarce, noisy, and highly variable settings.

 

Target Audience

Who should attend: AI/ML researchers and practitioners, data scientists in finance/engineering, complexity scientists, and graduate students interested in robust modeling under uncertainty.

Prerequisites: Comfort with linear algebra, probability, and regression; working knowledge of ML model selection; optional Matlab/Python familiarity for code demos.

Practical outcomes: Attendees will be able to (i) reason about systems via feedback and requisite variety; (ii) implement MSSP-style model synthesis; (iii) mitigate overfitting with directed selection; (iv) analyze/visualize model genealogy and error dynamics; and (v) apply the workflow to noisy, short, or non-stationary datasets.

 

Outline and Description of the Tutorial

1 Conceptual Core

  • Complexity in AI: Why “good regulators are models of the system”; feedback and emergent behavior; information flow. (Ashby, Wiener, Shannon; emergence à la Huxley.)
  • From Early AI to Self-Organization: McCulloch–Pitts neurons, Rosenblatt’s perceptron limits, and Ivakhnenko’s GMDH as adaptive structure discovery.
  • Directed vs. Natural Selection: Gabor’s principle of inconclusive decisions and Marchev’s MSSP—growing model complexity via staged selection to avoid overfitting.

2 MSSP Methodology

  • Population & primitives: Feature primitives (linear, log, exponential, power, reciprocal, sine, etc.).
  • Generation: Exhaustive pair crossing; linear/power mating functions; tensor-friendly breeder representation.
  • Selection: Thresholding by mean relative error; complexity-by-layers rule; terminal conditions (generations/time/error progression).
  • Explainability: Genealogical trees, progression plots, pruned graphs, residual diagnostics, and symbolic forms.

3 Applications & Demos

  • Portfolio Modeling: Treat feature weights as portfolios; evolve weight structures under constraints; risk-weighted performance analysis.
  • Encrypted-Signal Prediction (Numerai): Handling low-signal, era-based generalization with MSSP; cross-validation by eras; performance visualization.
  • Matlab Library (v0.9 → roadmap): Data prep, lagging/normalization, solver stack (QR, Cholesky, banded, etc.), logging table, and analysis UI; roadmap to 1.0+.

4 Hands-On Segment

  • Recreate a small MSSP pipeline from primitives to selected breeders; interpret error dynamics; export a symbolic model; discuss deployment considerations.

 

Reading List

Foundations / Classics

  • Shannon, C.E. (1940) An Algebra for Theoretical Genetics. PhD thesis. Massachusetts Institute of Technology. (Also: Shannon, C.E. (1993) ‘An Algebra for Theoretical Genetics’, in Sloane, N.J.A. and Wyner, A.D. (eds.) Claude Elwood Shannon: Collected Papers. New York: IEEE Press, pp. 891–896.)
  • Huxley, T.H. and Huxley, J. (1947) Evolution and Ethics 1893-1943. London: Pilot Press.
  • Wiener, N. (1948) Cybernetics: or Control and Communication in the Animal and the Machine. New York: John Wiley & Sons. (Also: Wiener, N. (1961) Cybernetics: or Control and Communication in the Animal and the Machine. 2nd edn. Cambridge, MA: MIT Press.)
  • Ashby, W.R. (1958) ‘Requisite variety and its implications for the control of complex systems’, Cybernetica, 1(2), pp. 83–99.

Self-Organization & Early AI

  • Turing, A.M. (1950) ‘Computing machinery and intelligence’, Mind, 59(236), pp. 433–460.
  • von Neumann, J. (1966) Theory of Self-Reproducing Automata. Edited by A.W. Burks. Urbana: University of Illinois Press. (Based on manuscripts from 1948–1953.)
  • Ivakhnenko, A.G. (1970) ‘Heuristic self-organization in problems of engineering cybernetics’, Automatica, 6, pp. 207–219.
  • Ivakhnenko, A.G. (1971) ‘Polynomial theory of complex systems’, IEEE Transactions on Systems, Man, and Cybernetics, SMC-1(4), pp. 364–378.

Decision & Evolutionary Perspectives

  • Gabor, D. (1969) ‘Open-Ended Planning’, in Jantsch, E. (ed.) Perspectives of Planning. Paris: OECD, pp. 329–350. (Based on ideas from Gabor, D. (1959) Electronic Inventions and Their Impact on Civilization. Inaugural Lecture, 3 March. London: Imperial College of Science and Technology.)
  • Fogel, L.J., Owens, A.J. and Walsh, M.J. (1966) Artificial Intelligence through Simulated Evolution. New York: John Wiley & Sons.
  • Ivakhnenko, N.A. and Marchev, A.A. (1978) ‘Self-organization of a mathematical model for long-range planning of construction and installation activities’, Soviet Automatic Control, 11(3), pp. 9–14.
  • Ivakhnenko, A.G., Ivakhnenko, G.A. and Müller, J.-A. (1994) ‘Self-organization of neural networks with active neurons’, Pattern Recognition and Image Analysis, 4(2), pp. 185–196.

MSSP & Applications

  • Marchev, A.A. and Motzev, M.R. (1989) ‘Principles of Multi-Stage Selection in Software Development in Decision Support Systems’, in Lewandowski, A. and Stanchev, I. (eds.) Methodology and Software for Interactive Decision Support. Berlin: Springer-Verlag (Lecture Notes in Economics and Mathematical Systems, Vol. 337), pp. 181–189.
  • Marchev A. (2012) Multi-stage selection procedure for investment portfolio management, Proceedings of the IEEE International Conference on Control Applications, art. no. 6402732, pp. 593 – 598, DOI: 10.1109/CCA.2012.6402732
  • Marchev A., Jr., Marchev A. (2014) Autonomous portfolio investment by multi-stage selection procedure, AIP Conference Proceedings, 1631, pp. 313 – 322, DOI: 10.1063/1.4902492
  • Marchev A., (2016) Self-organization types for autonomous investment portfolio, 2016 IEEE 8th International Conference on Intelligent Systems, IS 2016 – Proceedings, art. no. 7737498, pp. 658 – 663, DOI: 10.1109/IS.2016.7737498
  • Marchev A., Jr., Piryankova M. (2022) Evolution of the Concept of Self-Organization by the Founding Fathers of A.I., 10th International Scientific Conference on Computer Science, COMSCI 2022 – Proceedings, DOI: 10.1109/COMSCI55378.2022.9912577
  • Marchev A. (2023) An Implementation of Self-Organizing Multi-Stage Selection Procedure, 2023 11th International Scientific Conference on Computer Science, COMSCI 2023 – Proceedings, DOI: 10.1109/COMSCI59259.2023.10315898

 

Vertical

Cutting-edge AI Research

Timeline

4 hours