This briefing is the first step for those considering research and development (R&D) involving CORTECONs (COntent-Retentive, TEmporally-CONnected neural networks), which have been developed by Themesis Principal Alianna J. Maren, Ph.D.
We organize this briefing using the five well-known questions for reporting:
- Who,
- What,
- Where,
- How, and
- Why.
This briefing accompanies a YouTube presentation, and the link to that presentation will be included HERE as soon a that presentation is complete and uploaded to YouTube.
WHO – Who You Are; Who We Are
The “who” is really two parts: who YOU are, and who WE are.
Who YOU Are
This briefing is an important first step if you are deciding to invest corporate or academic team resources into CORTECON R&D. This means that you have decision-making and budgetary authority to direct a long-term research effort.
We say “long-term” because it take a while to get up to speed with CORTECONs. There is physics as well as code.
CORTECONs use free energy minimization within a cluster variation method (CVM) grid as their primary internal engine. This is a physics method; it is an extension of the well-known Ising equation. This means that sufficient people on your team should be able to understand the physics as well as the architecture and the code.
Thus, the first question that any executive-tier research director should ask is actually a set of two related “people” questions:
- Do I have the right people already in place to take on this work? Do they understand enough of the basic physics? That is, do they already understand how a basic Boltzmann machine works? Do they understand variational inference?
- If your people (or you, if you are self-directing your effort) do not have this background in place, then – are you willing to invest the effort to get your people (or yourself) up to speed?
The second question to be asking yourself, before committing to any CORTECON work, is: what is your long-range goal in working with CORTECONs? Are you ready to commit sufficient resources (both money and time) to getting a project done – given that there are no “scripts,” there is nothing that you can download from Hugging Face, and that you may not even want some of the basic Keras scripts. (All CORTECON code is pure, raw Python. No scripts.)
The third and final question to ask, before starting any work – or before contacting us – is: Are you willing to accept that right now, CORTECONs are in their very early R&D stage? That means, while we can talk in general terms about potential applications, there are no immediate applications from this work – and there may not be for at least a year or more. Very likely, at least two years.
If your answer to all three questions is affirmative, then read on!
Who WE Are
Themesis, Inc. is a very small, boutique AI R&D company. Our Founder and Chief Scientist is Dr. Alianna J. Maren, a serial inventor and entrepreneur.
CORTECON History – A Very Brief Overview
Dr. Maren invented CORTECONs, drawing on both her Ph.D. research in statistical mechanics (on the cluster variation method), as well as early work in neural networks. (She was the senior author of the Handbook of Neural Computing Applications, Academic, 1990.) She got the inspiration for creating CORTECONs while teaching at Radford University in 1991, and while associating with the renowned Dr. Karl Pribram, who had established the Center for Brain Research and Informational Sciences (B.R.A.I.N.S.) at Radford.
1n 1992-1993, Dr. Maren published three conference papers based on the earliest CORTECON work, but had to put CORTECONs into “hibernation” for over two decades while she worked in other corporations.
Starting in 2014, Dr. Maren re-invigorated CORTECON work – and in 2016, began investigating how CORTECONs could be used within a variational inference context for artificial general intelligence (AGI). (Actually, her focus was on using CORTECONs within active inference – inspired by Karl Friston’s work.)
In 2022, she devised the necessary new divergence method that would allow the cluster variation method to be more effectively used, and termed this the “Kikuchi-Maren divergence” (as compared with the more well-known Kullback-Leibler divergence).
In 2023, Dr. Maren began public presentation of CORTECONs, beginning with an extensive series of YouTube videos culminating in CORTECONs as a new neural networks class. (See the Themesis, Inc. YouTube channel.)
Themesis, Inc. and Intellectual Property Ownership
Dr. Maren founded Themesis, Inc. in 2021 as a “container” for the intellectual property of CORTECONs and associated materials. She has transferred her intellectual property rights to Themesis, Inc.
During the entire CORTECON development period of 1992, Dr. Maren received only a small grant from the Jeffries Foundation, which was used to support student research assistants and conference travel. All CORTECON concepts and architecture design were done without any funding support. The primary CORTECON concepts and architecture visualizations were done before receiving the Jeffries Foundation grant.
Since re-invigorating CORTECON work in 2014, Dr. Maren has worked independent from any funding sources. She has taught in adjunct positions at colleges and, since 2015, in Northwestern University’s Master of Science in Data Science program (within the School of Professional Studies). Dr. Maren has had exclusive ownership of CORTECON and related inventions during this time. She has transferred intellectual property ownership to Themesis, Inc.
Themesis, Inc. (“Themesis”) has been privately funded by those intimately involved with Themesis corporate formation and continued work. There have been no government contracts and no external (e.g., venture capital) investments. Themesis owns all of the CORTECON and related inventions that it has created, without obligation to any other party.
Themesis chooses to make certain elements of the CORTECON architecture and code publicly available under the MIT license agreement.
WHAT – What CORTECONs Are, and What They Are (Potentially) Good For
CORTECONs are a new class of neural network architecture. This means that they are on a peer level with such major innovations as the Boltzmann machine, convolutional neural networks, and the like.
Most neural networks operate in a “vertical” sense – inputs go into one (“bottom”) end, various operations occur at successive network layers, and the outputs emerge at the other (“top”) end. Connection weights between nodes on the different layers are learned during a training stage, whereby the connection weights take on a value that minimize some function.
In all of the primary existing neural networks classes, once an input has been presented, the influence of this input works its way through the network until it reaches the output layer.
If there is no input, then there is no activity within the network.
This means that the network has no capacity for “thought” in the absence of input stimulus. It cannot ruminate, free-associate, or daydream.
Existing neural networks are very tied to achieving specific goals, e.g. auto-association or classification. The network’s training regime is designed to achieve maximal performance, based on training with a pre-specified training data set.
We will not address various methods to update the connection weights as new training data is obtained, etc.
Rather, we focus on the fact that there is room for neural networks whose performance is not rigidly locked into specific goal-achievement. This will allow various forms of flexible thinking to emerge.
CORTECONs can engage in this more “flexible” thinking because they have an additional process; ongoing free energy minimization across the lateral layer. This allows for activity to take place even when there is no immediately-presented input stimulus. The free energy minimization uses the cluster variation method, a physics (statistical mechanics) method invented in 1951 by Ryoichi Kikuchi, and further developed by Kikuchi and Brush in 1967. (See THIS BLOGPOST for more descriptions and reference details, as well as link to the YouTube overviewing CORTECON architectures and equations.)
In short, CORTECONs will (likely) be useful for advanced AI (artificial general intelligence, or AGI) systems that need to independently associate ideas and concepts that have been presented to it over time. CORTECONs should be able to engage in a range of “flexible thinking,” specifically (as identified in THIS BLOGPOST):
- Memory persistence (“Holding that thought”),
- Learned temporal associations (“That reminds me …”), and
- Random activation foraging (“California Dreamin’ …”).
We further envision that CORTECONs can play a role within variational inference systems (active inference), providing a model that in itself can be brought to free energy equilibrium, so that the model (for any pair of control parameters) can be assuredly at or close to equilibrium and thus likely representing a natural or desired state.
While variational inference (or active inference, which is an evolution from variational inference) is not – in and of itself – a form of AGI, we envision that it is a necessary foundation and building block for creating AGIs. Variational (active) inference requires a model that can be adapted (through parameter variance) to be brought close to a set of observations. Traditional models are, for example, “families of exponential equations.” CORTECONs offer an entirely different class of models, which should allow greater versatility when using variational (or active) inference.
WHERE – Where We Are in CORTECON Evolution
CORTECONs are still early in their evolution.
We have enough done to present crucial elements to the R&D community. These include:
- Overall architecture and equations,
- Preliminary code (2nd-gen, object-oriented beginning fourth quarter of 2023), and
- Preliminary phase space investigations and worked examples.
In other words, if a team wishes to start CORTCON research, there is a good starting place – the initial equations are understood; the proofs have been worked out and published (in one or more of our various arXiv and privately-published papers). We have identified parameters for the edges of the phase space. We have starting-point code, which we are willing to share. (See various CORTECON YouTubes and blogposts.)
The literature review has been addressed in our various publications.
To the best of our knowledge, there is no competitive work at this time (fourth quarter, 2023).
There is sufficient material available so that a research team or an individual can make rapid progress, on their own.
HOW – How to Work with CORTECONs
You can pursue CORTECON work on your own. We have made sufficient materials available so that you can get started and make good progress.
In our YouTubes and associated blogposts, we describe the architecture and equations. We have published papers, and will shortly publish a summary/anthology that collects and annotates various key resources. (This briefing will be updated when we make that available.)
We make certain code elements available in our various Themesis, Inc. GitHub repositories, offered under the MIT license agreement.
If you wish to accelerate your progress, you can work with Themesis as your Muse.
We will make details available shortly, and post a link to our protocols here when they become available.
WHY – Why CORTECONs?
CORTECONs are an emerging neural network class. As such, the payoff cannot be readily determined.
Those who wish to engage in CORTECON R&D should seriously consider both their risk tolerance AND their tolerance for long-term (as opposed to short payoff) work.
We will offer more details and guidance on working with CORTECONs as time goes on.
Please continue to scan our blogposts (be certain that you have “opted-In” with us, see www.themesis.themsis for the “Opt-In” form), and check our YouTubes. (Subscribe to the Themesis, Inc. YouTube channel.)
Once you have “opted-in” with us, be certain that you have moved Themesis emails to your primary folder. (We will have some follow-up emails as soon as you join us, and also send out a general email on Thursday or Friday of each week.)
Certain emails will ask you whether or not you wish to work more closely with us. Your response to those emails will let us know if you want more information.
From time-to-time, we will make special offers available – for VERY SHORT durations. Thus, it is in your best interests to track emails from us, as we will make announcements and offers via email that are not presented to the community-at-large.
Very best wishes, from all of us to all of you, for your successful work!
Alianna J. Maren, Ph.D.
Founder and Chief Scientist, Themesis, Inc.