C(S) Constraint Convergence

By Jessica Lake

\

I am finally catching on to how I think.

More importantly, I cleared up two things I had confused as one.

One is an algorithm.

The other is a heuristic way of learning.

They work together so closely that I had mistaken them for one mechanism.

They are not.

And the algorithm is not peculiar to me.

Everyone does it.

Poet or not. Consider the agony of finding that exact right word.

The nib hovered, stalled over the white, waiting for its one and only. The word “love” was DOA—too tired, emblazoned on too many greeting cards. Jessica needed something tender with teeth. Every attempt withered before reaching the page.

Devotion? No, too clerical. It spoke of ornate altars and ritual.

Hunger? Too fierce. Too much of the profane. Too little sacred.

Adoration? That is a removal. She is on top of him.

Ache? Whining. Like he is a decayed tooth.

Fealty? Round tables. Said once and gone.

Grace? Too detached from the dirt and warmth of the world.

The mantel clock ticked sans love and demanding. Outside, the gusts raked the bare branches against the window. Skeletal fingers.

Jessica pressed pen to parchment.

Gravity.

Notice what she did not do?

She did not say what word she wanted.

How could she?

What’s that word…

She said the words she did not want.

A process of elimination.

Each failure narrowed the admissible candidates. More importantly, asking why it failed exposed a new constraint. That constraint could then eliminate other candidates that failed in the same way.

That is C(S): Constraint Convergence.

A WORD to the reader. I am going to discuss semantic knowledge using words. That does not mean semantic knowledge is necessarily stored in words. Unfortunately, I cannot inspect abstractions. So when I speak of realms and their marvels, I will express them linguistically.

C(S) is remarkably modest.

It does not understand a problem.

It does not know what the answer should be.

    Populate a space with
    probables. 
┌─► Coin a constraint.
│   
│   Apply the constraint. 
│
│   Some probables bite the
│   dust.
│
│   Ditch the inadmissible.
│
└─  Ramírez: "What I do
    know is … we are all
    drawn to a single place. 
    In the end, there can be
    only one."

This is glory C(S).

C(S) is for…

•               When you know what you want.

•               But you cannot quite put your finger on it.

REALMS

C(S) does not operate over everything at once.

Knowledge is partitioned into realms.

A realm is simply a bounded space in which knowledge can be expressed in compatible terms.

Geometry is a realm.

Language is a realm.

Neuroscience may be a realm.

Social relationships may be a realm.

Each realm has its own lingo.

Geometry can express position, distance, orientation, containment, topology, proportion, and shape.

Language can express meaning, reference, category, relation, and specificity.

Neuroscience has another vocabulary entirely.

Knowledge within a realm is expressed in the lingo of that realm.

This gives the realm a system of relations and permissible operations.

That system determines what can interact with what, which relationships are meaningful, and how constraints can operate over them.

Most knowledge remains bound to its realm because it is expressed in terms peculiar to that realm.

C(S) itself does not care what the realm means.

Change the realm and its system of relations, and the same mechanism can operate on something entirely different.

FUNDAMENTAL PRINCIPLES

Most knowledge begins parochial.

It belongs to its realm.

A relationship discovered in neuroscience may initially make sense only in the lingo of neuroscience.

A relationship discovered in geometry may initially be expressed only geometrically.

But occasionally a relationship proves more general than the realm in which it was discovered.

It survives translation.

It can be carried into another realm and expressed in that realm’s lingo without losing the relationship itself.

That is what I mean by a fundamental principle.

By fundamental principle, I do not mean an absolute invariant.

Reality is too messy for that.

A fundamental principle may instead be statistical:

A fuzzy regularity.

Something that tends to remain true.

Something that survives repeated encounters with reality.

This distinction matters.

Earlier I called collections of these things generators.

That was wrong.

I had confused what was stored with something that actively generated an expression.

Nothing needs to be generated.

LEARNING A SOLUTION

Now suppose the system encounters a problem for which no solution yet exists.

It begins with a learning space.

The system does not initially know which pieces of available knowledge belong to the solution.

The discovery methodology exposes constraints and relationships that were not previously available to the problem.

Those constraints progressively exclude what cannot belong.

C(S) operates over what is available.

As constraints accumulate, the admissible space contracts.

Eventually a collection survives.

It constitutes a solution.

The expensive part has been done.

There is no reason to solve the same problem from scratch every time it appears.

Retain it.

THE TAG

A tag simple.

It is two things.

It binds l the {C}.

And it gives it a referent.

A retained, stabilized specification is a tag.

It preserves the constraint structure sufficient to recover or recognize a solution produced by C(S), so the system does not have to rediscover that solution every time it is needed.

This gives us a lineage of identities.

Animal.

Dog.

Labradoodle.

Molly.

Each may be a stable tag at a different degree of resolution.

The system does not necessarily have to resolve all the way to Molly.

It resolves only as far as necessary.

Criticality determines resolution.

Consider language.

“Watch out for that dog!”

Dog is sufficient.

“Watch out for that Labradoodle!”

Usually, Labradoodle contributes nothing useful.

But ask:

“What breed is that dog?”

Now Labradoodle is critical.

The available identity did not necessarily change.

The required resolution did.

This gives us a general principle:

Criticality determines the specificity at which a tag must resolve.

Language merely makes the principle particularly easy to see.

RECOGNITION

This is where my earlier architecture became unnecessarily complicated.

I knew from the beginning what function I needed.

I needed the difference between reality and what was already understood.

The delta.

My mistake was in how I thought that delta had to be obtained.

At one point I imagined a prediction.

Later I imagined a projector.

Then I replaced the projector with a generator.

The names changed, but the architecture remained fundamentally expressive.

The system possessed compressed knowledge.

It used that knowledge to construct an expectation.

Reality supplied another representation.

Pop

Then an automated process compared the two.

Conceptually, I had built two screens.

One displayed what the system expected.

The other displayed what reality supplied.

Then something mechanical compared the screens and reported the difference.

There was nothing inherently wrong with that.

It was simply unnecessary.

You do not need the screens.

THE SEMANTIC FILTER

A retained tag already contains what the system understands.

Why express that knowledge as a simulation merely to compare the simulation with reality?

Turn the problem around.

Make the tag receptive.

Reality enters.

In the architecture I am proposing, the tag acts like a carve out filter.

What conforms to the retained solution is suppressed.

What does not conform survives.

Reality → tag → Δ

The comparison can be inherent in the structure.

There is no need to generate an expected example.

There is no simulated screen.

There is no second screen representing reality solely for comparison.

There is no separate comparison between the two.

Instead, the retained knowledge is represented in a form capable of interacting directly with reality.

What it already understands disappears into the carve out.

What remains is information.

OBJECT RECOGNITION

This gives us an unusually simple way to think about vision.

Suppose the realm is geometry.

Its lingo contains geometrical relationships.

Relative position.

Connectivity.

Containment.

Orientation.

Proportion.

Topology.

And whatever other relationships prove necessary.

A learned object is represented by a stabilized collection of these relationships.

That stabilized specification is a tag.

Now present visual reality to it.

The tag need not contain a picture of a dog.

It need not generate a picture of a dog.

It need not rotate a simulated dog and compare that simulation against an image.

Its geometric constitution already expresses what matters about the identity.

As sensory information flows through the available constraint structure, stable identities can emerge at different resolutions:

Animal.

Dog.

Labradoodle.

Molly.

Criticality determines how much resolution is required.

Object recognition is therefore not necessarily a separate cognitive mechanism.

It may simply be C(S) operating within a geometric realm.

THE DISCOVERY METHODOLOGY

Now we can return to the thing I originally thought was C(S).

It is not.

C(S) operates once I have something to constrain.

Discovery is how I find out what the constraints are.

That sounds like a small distinction.

It isn’t.

I do not begin with a hypothesis and attempt to prove it.

Nor do I necessarily begin with a destination.

I enter a problem because something about it does not fit.

An anomaly.

Reality has done something my current understanding cannot accommodate.

There is a discrepancy.

I do not know the answer.

More importantly, I do not yet know the shape of the answer.

So I examine the problem.

What must be true?

What cannot be true?

What principles already seem reliable?

What happens if I follow one of them?

Usually that exposes another problem.

So I follow that one.

And another.

I am not trying to reach a predetermined answer.

I am trying to discover the nature of the problem.

As that nature becomes visible, constraints emerge.

Then C(S) can do its wonderfully stupid job.

Eliminate.

Eliminate.

Eliminate.

See what survives.

Failure is particularly useful.

Once something is known to fail, I can ask what property made it unacceptable.

That property may expose a constraint.

The admissible space contracts.

Eventually the local exploration may stall.

I have learned everything I currently know how to learn from where I am standing, but the problem remains insufficiently specified.

This is where my thinking can look like wandering.

It is not.

This distinction took me an embarrassingly long time to see because discovery and C(S) occur together.

Discovery exposes constraints.

C(S) exploits them.

Its failures expose anomalies.

Those anomalies provoke more discovery.

One expands what can be known.

The other contracts what can survive.

ASSOCIATIVE THINKING

When a question within one realm is difficult to resolve, I look for another realm in which the same structural problem is easier to observe.

A fundamental principle provides the bridge.

I follow it.

This is not merely saying that two subjects look alike.

I am looking for another expression of the same underlying relationship.

Most knowledge cannot make this journey. It remains expressed in the lingo of its realm.

But a sufficiently general principle can.

The second realm becomes a model.

I can inspect the behavior there because its lingo may make something visible that was obscure in the original problem.

That is precisely what happened when I needed to understand how far identity recognition should proceed.

I moved into language.

Language gave me:

Animal.

Dog.

Labradoodle.

Molly.

She.

Then I changed the message and watched the required specificity change.

From that model emerged:

Criticality determines resolution.

Once I had the principle, I no longer needed language.

I carried the principle home.

The discovery methodology therefore looks something like this:

Unresolved problem

→ Apply what is already known

→ Converge

→ Encounter anomaly or stall

→ Follow a fundamental principle

→ Enter another realm

→ Use that realm as a model

→ Inspect its behavior

→ Discover a new principle or constraint

→ Return

→ Converge again

The excursion expands what is available to the problem.

The new constraint contracts what can survive.

I am not leaving the problem.

I have temporarily exhausted what can be learned from where I am standing.

And sometimes I do not return with merely another constraint.

Sometimes I return with a fundamental principle that changes the problem itself.

DISCOVERING DISCOVERY

Ironically, I discovered this distinction by using the methodology itself.

I knew I was doing something I called C(S), but I could not separate the algorithm from the way I explored problems.

They were woven together.

So I stopped trying to reason about the distinction directly.

I took The Invention of Color, a paper in which I had already followed one of these trails, and treated it as evidence.

What did I actually do?

Where did the problem change?

Why did I jump from one subject to another?

What did I bring back?

Working through that trace exposed two different mechanisms that I had mistaken for one.

C(S) was the eliminator.

Discovery was the wanderer.

One carved the marble.

The other kept discovering new places to quarry stone.

THE INVENTION OF COLOR

The Invention of Color preserves a trace of this process.

I began with light and reflected wavelength.

That explained how coloration was physically possible, but it did not explain the riot of color on a living Earth.

Mars has sunlight.

Mars has matter.

Mars has absorption and reflection.

Yet Mars is comparatively drab.

Reflection alone was insufficient.

So I followed sight into the larger realm of sensing.

There I compared sight, hearing, smell, taste, and touch.

Different physical implementations made failure easier to see.

Excessive background noise.

Distortion.

Attenuation.

Those were useful negatives.

A useful sensory channel cannot tolerate too much noise, distortion, or attenuation.

I had acquired constraints that were not apparent while looking only at color.

Later I moved from human vision to bee vision.

Again, color supplied the bridge.

But difference supplied the information.

Flowers contain ultraviolet structures humans cannot see.

Therefore the model could not define color by human perception.

Expression had to be understood relative to the sensory capabilities of its receiver.

Another constraint.

Another convergence.

What appears on the page to be wandering is a record of discovery.

THE NULL SET

C(S) does not promise that something will survive.

Sometimes the result is the null set.

That matters.

The null set does not explain what went wrong.

It says only:

The present constraints and available knowledge cannot produce an admissible solution.

Perhaps a constraint is wrong.

Perhaps something is missing.

Perhaps an accepted fact is wrong.

Perhaps this realm cannot express what is needed.

Perhaps something genuinely new has been encountered.

I don’t know.

The null set does not answer the question.

It creates an anomaly.

And the anomaly gives discovery somewhere to begin again.

This is where the two processes meet beautifully.

C(S) can fail without knowing why.

It does not need to know why.

Its failure becomes information.

The null set hands the problem back to discovery.

WHAT REMAINS

This architecture is smaller than the one I began with.

There are no projectors.

There are no generators.

There is no requirement to manufacture an internal simulation merely to compare it with reality.

There are realms.

Within realms is knowledge expressed in compatible lingo.

Most of that knowledge remains bound to its realm.

Some relationships mature into fundamental principles capable of spanning realms.

Constraints operate upon what is available.

C(S) eliminates what cannot survive.

A stabilized specification can be retained as a tag.

A tag can subsequently act receptively, like a carve out filter, allowing reality itself to expose discrepancy.

Criticality determines how specifically something needs to resolve.

And when the existing structure fails, discovery begins again.

That distinction is important enough to say once more.

Discovery does not solve the problem.

It discovers its nature.

It exposes principles.

It finds constraints.

It follows anomalies.

It can cross into another realm when the present one has nothing more to say.

C(S) does something much simpler.

It takes whatever has been made available and eliminates what cannot survive.

One explores.

One prunes.

Together they can produce something that, viewed afterward, looks almost inevitable.

It wasn’t.

C(S) does not know the truth.

It does not even know what a solution means.

It only knows what cannot survive.

And when the pruning is done, what remains is not necessarily an identity.

It is a specification.

A collection of constraints sufficient to distinguish whatever has survived.

The constraint set:

{C}

Identity may emerge from {C}.

A word may emerge from {C}.

A solution may emerge from {C}.

C(S) does not care what it is.

It only knows what it isn’t.