4 posts tagged with "harness-engineering"

We recently measured two implementations of the same application request over the MusicBrainz dataset:

Load the newest 100 recordings that have linked works, and load at most ten work relations for each recording.

The controlled SQLx test returned the same 100 recordings, 103 relation rows, 103 links, 103 link types, and Work-ID checksum through both paths. One took 5,871.169 milliseconds. The other took 2.469 milliseconds—a 2,378× difference inside SQLx itself.

TeaQL Rust previously completed the corresponding typed graph workload in 2.864 milliseconds. That does not mean TeaQL has a PostgreSQL driver 2,000× faster than SQLx. The difference was the amount of work requested from the database. The expert SQLx control proves it.

A foundation model can know a great deal about programming, finance, manufacturing, medicine, and law. It still cannot know the private operating knowledge of a particular organization: an internal approval rule, a proprietary engineering method, an undocumented exception, or the reason one production procedure is trusted while another is prohibited.

This gap is often described as a knowledge-access problem. Give the model the right documents, retrieve the right passages, and it will know what to do.

Retrieval is necessary, but it is not sufficient.

The real enterprise question is not only:

Can the AI find the knowledge?

It is:

Can the AI turn that knowledge into a correct, verifiable, and constrained capability?

Our work with TeaQL suggests a three-layer answer: Skill, Build Harness, and Runtime Harness. A Skill keeps the agent's direction and method sound. A Build Harness makes the constructed artifact mechanically verifiable. A Runtime Harness keeps actual execution inside engineering, security, permission, resource, and audit boundaries.

Together, these layers provide a practical way to give AI capabilities that do not exist inside the model itself.

As AI agents become more capable, it is tempting to make their Skills increasingly detailed. Every new failure becomes another instruction. Every exception becomes another paragraph. Eventually, the Skill tries to describe the domain, encode the workflow, enforce safety, teach the tool interface, and anticipate every future mistake.

That approach does not scale.

Our practical conclusion is different:

Keep Skills lightweight. Make Tools strong.

This is not an argument for fewer safeguards. It is an argument about where safeguards belong.

It is also a principle for harness engineering. The core of an agent harness should not be an ever-growing library of Skills. It should be a set of strong, model-aware Tools and deterministic feedback loops that turn guidance into enforceable action boundaries.

TeaQL Java did not remove every use of reflection from the repository. We did something more practical: we removed reflection from the core entity execution path, isolated the remaining reflective utilities, and added build-time guardrails to stop reflection from quietly returning.

GraalVM Native Image was an important forcing function, but it was not the highest-level reason for the work. The larger goal was to make the runtime environment more deterministic: important behavior should be explicit, inspectable, bounded, and enforceable before a production request reaches it.

We see that as harness engineering. The reliability of a system should not depend only on developers or coding agents remembering the right conventions. The surrounding environment—generated code, metadata, module boundaries, compiler-visible calls, build rules, and tests—should constrain execution into known-good paths.