Tenstorrent Patterns for Analog CIM

Coverage boundary

This page turns architecture comparison into work you can perform

The site already explains Tenstorrent, NPU compiler foundations, SoC software, Analog CIM physics, accuracy evidence, and device bring-up. This guide connects those lessons through diagrams, implementation choices, exercises, artifacts, and self-checks.

Tenstorrent architecture Grid, NoC, tiles, circular buffers, kernels, TT-NN, TT-Metalium, and TT-Forge already exist. Review existing Tenstorrent study

Analog CIM fundamentals Array physics, converter boundaries, quantization, calibration, and mixed-signal limits already exist. Review Analog CIM architecture

Production NPU stack Artifacts, runtime, driver, firmware, profiling, debugging, and CI contracts already exist. Review SoC software architecture

New: architecture translation Convert each Tenstorrent pattern into an explicit ACiM compiler, runtime, or device concept. Open the translation map

New: performance dataflow Decide where buffering, prefetch, dispatch, caching, multicast, tracing, and concurrency still help. Follow the optimized ACiM flow

New: choice guide Compare framework, compiler, simulator, operator-placement, and device-interface paths. See the choices and recommendations

New: applied laboratories Produce an IR, target descriptor, lowering, artifact, fake-device runtime, and correlation report. Follow the six implementation labs

Learning method

Study a pattern, expose its assumptions, then rebuild it for ACiM

The question to ask at every layer

Which concept is a general accelerator-software pattern, and which part exists only because Tenstorrent executes digital dataflow kernels on Tensix cores? Write both answers before designing the ACiM equivalent.

1. TraceObserve the working pathFollow one tensor, IR operation, command, buffer, and trace ID.

2. ExplainName the invariantState the general compiler or runtime responsibility in hardware-neutral language.

3. ContrastExpose target assumptionsMark every Tensix core, tile, NoC, circular-buffer, and device-ABI dependency.

4. TranslateDefine the ACiM contractSubstitute arrays, converters, calibration, programming, digital fallback, and device state.

5. VerifyBuild and correlateRun the same vectors through reference, simulator, runtime, and hardware levels.

High-value code-reading targets: TT-MLIR's dialect boundaries, TT-UMD's SoC descriptor, MeshDevice's virtual topology, tt-emule's alternate backend, TT-Fabric's control/data planes, and Tenstorrent's trace/report tooling. Each becomes a concrete exercise in the transfer map below.

Step A | Target architecture worksheet

Learn which hardware facts create software requirements

IHW's public patent record reveals part of the numerical architecture: positional digit-sliced weights, adaptive converter range, grouped quantization, and signed-current cancellation. It does not show that every pending mechanism is implemented in a product, or disclose the complete array, command, calibration, compiler, firmware, and runtime contract. Use the checklist to construct a target descriptor and to mark each field as measured, specified, assumed for simulation, or still unknown. This section inventories the inputs; the discovery workbook later converts them into implementation rules and tests.

Why this comes first: every item below changes an IR type, legalization rule, mapping cost, artifact field, runtime state, or validation test. The iHW patent deep read is one example of extracting possible software-visible mechanisms from hardware evidence.

0%

0 / 12 complete

Learning output: a versioned target descriptor whose required fields are explicit. A simulator may carry clearly labeled assumptions; a production compiler should diagnose any unresolved field needed for correct lowering.

Tenstorrent learning map

Turn each code-reading target into an ACiM implementation exercise

Read a narrow path, identify the architectural pattern, build its ACiM equivalent, and retain a concrete artifact that proves what you learned. Filter by the layer you want to study next.

Tenstorrent reference	Pattern to learn	ACiM exercise	Evidence you produce
TT-Forge ONNX and TT-XLA	Framework import, model diagnostics, capability reporting, and framework/device plugin boundaries.	Import one ONNX MatMul and one StableHLO dot, normalize both to the same target-neutral operation, and report unsupported semantics.	Importer tests, normalized IR snapshots, shape/dtype diagnostics, and an operator capability report.
TT-MLIR StableHLO to TTIR path	Progressive lowering, dialect ownership, conversion targets, rewrite patterns, and reproducible compiler pipelines.	Define `acim` operations and types, then legalize one high-level dot into an ACiM MVM plus explicit digital work.	ODS definitions, verifier tests, pass pipeline, before/after IR, and failure diagnostics.
TTIR, TTNN, and D2M	Separation between tensor meaning, physical layouts, sharding, memory spaces, and explicit data movement.	Represent logical tensors separately from array slices, ADC groups, local SRAM, and partial-sum layouts.	Layout schema, conversion-cost report, memory plan, and tests that reject impossible mappings.
TTKernel and TTMetal dialects	How a low-level IR mirrors the executable machine and exposes synchronization, memory, and engine actions.	Create low-level operations for program/verify weights, drive inputs, convert columns, accumulate slices, invoke DSP work, and signal completion.	Target command IR, binary encoding test vectors, and a graph-to-command debug map.
TT-Lang DSL	Kernel-authoring abstractions, static checking, specialization, and lowering from a concise programming model.	Sketch a small DSL for array programs and digital epilogues; lower one program through the same ACiM IR used by imported models.	Language examples, parser/type rules, generated IR, and equivalence tests against direct IR construction.
TT-Metalium host workflow	Program creation, buffer ownership, command queues, asynchronous events, tracing, and explicit synchronization.	Implement `load -> bind -> program -> calibrate -> submit -> wait` behind an ACiM HAL.	C API, C++ wrapper, fake-device transcript, timeout test, and lifecycle sequence diagram.
TT-UMD SoC descriptor	A machine-readable topology contract separating logical coordinates and usable resources from physical discovery.	Describe arrays, rows, columns, ADC groups, SRAM banks, digital engines, defects, and calibration classes.	Versioned target-descriptor schema, sample devices, validation tool, and logical-to-physical maps.
TT-UMD transport boundary	Separation between device-neutral objects and hardware-specific discovery, MMIO, DMA, firmware messages, and telemetry.	Make simulator, queued fake device, VFIO lab backend, and production driver backend implement one narrow HAL.	Backend conformance suite and identical application traces across at least two backends.
MeshDevice and cluster descriptors	Virtual topology, explicit distribution, static test topologies, collectives, and logical/physical remapping.	Model one card and one two-card ACiM mesh, then execute row-parallel and column-parallel linear layers.	Topology files, shard maps, collective byte counts, and deterministic degraded-topology tests.
TT-Fabric	Persistent control/data planes, route epochs, fault detection, rerouting, checkpoint restart, and fault injection.	Specify ACiM route ownership and recovery for a failed link, tile, array region, or chiplet.	State machine, recovery sequence, fault-injection test, and before/after route graph.
`tt-emule`	A same-host-API alternate backend that allows software tests before target hardware is available.	Run one artifact through functional, nonideality, queued, and hardware backends without changing the application API.	Differential results, cycle/trace correlation, declared model fidelity, and backend-specific error bounds.
TT-KMD lifecycle	PCIe probe/remove, DMA isolation, buffer mapping, interrupts, reset, firmware loading, power, and process cleanup.	Trace those responsibilities, then map each one to the actual ACiM BAR, queue, firmware, and fault contracts described in the bring-up guide.	Responsibility matrix, UAPI draft, reset state machine, and DMA/interrupt test plan.
TT-NN Visualizer and performance reports	How graph operations, buffers, memory, traces, and bottlenecks remain connected by stable identifiers.	Display graph-to-array placement, ADC use, defect remaps, calibration version, drift state, and first divergent layer.	Trace schema, one generated report, and a debug path from wrong output back to physical mapping.
Model corpus and CI patterns	Golden models, operator sweeps, retained artifacts, differential tests, and regression triage.	Build a small suite containing MatMul, Conv, YOLO subgraphs, transformer linear layers, and unsupported-op fallbacks.	Repeatable test manifest, reference outputs, accuracy thresholds, performance baselines, and failure triage report.

Architecture mismatch explorer

A digital dataflow processor and an analog array expose different contracts

Different primitive

Core-grid kernels versus physical array mappings

TenstorrentPrograms reader, compute, and writer kernels across Tensix cores using tiles, circular buffers, and NoC traffic.

Analog CIMPrograms conductance cells, encodes inputs, performs array MVM, digitizes columns, and combines slices or partial sums.

Compiler consequenceReplace core placement with physical array tiling, signed mapping, converter assignment, and analog/digital partitioning.

ProofOne matrix must map to arrays and produce an explainable artifact with no hidden Tensix assumptions.

TT-Metal performance patterns translated

Keep the dataflow machinery, but move it around resident analog weights

TT-Metal optimizes a digital machine whose operands move through DRAM, local SRAM, circular buffers, a NoC, and compute engines. ACiM removes most steady-state weight traffic when conductances remain programmed, but it still has commands, activations, converter traffic, partial sums, digital epilogues, and outputs to move. The useful abstraction is therefore not "no data movement"; it is "weight-stationary dataflow with explicit movement and synchronization around the array."

KeepDouble buffering

Ping-pong activation ingress and digitized-result egress so DMA and digital work can overlap array execution.

AdaptPrefetch

Prefetch commands and activations every run. Stage weights only before planned program/verify work; do not page conductances per token.

KeepFast dispatch

Let a device controller consume queued descriptors without waiting for one host call per layer or array firing.

ExtendProgram cache

Cache schedules and digital kernels, but bind every hit to the target, weight map, calibration, health, and ABI epochs.

CB: bounded producer-consumer storage CQ: ordered device work stream DMA: transfer engine without CPU copying NoC: on-chip packet network DAC: optional multi-level input converter ADC: sensed analog result to digital code MVM: array compute primitive

Recommended ACiM execution flowControl, persistent physical state, and per-inference tensor traffic remain separate.

Control

Host SDKBind buffers and enqueue

CQ + eventsDependencies and fences

Device dispatcherValidate and issue

Resident programTrace or command template

Array schedulerConverters, power, hazards

State

Weight packageLogical tensors and scales

Program + verifyRare load or remap path

Resident weight handleArrays and conductance map

Bind device stateCalibration, defects, health

Validated bindingWeight, profile, and health epochs

Data

Input sourceHost, camera, or LPDDR

Ingress CB A/BActivation prefetch

Input encode / driveBits, pulses, or DAC levels

Array MVMResident weights

ADC / rebuildDigitize and combine slices

Egress CB A/BResult ping-pong

Digital epilogueBias, scale, activation, reduce

Output DMANext layer or host

The bottleneck moves; it does not disappear

In steady state, an ACiM layer should not stream its full weight matrix from DRAM. The likely critical path is activation delivery, input encoding, row/word-line drive, optional DAC conversion, array settle time, ADC conversion, bit-slice reconstruction, digital reduction, and output movement. Once the pipeline is filled, its ideal tile interval is approximately max(Tin, Tfront_end + Tmvm + Tadc, Tepilogue, Tout), but only stages backed by independent hardware can overlap. Shared converters, power rails, thermal limits, or SRAM banks must be modeled as one serialized resource even when software exposes several queues.

Ping-pong schedule

Double buffering is still useful on both sides of the array

The diagram shows a legal overlap only when input DMA, analog execution, digital epilogue, and output DMA use independent resources.

Three activation tiles moving through two ingress and two egress slots

Enginet0t1t2t3t4t5

Input DMAA0A1A2

Encode/drive + MVM + ADCA0A1A2

Digital epilogueA0A1A2

Output DMAA0A1A2

Storage	Producer to consumer	Initial policy	ACiM-specific rule
Command ring	Host runtime to device dispatcher	Enough descriptors to keep the device ahead of the host; use sequence numbers and completion fences.	Descriptors reference immutable weight handles and explicit calibration/profile epochs.
Ingress CB	DMA or sensor to encoder/DAC	Two activation macro-tiles, banked as ping and pong.	Layout must match row drive, bit-plane order, scale group, and multicast destinations.
Weight residency table	Programmer to array scheduler	Metadata and handles, not a per-inference copy of the weight bytes.	Invalidate on reprogram, remap, failed verify, incompatible calibration, or device replacement.
Egress CB	ADC/reconstruction to digital epilogue	Two result blocks; use more only after a measured consumer-jitter problem.	Entries carry ADC mode, saturation flags, slice count, and uncertainty or retry status.
Partial-sum scratch	Array slices or chips to reduction	Keep local until all slices are combined; avoid round-tripping through external memory.	Accumulator width and reduction order are part of the numerical contract.
Calibration cache	Calibration service to array and digital stages	Pin the active profile and publish an immutable epoch during execution.	A profile change drains dependent queues before replacing scales, ranges, or trims.
Output ring	Digital epilogue to next layer, CPU, or host	Use events or credits so a producer never overwrites unread output.	Prefer direct layer-to-layer routing; host readback is a graph boundary, not a default after every MVM.

Cache lifetime model

Reuse three different objects without confusing their validity

1. Kernel binary cache Compiler and ISA lifetime Stores DSP/vector kernels and firmware fragments. Key by source or IR, compiler flags/version, ISA, target revision, and firmware ABI. Invalidated by code generation or binary compatibility changes.

2. Program template cache Structural schedule lifetime Stores placement, CB plan, engine sequence, synchronization, and parameter slots. Mutable addresses and state handles remain runtime arguments. Invalidated by shape, layout, precision, topology, mapping, resource, or ABI changes.

3. Bound trace / residency cache Device-state lifetime Stores replayable descriptors bound to concrete buffers, programmed weights, calibration, health, and stable dependencies on one device. Invalidated by address movement, reprogram/remap, profile update, drift/health epoch, reset, or replacement device.

Technique-by-technique decision

What survives the architecture change

"Keep" means the mechanism transfers directly. "Adapt" preserves the pattern with a different cache key, tile, or producer. "Conditional" requires measured independence or a specific workload. "Replace" rejects the literal Tensix assumption.

TT-Metal technique	Decision	ACiM implementation	Evidence before enabling it
Reader / compute / writer pipeline	Adapt	Use activation-ingest, array-sequencer, and result-drain roles. Keep explicit producer/consumer ownership, but do not require a Tensix-style RISC-V core for every role.	Timeline shows each role executing independently and queue starvation has an attributable owner.
Two-page CB / double buffering	Keep	Start with two activation slots and two ADC-result slots. Alternate ownership with credits or semaphores while the array works on the other slot.	Lower ingress/egress stall cycles without violating SRAM, power, thermal, or numerical limits.
Deeper circular buffers	Conditional	Add a third or deeper slot only for bursty DMA, variable ADC retries, multicast skew, or digital-consumer jitter.	Queue-depth histogram reaches two and deeper storage improves end-to-end throughput enough to justify SRAM cost.
Async DMA and retained transfer state	Keep	Preprogram common source, destination, stride, page, route, and format state, then issue compact descriptors for repeated activation blocks and outputs.	Descriptor and firmware overhead falls; explicit barriers still prove descriptor, payload, and completion ordering.
DRAM prefetcher + global CB	Adapt	Use a persistent producer for activation streams. Weight prefetch means staging bytes before rare program/verify work; frequently used layers or experts should occupy separate resident arrays or chiplets.	The plan includes program/verify latency, endurance, and eviction cost. It never assumes that changing conductance is as cheap as filling SRAM.
Fast dispatch	Keep	Place descriptor validation, dependency tracking, and engine issue in device firmware. The host submits graphs or batches, not one blocking command per layer.	`queue_empty_cycles` approach zero and host scheduling gaps disappear from the device timeline.
Program cache	Two-level reuse	Cache structural command templates, placements, buffer plans, and digital kernels under an exact canonical key. Bind mutable addresses, weight handles, and profile handles as checked runtime arguments.	The structural key includes op, shape, layout, precision, target revision, ISA/firmware/ABI, macro-tile, converter topology, mapping version, and power mode. Submission separately rejects stale residency, calibration, or health epochs.
Kernel compilation cache	Keep narrowly	Reuse binaries for DSP/vector kernels, device firmware fragments, and format converters. Keep this distinct from physical weight and calibration state.	A cache clear and cold compile are testable; binary compatibility is tied to compiler, ISA, and firmware versions.
Metal Trace-style replay	High value	Capture a static graph's already-validated descriptors on device and replay them against stable buffers, resident weights, and profile bindings. Build trace families for bounded dynamic shapes.	Commands, addresses, weight handles, calibration/health epochs, and dependencies still match. A changed binding invalidates or patches the trace through a defined validation path.
Multiple CQs + events	Conditional	Use one queue for input/output transfers and another for execution, or independent queues for resource-disjoint array groups. Synchronize with device-visible events.	The queues use independent DMA/array/converter resources; concurrent execution improves throughput instead of overloading shared ADCs, NoC, or power delivery.
Tensor sharding	Essential	Shard output channels, input rows, bit slices, experts, or batches across physical arrays. Placement must include converter groups, spare rows/columns, and calibrated quality.	Mapping report predicts utilization, cross-shard reductions, converter occupancy, defects, communication bytes, and error impact.
Multicast + semaphores	Keep	Broadcast one activation block to several arrays that hold different output-channel weights. Use receiver-ready credits before overwriting the source slot.	One injected copy replaces many unicasts, receivers do not starve, and skew does not force excessive buffering.
Block reuse + intermediate CB	Keep	Retain activation blocks, scales, and partial sums near the arrays across output blocks or slices. Choose an array-native macro-tile rather than the literal 32x32 TT tile.	External and NoC bytes fall, local capacity remains legal, and the numerical reduction order remains reproducible.
Fused bias and activation	Keep	Fuse scale, offset, bias, clamp, quantize, and supported activation into the post-ADC digital path before storing the tensor.	Fusion removes a materialized tensor and preserves the reference result within the declared tolerance.
SubDevices + persistent programs	Conditional	Partition array groups, SRAM, queues, and digital engines into independently scheduled regions; reserve one region for a persistent sensor or prefetch pipeline.	The target descriptor proves that converters, routes, calibration state, power, reset, and fault domains are separable. Shared resources remain globally arbitrated.
NoC ordering and barriers	Keep	Define ordering for payloads, descriptors, doorbells, multicast, and completion writes. Batch barriers where legal, but never infer cross-route ordering.	Litmus and stress tests cover wraparound, multiple producers, reset, timeout, and weakly ordered transport behavior.
TT 32x32 tile and core coordinates	Replace literal form	Expose an ACiM macro-tile derived from physical rows, columns, bit slices, ADC grouping, local SRAM, and digital reduction width.	The tile is generated from a versioned target descriptor and changes legally across device revisions.

Runtime decision graph

Choose an optimization from observable hardware facts

1Are the required weights programmed and valid?Yes - use the resident handle; do not prefetch weights. No - enqueue program/verify or page replacement before execution.

2Can DMA overlap converter and array use?Yes - allocate ingress ping-pong buffers. No - keep the queue contract, but do not claim hidden latency.

3Do arrays share ADCs, input drivers/DACs, SRAM banks, routes, or power?Yes - model tokens for each shared resource and cap concurrency. No - schedule independent array groups in parallel.

4Are shape, addresses, weights, and profile epochs stable?Yes - replay a resident trace. No - use a cached parameterized program template or compile a new variant.

5Does one activation feed several resident weight shards?Yes - multicast once and synchronize receivers. No - use local or unicast delivery.

Software ownership

Organize the implementation around the optimized flow

compiler/schedule/Pipeline and resource plannerChooses macro-tiles, CB depth, multicast groups, converter tokens, fusion, and overlap.

runtime/cache/Three cache lifetimesSeparates kernel binaries, structural program templates, and device-bound traces or residency bindings.

runtime/queue/CQs, events, and trace replayOwns enqueue, dependency, capture, replay, completion, timeout, and cancellation semantics.

runtime/residency/Weight and profile handlesTracks what is programmed, verified, calibrated, pinned, stale, remapped, or evictable.

firmware/dispatch/Command prefetch and validationConsumes rings, validates epochs and ranges, resolves handles, and feeds the scheduler.

firmware/dataflow/Ingress, array, and egress stagesImplements DMA, CB credits, input encoding, array firing, ADC drain, reconstruction, and barriers.

firmware/resources/Physical arbitrationOwns ADC/DAC groups, SRAM banks, routes, power envelopes, thermal throttling, and reset domains.

tools/profiler/One correlated timelineJoins graph, command, buffer, array, converter, DMA, digital, calibration, power, and accuracy events.

Order	Implement	Required counters and tests	Acceptance condition
P0	Resident weight handles, one CQ, device dispatch, ingress/egress ping-pong CBs, async DMA, events, local post-ADC epilogue.	Queue-empty cycles, every engine's busy/stall cycles, CB occupancy, bytes, saturation/retry flags, accuracy delta, timeout/reset tests.	One static MVM stream runs continuously, matches the reference tolerance, and attributes every idle interval.
P1	Program cache, trace replay, activation multicast, array sharding, partial-sum locality, fusion, and two-queue I/O overlap.	Cache hit/miss reason, trace replay count, multicast fanout/bytes, reduction bytes, queue event latency, power and thermal counters.	Each feature improves end-to-end latency, throughput, or energy on a named workload without changing accepted accuracy.
P2	Global streaming CB, persistent prefetch program, deeper buffers, subdevice concurrency, weight paging, and multi-chip scheduling.	Prefetch usefulness, eviction/program time, cross-partition contention, fairness, tail latency, fault injection, recovery, and profile-epoch transitions.	Measured workload demand justifies the added state and concurrency; recovery remains deterministic.

Optimization rule: never approve a technique from array utilization alone. Record end-to-end latency, throughput, joules per inference or token, task accuracy, converter saturation/retries, temperature, and tail behavior. A schedule that drives every array at once can be slower or less accurate when it overloads shared ADCs, the NoC, the power network, or the digital reconstruction path.

Recommended architecture

Keep the portable graph stack above an explicit ACiM boundary

Tenstorrent can remain a comparison target or reference implementation without becoming the core dependency of the analog backend.

ImportPyTorch / ONNX / StableHLOPreserve source semantics and shape constraints

NormalizeTarget-neutral graph IRCanonical ops, quantization intent, fallback policy

DecideCapability and partitionAnalog, digital companion, CPU, or rejection

LowerACiM IRArrays, slices, converters, calibration, digital ops

PlanPhysical mappingPlacement, programming, memory, routes, schedule

PackageVersioned artifactCode, weights, maps, calibration schema, debug data

ExecuteACiM HAL and runtimeSimulator and device backends behind one contract

Design rule: the target-neutral IR should describe model meaning, while ACiM IR should describe physical analog intent. Neither layer should pretend that a conductance array is a Tensix core.

Analog-specific work

The custom backend begins where physical state changes correctness

Implementation companion: this page decides what to learn, adapt, or replace from Tenstorrent. The SDK and compiler guide turns that decision into concrete libraries, APIs, schemas, compiler passes, simulator backends, tests, and a first runnable slice.

Mapping

Array tiling and decomposition

Split matrices by physical rows, columns, differential representation, bit slices, ADC lanes, and available arrays. Define partial-sum combination and padding.

Evidence: mapping report includes utilization, slices, conversions, predicted error, and digital work.

Numerics

Range and converter planning

Select scales, input encoding, accumulation range, ADC mode, saturation policy, and per-layer tolerance using calibration and workload distributions.

Evidence: reference versus simulated histograms identify the first divergence and its physical cause.

Partition

Analog and digital placement

Keep dense supported MVMs on arrays; assign nonlinear, control, normalization, unsupported, or precision-sensitive work to digital engines or CPU fallback.

Evidence: capability report states every fallback and conversion cost.

Programming

Conductance and fault-aware placement

Translate logical weights into programming targets, write/verify actions, remapped cells, spare resources, and per-chip state with endurance constraints.

Evidence: deterministic programming manifest and post-program verification report.

Calibration

Compensation insertion

Attach calibration versions, converter corrections, array gains/offsets, drift models, temperature ranges, and recalibration triggers to executable regions.

Evidence: stale or incompatible calibration is rejected or repaired explicitly.

Artifact

Device-aware packaging

Separate portable model content from device-bound programming and calibration state so builds, deployment, updates, rollback, and replacement remain auditable.

Evidence: schema validation and ABI compatibility tests cover every section.

Runtime lifecycle

Inference is only the steady-state middle

1DiscoverIdentify revision, arrays, faults, firmware, features

2ValidateCheck artifact, target, ABI, calibration, shapes, and required features

3ProgramWrite and verify weights under endurance policy

4CalibrateLoad or measure coefficients and health state

5ExecuteSchedule mixed analog/digital work and trace it

6MaintainMonitor drift, temperature, faults, updates, recovery

Accuracy and performance evidence

Use a correlation ladder, not a single simulator result

IBM AIHWKit and CrossSim are useful analog-behavior study tools. They are not substitutes for a target specification, circuit model, RTL, or measured silicon. Declare which effects and timing each model includes.

Level 0Framework FP32Semantic golden model and dataset baseline

Level 1Digital quantizedIsolate format, rounding, clipping, and scaling loss

Level 2Ideal arrayValidate decomposition, signed mapping, slices, ADC range

Level 3Nonideality modelVariation, noise, drift, parasitics, faults, converters

Level 4C++ / SystemCArtifact, runtime, queues, memory, cycles, trace, recovery

Level 5RTL / mixed signalProtocol, timing, firmware, circuit corners, co-simulation

Level 6Silicon populationTask SLA across chips, temperature, age, and recalibration

Evidence family	Required measurements	Failure question
Numerical	Task metric, first divergent layer, absolute/relative error, distribution tails, saturation, slice and ADC contribution.	Which declared mechanism explains the loss?
Performance	End-to-end and layer latency, throughput, array utilization, converter stalls, memory traffic, programming and calibration time.	Did reuse preserve the wrong cost model?
Energy and thermal	System and subsystem energy, ADC/DAC share, idle/control overhead, thermal steady state, DVFS or power-mode behavior.	Is the analog advantage visible at product boundaries?
Reliability	Chip distribution, defects, yield policy, drift versus time, retention, endurance, temperature corners, recovery and fallback rate.	Does the product SLA hold beyond a favorable sample?
Correlation	Same vectors, artifact hash, calibration version, device state, seeds, trace IDs, and uncertainty across every level.	Can another engineer reproduce and localize the difference?

Architecture choice flow

See the available paths, when each fits, and what to learn first

"Best" depends on the immediate learning question. The highlighted option is the recommended starting point for this study; the other choices remain useful at a later integration stage or for a different product goal.

1Model entryONNX, ORT plugin, or StableHLO/PJRT

2Compiler baseMLIR, TVM, or a minimal custom IR

3Execution modelFunctional, queued/cycle, or real hardware

4Operator homeAnalog array, DSP, or CPU

5Device accessSimulator HAL, VFIO, or UMD/KMD

Decision 1 | 3 choices

Which model entry should you implement first?

Choose the boundary that matches the models and framework behavior you need to observe.

Recommended firstDirect ONNX importerBest for learning compiler semantics with YOLO and common deployment models. It keeps the first debug path small: model, importer, IR, lowering, simulator.

Integration stageONNX Runtime plugin EPBest when applications already use ORT and the compiler/runtime API is stable enough to support subgraph capability, compilation, caching, and CPU fallback.

OpenXLA pathStableHLO plus PJRTBest when JAX, XLA, or StableHLO portability is a product requirement. PJRT provides a uniform device API, not the ACiM lowering itself.

Recommended sequence: direct ONNX import for the first runnable slice, then an ORT plugin EP; add StableHLO/PJRT when OpenXLA clients are an explicit requirement.

Decision 2 | 3 choices

Which compiler foundation should carry ACiM semantics?

Choose based on how many abstraction levels, custom types, and target legality rules the compiler must express.

Recommended product pathMLIR dialect stackBest for a real ACiM backend with target-neutral tensors, analog-aware operations, physical mapping, command IR, verifiers, and progressive legalization.

Exploration pathTVM backendUseful for schedule search, tensor-expression experiments, and Python-centered prototyping when the analog state and artifact model remain modest.

Smallest experimentCustom C++ or Python IRUseful for one operator and quick cost-model experiments. It becomes expensive when adding graph regions, diagnostics, serialization, rewrites, and multiple targets.

Recommended for this architecture: MLIR, because ACiM needs several explicit levels: model meaning, analog intent, physical placement, and executable commands. Use a minimal Python model alongside it as the numerical oracle.

Decision 3 | 3 levels

Which execution backend should you build now?

These are successive fidelity levels rather than competitors; each reveals a different class of bug.

Start hereFunctional and nonideality modelTests operator semantics, mapping, quantization, converter range, variation, and layerwise accuracy without queue or timing complexity.

Build secondQueued or cycle-oriented fake deviceTests artifacts, buffers, dependencies, command ordering, timing estimates, trace IDs, timeouts, and recovery while hardware is unavailable.

Correlate thirdFPGA or silicon backendTests actual transport, firmware, converters, calibration, PVT behavior, faults, performance, and energy.

Recommended flow: implement all three behind one HAL in this order. Skipping the functional model mixes numerical bugs with runtime bugs; skipping the queued model makes hardware bring-up unnecessarily opaque.

Decision 4 | 4 choices

Where should each graph operation execute?

The compiler should choose per operation and shape; no single engine is best for the whole model.

Recommended policyHybrid cost-based partitionEvaluate accuracy, conversion, transfer, memory, latency, and energy for every candidate placement, then keep every fallback visible in the artifact and report.

Dense computeACiM arrayBest for large, supported, weight-resident MVM or convolution regions whose mapping, converter range, and measured accuracy satisfy the target.

Digital companionDSP or vector engineBest for activation, normalization, reduction, softmax, partial-sum repair, small tensors, and precision-sensitive work near the arrays.

Host fallbackCPUBest for rare unsupported operations, dynamic control, preprocessing, postprocessing, and early bring-up; transfer and synchronization costs must be visible.

Recommended rule: place an operation on ACiM only when end-to-end cost includes conversions, partial sums, calibration, and error recovery. Otherwise choose the closest digital engine, with CPU as the explicit final fallback.

Decision 5 | 3 maturity stages

How should host software reach the device?

Match the access path to the hardware stage and isolation requirements.

Start hereSimulator HALNo kernel driver is needed. The runtime calls an in-process or service backend implementing the same buffer, submit, wait, trace, and reset contracts.

Trusted labVFIO prototypeUseful for a single-purpose PCIe lab path with IOMMU-protected userspace access while the production UAPI is still changing.

Production deviceUMD plus KMDBest for multi-process isolation, DMA ownership, interrupts, reset, power, firmware, security, cleanup, and stable deployment. An integrated SoC uses equivalent platform-driver responsibilities.

Recommended flow: stabilize the HAL with simulator backends, use VFIO only for controlled bring-up when appropriate, then implement the production UMD/KMD boundary from the measured device contract.

Applied implementation labs

Six connected builds that teach the complete software path

Each lab has a learning objective and a reviewable output. Complete them in order because every later lab consumes contracts and test vectors from the earlier ones.

Lab 1TraceTenstorrent model-to-device path

Lab 2ModelDigital and analog numerical ladder

Lab 3SpecifyTarget descriptor and ACiM IR

Lab 4LowerOne operator to physical mapping

Lab 5ExecuteArtifact, HAL, queues, and trace

Lab 6CorrelateAccuracy, latency, energy, and faults

0%

0 / 6 complete

Lab 1Trace one Tenstorrent operationFollow MatMul from StableHLO or ONNX through TT-MLIR dialects, layout decisions, runtime calls, command queues, buffers, and device completion. Annotate every target-specific concept.Output: stack diagram, IR snapshots, call trace, and a one-page explanation of each boundary. Lab 2Build the numerical reference ladderImplement FP32, INT8/INT4, ideal-array, and nonideality models. Sweep ADC range, slices, programming error, noise, drift, and clipping with identical test vectors.Output: reproducible notebook, layerwise error plots, and the first-divergence report. Lab 3Specify the ACiM machineCreate a target descriptor and MLIR ODS definitions for arrays, conductance formats, converters, SRAM, digital engines, calibration classes, defects, and command capabilities.Output: schemas, sample target, verifier tests, and an architecture-to-software field map. Lab 4Lower MatMul end to endImport MatMul, legalize it, tile rows and columns, select signed/bit-sliced mapping, assign ADCs, plan partial sums, insert digital epilogue work, and emit executable commands.Output: pass pipeline, mapping report, command listing, debug map, and simulator result. Lab 5Run the artifact through a fake devicePackage commands, constants, layouts, and calibration requirements. Implement sessions, buffers, submit/wait, trace, timeout, reset, and functional plus queued backends behind one HAL.Output: C API, artifact schema, backend conformance tests, trace viewer input, and recovery transcript. Lab 6Correlate and optimize the full pathCompare reference, nonideality, queued/cycle, RTL/FPGA, and silicon when available. Explain accuracy loss, converter stalls, memory traffic, programming time, energy, and fault behavior.Output: correlation dashboard, bottleneck report, model-fidelity statement, and prioritized optimization list.

Capstone: a runnable repository in which one ONNX MatMul compiles into an ACiM artifact, executes unchanged on numerical and queued backends, emits a layer-to-command trace, and reproduces the reference comparison. Use the accelerator repository blueprint for directory ownership, GoogleTest/pytest placement, CI gates, performance tests, contribution rules, and debugging artifacts.

Step B | Architecture discovery workbook

Convert every unknown hardware fact into a software task

This section consumes the Step A worksheet. For each answer, update a machine-readable contract, implement the affected rule, and add the test that proves the rule.

1. ObserveHardware factSpecification, measured data, simulator behavior, or an explicitly labeled assumption.

2. RecordTarget descriptorTyped field, units, range, version, source, confidence, and required/optional status.

3. CompileIR and mapping ruleVerifier, legality, cost, placement, precision, and fallback behavior.

4. ExecuteArtifact and runtime stateCommands, calibration binding, programming data, queues, telemetry, and recovery.

5. ProveCorrelation testGolden vectors, corners, trace IDs, expected errors, and acceptance threshold.

Fact to learn	Evidence to inspect or measure	Software you can implement after learning it
Compute primitive	Array dimensions, MVM direction, supported shapes, cell encoding, differential mapping, bit slices, and accumulation path.	ACiM op semantics, shape verifier, tiling rule, physical layout, utilization report, and partial-sum plan.
Converter path	Input encoding, DAC or bit-serial drive, ADC count/sharing, range modes, ENOB, sample rate, latency, and energy.	Range planner, converter allocator, saturation estimator, schedule resource, and performance/energy cost model.
Programming behavior	Pulse algorithm, write-verify, target tolerance, programming time, endurance, retention, update granularity, and verify readback.	Programming manifest, weight lifecycle state machine, cache policy, update/rollback flow, and endurance-aware placement.
Variation and defects	Cell/array distributions, unusable rows or columns, spare resources, remap rules, yield data, temperature, and aging.	Fault-map schema, logical/physical remapper, calibrated cost, spare allocation, degraded mode, and population tests.
Memory and interconnect	SRAM banks, external memory, NoC topology, multicast, DMA paths, bandwidth, alignment, coherency, and address spaces.	Buffer planner, tensor layouts, transfer operations, double buffering, route schedule, and bandwidth diagnostics.
Digital companion	DSP/vector/scalar ISA, supported nonlinear and reduction operators, local memory, synchronization, and conversion formats.	Capability partitioner, digital dialect, fusion rules, precision-repair path, and analog/digital boundary cost.
Command and device model	Descriptors or ISA, queues, dependencies, interrupts, preemption, timeout, reset, firmware ABI, and telemetry counters.	Command IR, artifact encoder, HAL, runtime queues/events, UMD/KMD contract, profiler, and recovery tests.
Calibration lifecycle	Factory, boot, thermal, periodic, per-model, and drift-triggered calibration; storage format and invalidation rules.	Calibration schema, compatibility check, runtime state owner, recalibration scheduler, stale-state error, and trace fields.
Existing software path	Current importer, simulator, compiler passes, runtime API, firmware, driver, profiler, debugger, tests, and example models.	Gap map showing what already runs, which contracts are implicit, and the smallest new vertical slice to implement.
First workload	Representative tensor shapes, operator mix, accuracy metric, batch, latency, throughput, power, and update requirements.	Benchmark manifest, compiler priorities, fallback policy, acceptance thresholds, and regression suite.

Study exercise: add status = specified, measured, assumed, or unknown to every target field. Attach its source and confidence, then make the compiler reject any unknown field required by the selected lowering.

Primary sources

Architecture, compiler, runtime, and simulation references

Use these sources to reproduce the learning exercises and distinguish documented software behavior from proposed ACiM translations.

Tenstorrent TT-ForgeCurrent compiler-stack organization, TT-XLA, TT-Forge-ONNX, TT-MLIR, TT-Lang, and model corpus

TT-MLIR overviewTTNN, D2M, and TTKernel dialect responsibilities and their mapping to Tenstorrent execution

TT-Metalium programming guideTensix cores, RISC-V processors, SRAM, tiles, NoC, kernels, circular buffers, and host workflow

TT-Metalium single-core MatMulReader/compute/writer pipeline and two-page circular buffers used for overlapped data movement and compute

TT-Metalium optimized multi-core MatMulBlock reuse, intermediate circular buffers, multicast, semaphores, and fused bias/activation patterns

TT-Metalium Device 2.0 data movementAsynchronous reads/writes, retained transfer state, multicast endpoints, circular buffers, semaphores, and barriers

Metal Trace and multiple command queuesOn-device command replay, static-address constraints, event synchronization, and I/O/compute overlap

TT-Metal program cache sourceDevice-tied cached programs, runtime-argument overrides, and exact canonical keys alongside 64-bit hashes

TT transformer DRAM prefetcher sourceAsynchronous weight prefetch into a global circular buffer using DRAM sharding, sender/receiver mappings, and subdevices

TT-Metal SubDevices reportResource partitioning, independent execution state, persistent programs, global semaphores, and cross-program circular buffers

TT-NN tensor and sharding referenceExplicit tensor layouts, DRAM/L1 placement, shard specifications, and core-local ownership

TT-NN documentationTenstorrent tensor library and device operation surface

StableHLO specificationHigh-level model operation semantics and the framework-to-compiler portability boundary

MLIR dialect conversionConversion targets, legality, rewrite patterns, type conversion, and partial/full analysis modes

MLIR ODSTableGen operation definitions, generated builders, verifiers, parsers, printers, and documentation

ONNX Runtime Execution ProvidersSubgraph capability, hardware-provider execution, common application API, and CPU fallback model

ONNX Runtime plugin EP guideCurrent plugin-based path for adding a new accelerator Execution Provider

OpenXLA PJRTUniform C device API and opaque device-specific plugin boundary for framework clients

IBM AIHWKitPyTorch-integrated analog hardware simulation, nonidealities, calibrated device models, and hardware-aware training

Sandia CrossSimAnalog inference simulation for programming error, drift, read noise, ADC precision, parasitics, slicing, and array size

Apache TVM architectureTarget-extensible compiler and runtime architecture reference

TT-UMD custom-SoC guideExplicitly reusable SoC descriptor contract plus the boundary for custom device implementation

TT-NN MeshDevice reportVirtual mesh, explicit tensor distribution, SPMD dispatch, and collective communication

TT-Fabric architectureData/control planes, rerouting, mesh repair, checkpoint recovery, modeling, and fault injection

Tenstorrent tt-emuleSame-host-API alternate backend for hardware-free tests; execution semantics remain Tensix-specific

Tenstorrent cluster descriptorsStatic single-host, multi-host, subset, and fallback topology descriptions

TT-NN VisualizerGraph, tensor, buffer, memory, operation, and performance-report tooling reference

Tenstorrent performance report toolTrace, signpost, operation-ID, throughput, and bottleneck-analysis patterns

Analog CIM accuracy evidence ledgerMeasured-silicon papers, claimed mitigations, remaining issues, and evidence boundaries

iHW patent-to-software studyClaimed ACiM mechanisms translated into possible compiler metadata, runtime state, and validation questions

ACiM SDK and compiler guideConcrete IR, compiler-pass, artifact, HAL, runtime, simulator, and first vertical-slice implementation

Accelerator repository blueprintTT-Metal-inspired source boundaries, implementation chain, test ladder, CI routing, contribution workflow, and debug playbook

ACiM PCIe board bring-upHow to adapt the KMD/UMD separation into target-specific BAR, DMA, interrupt, firmware, queue, reset, and YOLO implementation contracts

ACiM scale-out and LLM designFault hierarchy, logical mesh, compiler/runtime contracts, hybrid LLM partition, and implementation plan

Learning Tenstorrent Patterns for an Analog CIM NPU

This page turns architecture comparison into work you can perform

Study a pattern, expose its assumptions, then rebuild it for ACiM

The question to ask at every layer

Learn which hardware facts create software requirements

Turn each code-reading target into an ACiM implementation exercise

A digital dataflow processor and an analog array expose different contracts

Core-grid kernels versus physical array mappings

Quantization is only one part of analog numerical behavior

The executable depends on the physical device population

Programming and conversion join compute and data movement

Correctness is a population and lifecycle claim

Keep the dataflow machinery, but move it around resident analog weights

The bottleneck moves; it does not disappear

Double buffering is still useful on both sides of the array

Reuse three different objects without confusing their validity

What survives the architecture change

Choose an optimization from observable hardware facts

Organize the implementation around the optimized flow

Keep the portable graph stack above an explicit ACiM boundary

The custom backend begins where physical state changes correctness

Array tiling and decomposition

Range and converter planning

Analog and digital placement

Conductance and fault-aware placement

Compensation insertion

Device-aware packaging

Inference is only the steady-state middle

Use a correlation ladder, not a single simulator result

See the available paths, when each fits, and what to learn first

Which model entry should you implement first?

Which compiler foundation should carry ACiM semantics?

Which execution backend should you build now?

Where should each graph operation execute?

How should host software reach the device?

Six connected builds that teach the complete software path

Convert every unknown hardware fact into a software task

Architecture, compiler, runtime, and simulation references