Analog CIM Scale-Out and LLMs

Executive answer

Can a Mythic- or iHW-style Analog CIM system run an LLM?

Architecture conclusion Yes, as a hybrid inference system No public evidence yet establishes a legacy M1076 or an iHW product running a production decoder LLM end to end.

Put static dense projections and feed-forward weights on Analog CIM arrays. Keep dynamic attention, KV-cache writes, normalization, nonlinear functions, routing, sampling, and precision recovery on digital engines and SRAM/DRAM. Scale the model with explicit tensor, pipeline, and expert parallelism. Judge success in tokens/second, time-to-first-token, inter-token latency, accuracy, energy/token, capacity, and recovery time, not array TOPS alone.

Measured silicon

Transformer feasibility is real but bounded

A 2025 IBM-led study mapped 7.1 million unique ALBERT weights to a fabricated 14 nm PCM chip and reported average accuracy 1.8 percentage points below FP. Attention, activations, pooler, and classifier remained in software, so this is strong evidence for analog linear layers, not an end-to-end decoder LLM.

Read the Nature Communications paper

Circuit and model simulation

Dynamic attention needs different memory

A 2025 gain-cell proposal stores K/V values in write-friendly charge memory and computes both attention dot products in the analog domain. Results use 64 × 64 SPICE simulations and PyTorch model evaluation, not fabricated full-system silicon. It should be a separate target capability, not assumed for flash arrays.

Read the Nature Computational Science paper

Mythic evidence

Legacy product facts and current roadmap differ

The M1076 brief documents 80 million weight capacity, 76 tiles, 25 TOPS, and a PCIe interface. Mythic's 2025 announcement claims a rebuilt chiplet architecture for 1T+ LLMs and publishes internal efficiency projections; public independent silicon results for that roadmap are not yet identified here.

Open the M1076 brief

Hybrid confirmation

Mythic's acquisition reinforces this partition

Mythic describes analog compute for dense matrix operations and the acquired Videantis digital architecture for attention, control, programmability, and other high-performance workloads. That is the system boundary this design makes explicit.

Read the acquisition announcement

iHW boundary: the public material reviewed for this study supports analog inference positioning, patent mechanisms, and smaller-model demonstrations, but does not establish a public decoder-LLM silicon result. The architecture below is a target-neutral proposal, not a disclosure of iHW's internal implementation.

Physical execution

What actually happens during one Analog CIM matrix-vector operation

Mythic publicly identifies a flash array and analog circuits, but not every internal encoding, converter, or accumulation detail. The flow below is a generic differential or bit-sliced ACiM contract whose exact choices must be supplied by the target descriptor.

1. ProgramEncode fixed weightsMap quantized weight slices into conductance states, verify writes, and record achieved values.Wq -> G+, G-

2. DriveEncode activationsDACs, pulse widths, or switched levels convert an input slice into row stimuli.xq -> V or t

3. MultiplyCell currentEach enabled cell contributes current according to its programmed state and applied input.Iij ≈ Gij * Vi

4. AccumulateColumn integrationKirchhoff summation and integration form one partial dot product per active column.Ij = ∑i Iij

5. ConvertRead analog resultADC or charge-to-pulse circuitry digitizes the integrated result using a calibrated range.qj = ADC(Ij)

6. RebuildDigital correctionCombine signs, slices, arrays, and chips; apply scale, bias, saturation, and optional residual correction.y = requant(sum(q))

y_hat = D(ADC(sum_i A(G_ij, x_i, theta))))

The software-visible result is not only W × x. A includes device and circuit behavior, theta identifies calibration and environment state, and D is digital reconstruction. A compiler must preserve all three in the executable and debug map.

Decision	Hardware owns	Compiler owns	Runtime/firmware owns
Weight representation	Cell levels, differential support, spare rows/columns, programming limits	Quantization, clipping, sign encoding, bit slicing, placement	Programming, verify/retry, checksum, provenance
Input representation	DAC or pulse modes, fanout, legal ranges	Activation precision, slice count, scale, schedule	Command sequencing and range selection
Output precision	Integration window, ADC channels, saturation behavior	ADC plan, partial-sum tree, digital correction	Calibration binding, telemetry, retry policy
Numerical guarantee	Characterized error envelope by operating point	Operator-specific tolerance and fallback variant	Verify live state remains inside the compiled envelope

Defect hierarchy

A bad cell, a bad ADC, and a bad tile are different problems

Each level needs a distinct health record and recovery action. Collapsing all defects into one bitmap makes placement unstable, wastes healthy capacity, and prevents useful diagnostics.

Important evidence correction: Mythic's public patent supports virtual addressing and harvesting of defective compute tiles, including removal of a minimal set of tile rows or columns. It does not by itself prove spare flash-cell rows, spare bitlines, or spare ADC channels inside Mythic's ACE array. The row/ADC example below comes from a separate EnCharge AI patent and is used only as an alternative design pattern.

L0

Cell or device stateProgramming error, stuck cell, noise, drift, retention

MitigationVerify-and-program, differential encoding, redundant devices, sensitivity-aware placement, calibration, retraining or digital residual.

CellHealth + achieved_conductance

L1

Row or wordlineDriver, line, or clustered cell failure

MitigationRedirect the logical row to a spare row, shift following rows, or reserve the region. Reprogram all affected weight slices.

RowRemap[logical] = physical

L2

Column or ADC channelBitline, integrator, ADC, or output lane failure

MitigationRoute the bitline to a spare converter, shift channels, or reduce usable output width. Rebuild the output-channel map.

ColumnRemap + AdcHealth

L3

Array or subarrayExhausted spares, local thermal/noise region

MitigationQuarantine the region, split the tensor differently, consume reserved capacity, or select a lower-stress precision mode.

ArrayCapability + health_class

L4

Compute tileArray, RISC-V, SIMD, SRAM, or router compute path

MitigationAssign only good tiles valid logical addresses. Preserve pass-through routing where hardware permits it.

LogicalTileMap + harvest_mask

L5

Chip, link, card, hostDevice, PCIe/fabric link, power, or host failure

MitigationReform the logical mesh, reroute collectives, restore a distributed checkpoint, and rebind or recompile affected stages.

ClusterTopology + route_epoch

Mythic patent mechanism

Virtualize the tile grid

Test tiles, mark low-performing or defective tiles, and create a physical-to-virtual coordinate map that maximizes usable compute. A null-address tile can be skipped as a compute destination while packet routing continues through it. Applications target a stable logical grid.

Open US11475973B2

Alternative array mechanism

Redirect rows and converter channels

EnCharge describes normally disabled redundant ADC channels, input/output selection networks, and spare rows. This is useful evidence for the required software contract, but it is a pending patent application and not evidence of Mythic or iHW product behavior.

Open WO2025122556A1

Health lifecycle

Discover once, bind explicitly, monitor continuously

FactoryCharacterizeBIST cells, rows, columns, ADCs, arrays, tiles, links, and corners

ProvisionCreate base mapBurn immutable IDs and repair data; sign the factory health record

BootDiscover live stateMerge factory map, boot tests, firmware revision, topology, and calibration

CompilePlace logical workUse capability classes and reserve capacity without hard-coding serial numbers

LoadBind and programSelect physical resources, program weights, verify checksums, publish map epoch

RunObserve and recoverTrack drift, saturation, retries, faults, route health, and rebind at a checkpoint

Artifact	Mutability	Owner	Required fields
Target descriptor	Versioned per architecture/SKU	Architecture + compiler	Arrays, dimensions, formats, commands, converters, memory, links, legal modes
Factory health	Immutable and signed	Manufacturing	Device ID, repair map, disabled resources, characterization bins, provenance
Live health	Changes at boot/runtime	Firmware/runtime	Available resources, temperatures, errors, spare budget, route epoch, timestamps
Calibration bundle	Changes with chip and environment	Calibration service	Ranges, offsets, gains, noise envelope, temperature/voltage window, expiry
Executable intent	Immutable build output	Compiler	Logical placement, variants, tolerance, constants, collectives, fallback, debug map
Binding record	One per deployment epoch	Runtime	Logical-to-physical map, programmed-weight hashes, calibration IDs, routes, checkpoint

Buildable contracts

The compiler targets capabilities; the runtime binds a particular device

This is the key separation borrowed from virtual device and harvesting designs. A portable executable should express logical resource requirements and legal variants. Serial-number-specific defects remain in the binding record.

Target and logical-mesh descriptor

schema: acim.target/v1
target: acim.flash.hybrid.v1
tile:
  logical_grid: [8, 8]
  array_shapes: [[512, 512]]
  weight_modes: [w4_diff, w8_sliced]
  activation_modes: [a8_serial, a16_sliced]
  adc_bits: [6, 8]
  spare_rows: 8
  spare_columns: 4
digital:
  ops: [layer_norm, softmax, rope, silu, topk]
mesh:
  shape: [2, 4]
  links: [pcie, acim_fabric]
  collectives: [all_gather, reduce_scatter, all_reduce]
abi: acim.runtime/3

Live health and calibration view

schema: acim.health/v1
device_id: chip-07
health_epoch: 184
calibration_id: cal-chip07-42c-1v0
usable_tiles: 59
tile_map: logical_to_physical.bin
arrays:
  - id: t03.a1
    class: nominal
    row_remap: {17: 519}
    adc_remap: {61: 515}
    noise_rms_lsb: 0.43
    saturation_rate: 0.0007
spare_budget:
  rows_remaining: 7
  columns_remaining: 3
routes:
  epoch: 12
  unavailable_links: [chip07.east]

Executable physical intent

schema: acim.executable/v2
graph_hash: sha256:...
target_compat: acim.flash.hybrid.v1
logical_mesh: [2, 4]
programs:
  - op: decoder.0.mlp.up
    kind: acim.mvm
    layout: column_parallel
    weight_mode: w4_diff
    adc_plan: adc6_x2
    tolerance: rel_l2_0p02
    reserve_class: one_array
    fallback: digital.int8
collectives:
  - after: decoder.0.attn.out
    kind: reduce_scatter
debug_map: graph_to_command.cbor

Stable C++ runtime boundary

class AcimDevice {
 public:
  virtual CapabilityView capabilities() const = 0;
  virtual HealthView health() const = 0;
  virtual Binding bind(const Executable&) = 0;
  virtual Event program(const Binding&) = 0;
  virtual Event submit(const CommandBatch&) = 0;
  virtual RecoveryResult recover(const Fault&) = 0;
};

class AcimMesh {
 public:
  virtual MeshTensor distribute(const Tensor&, ShardSpec) = 0;
  virtual Event collective(Collective, MeshTensor&) = 0;
  virtual Checkpoint checkpoint() = 0;
  virtual void reform(const TopologyView&) = 0;
};

Security and correctness: firmware validates descriptor bounds, executable signatures, ABI versions, programmed-weight hashes, calibration validity, DMA ranges, and route epochs. Host metadata is never trusted merely because it came from the compiler.

Fault-aware compiler

Placement is a constrained optimization, not a rectangular tiling exercise

Compiler and runtime placement algorithmCompile against capability classes; bind against one health epoch.

for op in topological_order(graph):
  variants = legal_lowerings(op, target, accuracy_policy)
  for variant in variants:
    shards = tile_tensor(variant, logical_mesh)
    candidates = resources_matching(shards, capability_class)
    score = (
        estimated_latency
      + lambda_energy * estimated_energy
      + lambda_error  * predicted_numerical_error
      + lambda_comm   * collective_bytes
      + lambda_wear   * programming_and_aging_cost
      + lambda_risk   * spare_consumption
    )
  choose pareto_feasible_variant_or_fallback()

at load time:
  verify health_epoch and calibration envelope
  match logical shards to physical arrays
  apply row/column/tile remaps
  program and verify weights
  publish signed binding record

on fault:
  retry -> recalibrate -> local remap -> tile rebind
        -> mesh reform/checkpoint restore -> digital fallback

Accuracy cost

Predict layer sensitivity

Use quantization sweeps, measured nonideality models, activation statistics, calibration state, and selective digital residuals. Sensitive layers can request wider ADC plans or digital execution.

Capacity cost

Reserve without wasting the fleet

Compile variants for nominal and degraded capacity classes. Keep spare rows, columns, arrays, and tiles out of normal placement unless policy explicitly spends them.

Communication cost

Place around collectives

Estimate bytes and hops for activation shards and partial sums. Local weights remove weight traffic, not all inter-chip traffic.

Research input: the 2026 Dyn-A-Map paper ranks vulnerable subarray regions and relocates important weights to healthier regions without retraining. Its evidence is pre-silicon SPICE and architecture simulation, so use it as a mapping heuristic to validate on real devices, not as a proven field-repair result. Read Dyn-A-Map.

Open-source architecture audit

What to adapt from Tenstorrent, and what must remain ACiM-native

Reuse decisions apply to selected modules and contracts after dependency and license review. Repository names are not blanket approval to copy an implementation.

Reference	ACiM decision	Concrete adaptation	Do not import
TT-MLIR + TT-XLA + Forge ONNX	Adapt frontend and pass patterns	Keep StableHLO/ONNX import, diagnostics, pass pipelines, and plugin boundaries; introduce ACiM target dialects and capability interfaces.	TTNN, TTMetal, TTKernel operations and Tensix legality assumptions.
TT-UMD SoC/cluster descriptors	Adapt isolated descriptor machinery	Use a grid descriptor as the compiler/runtime contract; extend harvesting into row, column, ADC, array, tile, and calibration views.	Tenstorrent discovery, PCIe transport, TLB, memory map, firmware, and device implementation.
MeshDevice + MeshTensor	Adapt virtual-device semantics	Create `AcimMesh`, explicit sharding/replication, mesh command queues, and explicit all-gather/reduce-scatter/all-reduce.	Assumed TT link topology, tensor layouts, kernels, and dispatch implementation.
Cluster descriptors + tt-run	Adapt topology/test patterns	Store static and discovered topologies independently; test subsets, fallback mappings, multi-host SPMD, and pipeline groups without hardware.	Tenstorrent-specific YAML fields and launcher/device behavior.
TT-Fabric	Study control-plane architecture	Separate persistent health/routing control from the data plane; pause, flush, reroute, reform a mesh, and restore checkpoints.	Fabric router, protocol, link, and firmware code tied to TT hardware.
tt-emule	Adapt same-API backend pattern	Run compiler/runtime tests against a functional ACiM emulator selected behind the same HAL as hardware.	Tensix core threads, circular buffers, NoC, SRAM, and kernel semantics.
ttnn-visualizer + tt-perf-report	Adapt reports and UX selectively	Visualize graph placement, analog ranges, defect maps, calibration epochs, converter use, collectives, and first divergent layer.	TT trace schemas, counters, memory layouts, and operation assumptions.
TT-Metal, UMD transport, KMD, firmware	Replace	Build ACiM commands, programming service, HAL, driver, firmware scheduler, power/reset, safety, and telemetry from the target ABI.	All device-specific implementations unless a narrow utility passes semantic, test, dependency, and license review.

Why the UMD decision changed

Descriptor concepts are explicitly reusable

Tenstorrent's custom-SoC guide calls the SoC descriptor reusable for other grid-like SoCs and identifies it as a higher-stack contract. That supports adapting the descriptor/coordinate boundary while replacing transport and device code.

Read the custom-SoC guide

Other foundations

Use target-neutral projects where they fit better

MLIR supplies IR infrastructure, OpenXLA PJRT supplies a framework plugin contract, TVM supplies a target-extensible compiler reference, and AIHWKit/CrossSim supply analog-aware numerical modeling. Evaluate each behind the same target contract.

Open the SDK implementation structure

Scale-up and scale-out

A logical mesh spans six physical scopes

"Weights stay in memory" removes repeated model-weight fetches. It does not remove activation, partial-sum, KV-cache, expert-routing, control, checkpoint, and telemetry traffic.

Level 0ArrayOne MVM slice, converter group, local spares

Level 1TileArrays, digital core, SRAM, local router

Level 2ChipLogical tile grid, NoC, shared services

Level 3CardSeveral chips, switch/fabric, power domain

Level 4HostRuntime executor, memory, queues, checkpoint shard

Level 5ClusterDistributed mesh, serving scheduler, fleet health

Data plane

WeightsProgram once per model/binding epoch; verify and retain locally.

ActivationsShard, multicast, pipeline, and exchange between analog and digital stages.

Partial sumsReduce across row-parallel shards with declared accumulation precision.

KV cacheAllocate and page in write-friendly digital or specialized dynamic memory.

CollectivesAll-gather, reduce-scatter, all-reduce, broadcast, all-to-all.

Control plane

DiscoveryEnumerate chips, arrays, links, topology, firmware, and capacity class.

HealthTrack calibration, temperature, saturation, retries, faults, and spare budget.

PlacementBind logical programs to a health epoch and route epoch.

RecoveryPause, drain, remap, reprogram, reform mesh, restore checkpoint, resume.

GovernanceSign artifacts, enforce ABI, record provenance, support rollback.

Parallel mode	Analog placement	Required communication	Primary risk
Column-parallel linear	Shard output channels and their local weights	All-gather output shards when the next operator needs the full tensor	Output/ADC imbalance and gather latency
Row-parallel linear	Shard input channels and produce partial outputs	Reduce-scatter or all-reduce partial sums in an explicit accumulator format	Reduction bandwidth and numerical order
Pipeline parallel	Place layer ranges on chips/cards/hosts	Point-to-point activation transfer and microbatch scheduling	Bubbles, stage imbalance, recovery scope
Expert parallel	Store different MoE experts in local arrays	Top-k routing and all-to-all token exchange; return weighted expert outputs	Hot experts, dynamic traffic, stranded capacity
Data parallel serving	Replicate the model or pipeline for independent requests	Request routing and model/adapter distribution; no per-token gradient reduction	Weight programming time and replica capacity

Mythic's statement that future APUs do not need a GPU-like high-speed APU-to-APU link is a company claim. Whether it holds depends on model partitioning, batch, context, topology, and how much activation/KV/partial-sum traffic remains. The runtime must model and measure that traffic rather than encode the claim as an assumption.

Hybrid LLM mapping

Place operators by state behavior, not by the word "MatMul"

DigitalNorm + RoPEReductions, reciprocal/sqrt, position-dependent transforms

AnalogQ/K/V projectionsStatic dense weights; shard by head or hidden dimension

Digital/dynamic CIMAttention + KVPer-token writes, QK, mask, softmax, AV, paging

AnalogOutput + MLPStatic dense projections, gate/up/down matrices

MixedResidual + sampleDigital accumulation, LM head option, logits, top-k/p sampling

LLM work	Default target	Why	Compiler/runtime requirement
Embedding / LM head	Analog when capacity and lookup behavior fit; otherwise digital memory	Static weights but irregular lookup or very wide output	Vocabulary partition, gather/top-k strategy, tied-weight handling
Q/K/V and output projections	Analog CIM	Large fixed dense matrices with repeated use	Head-aware sharding, activation scales, calibration, collectives
MLP gate/up/down	Analog CIM	Usually the largest fixed-weight compute share	Fuse digital activation where legal; balance array and converter use
QK^T and attention-value product	Digital engine by default	Both operands depend on runtime tokens; NVM reprogramming is a poor fit	Specialized gain-cell/dynamic-CIM capability may override this explicitly
KV cache	SRAM/DRAM/HBM or write-friendly dynamic CIM	Grows per request and changes every token	Paged allocation, eviction, sharing, compression, context policy
LayerNorm/RMSNorm, RoPE, softmax, SiLU/GELU	Digital vector/scalar engine	Reductions, transcendental/approximate functions, control, precision	Stable kernels, fusion, scratch planning, numerical contract
MoE router and top-k	Digital	Dynamic sparse decision and load balancing	Expert placement, capacity factors, all-to-all, hot-expert telemetry
LoRA / accuracy adapter	Digital companion	Small, updateable correction avoids reprogramming base analog weights	Version adapters separately and overlap their GEMM with analog MVM

Prefill

High parallelism favors array utilization

Many prompt tokens turn projection and MLP work into matrix-matrix execution. Tile tokens and output channels together, overlap digital attention with analog projections, and pipeline collectives. Time-to-first-token remains the product metric.

Decode

Weight locality helps, but dynamic state dominates

Each step is sequential and often matrix-vector at low batch. Converter latency, digital companion balance, KV-cache access, inter-chip activation traffic, and synchronization can dominate. Measure inter-token latency and energy/token.

Software pattern worth reusing: vLLM's PagedAttention shows why the KV cache needs a virtualized, dynamically allocated memory layer. Reuse the paging and serving concepts; replace GPU-specific kernels and memory APIs with the ACiM hybrid runtime. Read the PagedAttention paper.

Four-bit LLMs

Four logical bits do not automatically mean one cheap analog operation

Representation

Separate model bits from physical cost

Logical weight formatW4 may be symmetric/asymmetric and per-tensor, per-channel, or group-scaled.
Cell representationOne logical weight may use one multi-level cell, a differential pair, several devices, or bit slices.
Activation executionA8/A16 may require serial input slices and multiple integration cycles.
Output reconstructionADC width, partial-sum count, scale precision, and digital accumulation determine effective accuracy.

Accuracy policy

Use selective precision, not one global fidelity

Quantize with measured rangesCalibrate group scales and clipping using representative prompts and target nonideality.
Protect sensitive pathsKeep norms, accumulators, logits, outlier channels, and selected layers in higher-precision digital execution.
Train or adapt for hardwareUse noise-aware fine-tuning or a small digital LoRA/residual adapter when model access permits.
Promote on evidenceChoose more slices, wider conversion, repeat-and-average, or fallback only when accuracy gain exceeds latency/energy cost.

The M1076 brief reports capacity in "weight parameters," not packed bytes. A 4-bit software checkpoint cannot be assumed to double M1076 weight capacity. Physical packing depends on unpublished cell encoding, differential representation, redundancy, slicing, and metadata. The compiler must query a target-specific physical_slots_per_logical_weight function.

A recent transformer adaptation study places fixed base-model weights on analog tiles and small LoRA corrections on digital cores; it reports simulation through LLaMA 3.1 8B, not full-system LLM silicon. This is a useful software architecture for preserving programmable adaptation without repeatedly reprogramming NVM arrays. Read the AHWA-LoRA study.

Capacity sanity check

Legacy M1076 numbers show why a new scale architecture is necessary

Illustrative lower bounds assume one model parameter consumes one advertised weight slot and ignore spares, replication, calibration data, digital state, and unavailable capacity. They are not performance estimates.

80Mweights per M1076Legacy product brief

320Mweights per 4-chip cardMP10304 product brief

1.28Bweights per claimed 16-chip card2021 company announcement

ceil(P/C)minimum capacity unitsBefore reserve and system overhead

Model parameters	M1076 chips at 80M	4-chip cards at 320M	16-chip units at 1.28B	Interpretation
7B	88	22	6	Minimum advertised slots; practical count is higher.
70B	875	219	55	Communication, power, host, and yield become system-level constraints.
1T	12,500	3,125	782	Mythic's current 1T claim necessarily refers to a rebuilt architecture, not simple M1076 scaling.

A capacity fit is not an execution proof. The next gates are supported operators, physical weight encoding, programming time, yield reserve, activation/partial-sum bandwidth, collectives, digital companion capacity, KV memory, accuracy at scale, thermal limits, and end-to-end tokens/second.

Runtime state machine

Recovery escalates from cheap local action to distributed restart

ObserveDetect anomalyTimeout, checksum, saturation, drift, ECC, route, or numerical sentinel

RetryRepeat safelyReplay idempotent command or use repeat-and-average within policy

CorrectRecalibrateRefresh offset/range/drift state and verify the operating envelope

RemapSpend local spareRedirect row/column/ADC or bind an alternate array/tile; reprogram weights

ReformChange meshDrain queues, update route epoch, restore distributed checkpoint

DegradeFallback or rejectUse digital variant, lower service tier, quarantine device, or fail explicitly

Recovery invariant	Reason	Test
No stale physical address survives a map epoch	Prevents commands from reaching harvested or reassigned resources	Inject remap between queued batches and reject old-epoch descriptors
Weights match the binding record	Row/column/tile movement changes physical storage	Corrupt a programmed block and require checksum failure before execution
Calibration matches operating conditions	Converter and device behavior changes with time and environment	Cross temperature/voltage boundary and force recalibration or fallback
Collectives use one topology epoch	Mixed route views can deadlock or corrupt partial sums	Drop a link during reduce-scatter and verify drain/reform/restart
Retries preserve request semantics	Duplicate execution can alter stateful work or output ordering	Replay at every command boundary with idempotency and sequence checks

Implementation plan

Build one fault-aware distributed linear layer before a full LLM

0 / 8 complete

Version the schemasDefine target, health, calibration, executable, binding, topology, and trace schemas with compatibility tests. Implement the functional backendExecute ACiM MVM plus injected noise, saturation, drift, row/column/tile defects, and digital fallback behind the production HAL. Import one transformer blockUse ONNX or StableHLO; preserve QKV, attention, norm, residual, and MLP boundaries with support diagnostics. Lower one distributed linearGenerate column- and row-parallel variants, logical array placement, ADC plan, accumulator format, and collective intent. Bind around injected defectsApply row/column remaps and tile harvesting, program weights, verify hashes, and emit a reproducible binding record. Run on a fake AcimMeshExecute all-gather and reduce-scatter across simulated chips with link delay, loss, timeout, and topology reform. Execute a hybrid decoder blockAnalog projections/MLP, digital attention/KV/norm/activation, layerwise comparison, and first-divergence reporting. Correlate with siliconReplace modeled distributions with measured chips and gate accuracy, latency, energy, programming, drift, yield, and recovery.

Suggested repository boundary

Keep physical state out of the graph IR

frontend/, ir/, passes/, target/, artifact/, runtime/, hal/, sim/, firmware/, driver/, tools/, models/, and tests/ can evolve independently behind versioned contracts.

Acceptance gate

Prove end-to-end value

Report perplexity/task accuracy, TTFT, inter-token latency, throughput, energy/token, programmed capacity, converter utilization, communication bytes, spare use, reprogram time, fault coverage, and recovery time against digital baselines.

Professional claim ledger

What the evidence supports today

Statement	Evidence class	Supported conclusion	Do not conclude
M1076 has flash-based ACE arrays and 80M-weight capacity	Vendor product brief	Legacy architecture and advertised capacity are documented.	Exact cell encoding, defect repair, packed W4 capacity, or LLM performance.
Mythic tile grid can hide defective compute tiles	Granted patent	Virtual tile addressing and row/column tile harvesting are disclosed mechanisms.	That every mechanism ships, or that internal flash rows/ADCs use the same repair.
ALBERT achieves near-iso-accuracy on an analog chip	Measured silicon	Static transformer linear layers can work with mapping, programming, HWA fine-tuning, and recalibration.	End-to-end decoder LLM, on-chip attention, or billion-parameter scale.
Gain-cell analog attention can approach GPT-2 model quality	SPICE + software	A specialized dynamic-memory attention path is plausible and has a concrete co-design.	Fabricated system speed, energy, yield, retention, or production readiness.
Mythic supports 1T+ LLMs with large efficiency gains	Roadmap/internal benchmark	The company has declared a new chiplet and hybrid-system direction.	Independent, shipping, end-to-end LLM performance.
This page's AcimMesh and schemas	Engineering proposal	They are implementable requirements derived from public mechanisms and software patterns.	That Mythic, iHW, Tenstorrent, or another vendor uses these exact interfaces.

Primary sources

Architecture, patents, measured results, and open-source references

Mythic M1076 product brief76 tiles, flash ACE, digital tile resources, 80M weights, 25 TOPS, PCIe

Mythic MP10304 product briefFour-chip card, 320M weights, 100 TOPS, combined model execution

Mythic 2021 scale announcementCompany claims for a 16-chip, 1.28B-weight card

Mythic 2025 roadmap announcementCompany claims for rebuilt chiplets, 1T+ LLMs, scaling, CAMP SDK, and internal benchmarks

Mythic and Videantis acquisitionDisclosed analog/digital hybrid partition, including digital attention

Mythic tile redundancy patentTesting, virtual addressing, tile harvesting, and pass-through routing

EnCharge row/column redundancy patentAlternative spare-row and redundant-ADC selection mechanism

ALBERT on 14 nm analog siliconMeasured hardware mapping, accuracy, HWA fine-tuning, drift, and off-chip boundaries

Gain-cell analog attentionSPICE and model simulation for write-friendly KV storage and analog attention

3D AIMC mixture-of-experts studySimulation-based LLM scaling and published code/data links

Dyn-A-MapPre-silicon fault sensitivity and post-training spatial remapping

AHWA-LoRA transformer adaptationSimulation of analog base weights with small digital adapters, including an 8B decoder model

TT-UMD custom-SoC guideReusable SoC/cluster descriptor boundary and custom implementation guidance

Tenstorrent MeshDevice reportVirtual mesh, explicit sharding, replication, dispatch, and collectives

Tenstorrent TT-Fabric architectureData/control planes, routing, topology repair, checkpoints, and fault injection

Tenstorrent tt-emuleSame-host-API alternate backend pattern for hardware-free testing

Tenstorrent cluster descriptorsStatic topology, subset, fallback, and multi-host test descriptions

TT-NN VisualizerGraph, memory, operation, tensor, and performance-report UX reference

Tenstorrent performance report toolTrace analysis, operation IDs, signposts, and bottleneck reports

TT-MLIR device modelLogical grids, meshes, affine mapping, and virtual-device concepts

IBM AIHWKitOpen analog-aware training and inference simulation

Sandia CrossSimOpen Analog CIM accuracy and nonideality simulation

ACiM PCIe board bring-upPhysical host/device implementation below the logical mesh: UMD/KMD, queues, DMA, firmware, calibration binding, recovery, and first-model qualification

OpenXLA PJRT integrationFramework-to-device plugin contract

Apache TVM architectureTarget-extensible compiler and runtime reference

Analog CIM Scale-Out, Fault Tolerance, and LLM Inference

Can a Mythic- or iHW-style Analog CIM system run an LLM?

Transformer feasibility is real but bounded

Dynamic attention needs different memory

Legacy product facts and current roadmap differ

Mythic's acquisition reinforces this partition

What actually happens during one Analog CIM matrix-vector operation

A bad cell, a bad ADC, and a bad tile are different problems

Virtualize the tile grid

Redirect rows and converter channels

Discover once, bind explicitly, monitor continuously

The compiler targets capabilities; the runtime binds a particular device

Target and logical-mesh descriptor

Live health and calibration view

Executable physical intent

Stable C++ runtime boundary

Placement is a constrained optimization, not a rectangular tiling exercise

Predict layer sensitivity

Reserve without wasting the fleet

Place around collectives

What to adapt from Tenstorrent, and what must remain ACiM-native

Descriptor concepts are explicitly reusable

Use target-neutral projects where they fit better

A logical mesh spans six physical scopes

Data plane

Control plane

Place operators by state behavior, not by the word "MatMul"

High parallelism favors array utilization

Weight locality helps, but dynamic state dominates

Four logical bits do not automatically mean one cheap analog operation

Separate model bits from physical cost

Use selective precision, not one global fidelity

Legacy M1076 numbers show why a new scale architecture is necessary

Recovery escalates from cheap local action to distributed restart

Build one fault-aware distributed linear layer before a full LLM

Keep physical state out of the graph IR

Prove end-to-end value

What the evidence supports today

Architecture, patents, measured results, and open-source references