Implementation guide | reviewed 2026-07-12
Analog CIM Board Bring-Up: YOLO, PCIe, UMD, KMD, and Firmware
This is the missing bridge between a compiled .acx artifact and a controlled physical board. It
defines a proposed Linux host stack, Peripheral Component Interconnect Express (PCIe) endpoint contract,
queue and Direct Memory Access (DMA) protocol, device firmware loop,
hybrid YOLO placement, analog programming and calibration lifecycle, fault recovery, and a staged path from
simulator to silicon.
Choose the driver path
Terminology
Trace the system
Test the whole chain
Implement PCIe
Open the delivery plan
4deployment modes with different trust boundaries
3versioned interfaces: user-space API (UAPI), firmware ABI, artifact
9incremental hardware milestones
1correlation ID from graph node to board trace
Primary-source behavior
Proposed v0 design
Vendor input required
Gap audit
What the existing documents cover, and what this page adds
The compiler, artifact, Hardware Abstraction Layer (HAL), System-on-Chip (SoC), and scale-out contracts already exist. Their missing implementation boundary was how Linux safely discovers a board, transfers buffers, starts firmware work, receives completion, and recovers a device without exposing arbitrary host memory.
Terminology map
Define the contract, layer, and measurement behind each term
This glossary explains how each term is used in this ACiM design, not only what its letters expand to. A term may name a source interface, binary contract, compiler representation, transport mechanism, hardware block, numerical assumption, or licensing record; treating those categories as interchangeable creates architecture and implementation errors.
Software interfaces, ownership, and versioned contracts
- API
- Application Programming InterfaceThe source-level functions, classes, and data types one component exposes. Recompiling can adapt an API change; an already-built binary cannot.
- ABI
- Application Binary InterfaceThe binary contract for calling conventions, symbol names, data layout, alignment, opcodes, and version compatibility. This design has separate artifact, runtime-UMD, UMD-KMD, and firmware command ABIs.
- C ABI
- C language binary interfaceA stable function-call boundary using C-compatible symbols and layouts. It is often used for SDK plugins because C++ name mangling and standard-library ABIs vary by toolchain.
- UAPI
- Userspace API and ABIThe Linux kernel/userspace contract: device files, ioctl numbers, structures, flags, handles, mmap behavior, events, and errors. Once shipped, compatibility must be managed deliberately.
- SDK
- Software Development KitThe developer-facing product: headers, libraries, compiler, runtime, tools, examples, diagnostics, documentation, packages, and compatibility policy. An SDK is more than a compiler.
- HAL
- Hardware Abstraction LayerA narrow interface hiding transport details so the same runtime can target a simulator, VFIO prototype, Linux driver, FPGA, or silicon backend.
- Runtime
- Execution orchestration libraryLoads compiled artifacts, creates sessions, allocates and binds buffers, selects queues, submits work, waits on events, recovers faults, and exposes profiling.
- UMD
- User-Mode DriverAn unprivileged userspace library that discovers devices, owns process-local objects, prepares descriptors, and translates HAL/runtime requests into the kernel UAPI.
- KMD
- Kernel-Mode DriverPrivileged kernel code that owns PCIe resources, DMA isolation, interrupts, reset, power, firmware loading, process cleanup, and device lifetime.
- Driver
- Software controlling a device boundaryAn imprecise word unless qualified. In this guide it can mean UMD, KMD, or both; firmware is a separate device-resident layer.
- Firmware
- Device-resident control softwareCode running on an embedded controller that boots the accelerator, validates command descriptors, sequences engines, manages health, and reports faults.
- Device ABI
- Host-to-firmware binary contractThe exact command, descriptor, queue, register, status, and error formats accepted by a firmware/device generation. It is not the model IR.
- ISA
- Instruction Set ArchitectureThe programmer-visible instructions, registers, address model, and execution rules of a processor or accelerator engine. An ABI may select and constrain an ISA but is not the ISA itself.
- Microcode
- Low-level engine control programA compact target-specific sequence that drives internal datapaths. It normally sits below compiler IR and above fixed hardware state machines.
- Artifact
- Deployable compiler outputA versioned package containing code or commands, constants, buffer metadata, target requirements, debug maps, calibration references, and provenance. It is the hinge between offline build and runtime.
- Schema
- Machine-readable data definitionDefines fields, types, required values, defaults, and evolution rules for an artifact, command, trace, or capability description.
- SemVer
- Semantic VersioningA major.minor.patch convention. It helps communicate compatibility but does not create compatibility unless each ABI and schema has explicit rules and tests.
- SoC
- System-on-ChipA chip integrating processors, memory controllers, interconnect, accelerators, security, and peripherals. A PCIe accelerator card may contain one or more SoCs or chiplets.
- CI
- Continuous IntegrationAutomated build, static analysis, unit, compatibility, simulator, and regression gates run when code or generated contracts change.
- CD
- Continuous Delivery or DeploymentAutomated packaging and promotion after CI. Driver signing, firmware compatibility, rollback, and hardware qualification make accelerator CD different from a web-only release.
- DKMS
- Dynamic Kernel Module SupportLinux packaging that rebuilds an out-of-tree kernel module for installed kernels. It eases installation but does not guarantee source compatibility with future kernels.
- MVP
- Minimum Viable ProductThe smallest end-to-end workflow that proves user value. For this stack, a one-operator vertical slice is an engineering milestone, not necessarily a market MVP.
- SLO
- Service-Level ObjectiveAn internal measurable target such as p99 latency, availability, or recovery time. Several SLOs may support one customer commitment.
- SLA
- Service-Level AgreementAn external agreement defining promised service and remedies. A silicon accuracy target or benchmark alone is not an SLA.
Model formats, compiler representations, and transformations
- ONNX
- Open Neural Network ExchangeA serialized model graph format with typed operators, tensors, attributes, and versioned operator sets. It is an input interchange format, not a hardware schedule.
- Opset
- Operator-set versionThe ONNX version governing operator semantics. A model's IR version and opset imports are separate compatibility fields and both must be checked.
- Graph
- Operations connected by data dependenciesThe compiler view of model computation. Dynamic control flow, state, aliases, side effects, and symbolic dimensions make it richer than a simple directed acyclic graph.
- Tensor
- Typed multidimensional valueDefined by element type, rank, shape, layout, quantization, and memory placement. Equal shapes do not imply equal physical layouts.
- IR
- Intermediate RepresentationThe compiler's structured program form between source model and executable. Different IR levels preserve different semantics and expose different optimization choices.
- HLO
- High-Level OperationsA tensor-operation IR family associated with XLA. HLO names a level of representation; it is not one universal file format or device ABI.
- StableHLO
- Versioned high-level ML operation setA portability layer built on MLIR for carrying framework semantics to compilers with compatibility guarantees. It does not define ACiM placement, calibration, runtime calls, or firmware commands.
- MLIR
- Multi-Level Intermediate RepresentationCompiler infrastructure for defining dialects and progressively lowering programs. MLIR is the framework; a particular dialect and pass pipeline form the target compiler.
- Dialect
- MLIR vocabulary and rulesA namespace of operations, types, attributes, parsers, printers, and verification rules for one abstraction level or domain.
- Op
- Compiler operationAn IR instruction with operands, results, attributes, regions, and semantics. It may represent a whole convolution, a DMA copy, or one target command depending on the dialect.
- SSA
- Static Single AssignmentAn IR property in which each value is defined once. Dataflow and use-def chains become explicit, enabling analysis and safe rewrites.
- DSL
- Domain-Specific LanguageA language designed for one domain, such as tensor kernels or schedules. A DSL may lower into MLIR; it is not synonymous with an MLIR dialect, which is an IR extension mechanism.
- Pass
- Compiler analysis or transformationA staged unit that inspects or rewrites IR. Production pipelines declare ordering, preconditions, invariants, diagnostics, and tests rather than collecting unrelated rewrites.
- Canonicalize
- Normalize equivalent IR formsRewrites expressions into predictable simpler forms. It reduces pattern count but must preserve semantics and is not target lowering by itself.
- Legalize
- Replace unsupported constructsTransforms operations, types, shapes, or layouts until every remaining construct is valid for the next dialect or target capability contract.
- Lowering
- Move to a more concrete abstractionConverts high-level model meaning into target operations, schedules, buffers, commands, or machine code while recording any precision and fallback decisions.
- Fusion
- Combine operations into one execution regionCan remove intermediate traffic and launch overhead, but may increase live memory, analog range, converter work, or unsupported-op pressure.
- Partition
- Assign graph regions to execution backendsSeparates ACiM-supported work from DSP, CPU, or other fallback work. Boundary conversion and transfer costs belong in the decision.
- Frontend
- Source-facing compiler stageImports ONNX, StableHLO, or framework programs, validates semantics, normalizes shapes and types, and emits a compiler IR.
- Backend
- Target-facing compiler stageMaps legal IR to one device's arrays, memories, converters, command format, code, and artifact ABI. This is the least reusable part across Tenstorrent and ACiM.
- Codegen
- Code or command generationEmits machine instructions, microcode, command streams, constants, relocations, and metadata after scheduling and memory planning.
- AOT
- Ahead-of-Time compilationCompiles before deployment, enabling qualification and caching. It requires a clear target descriptor and artifact compatibility contract.
- JIT
- Just-in-Time compilationCompiles at or near execution time for actual shapes or hardware. It improves specialization but adds latency, security, reproducibility, and cache concerns.
- ODS
- Operation Definition SpecificationMLIR's declarative way to define operations, types, and attributes using TableGen records, from which C++ declarations, verifiers, builders, and documentation can be generated.
- TableGen
- LLVM declarative record language and generatorDescribes structured compiler data and emits repetitive C++ or documentation. It is a build-time generator, not the runtime artifact schema.
- LLVM IR
- LLVM low-level intermediate representationA typed SSA representation suited to CPU-like code generation. Tensor and ACiM array semantics usually need higher-level dialects before lowering toward LLVM IR.
- TOSA
- Tensor Operator Set ArchitectureAn Arm-defined tensor operator specification and MLIR dialect intended as a common layer between frameworks and hardware-oriented lowering.
- TVM
- Apache TVMAn end-to-end machine-learning compiler stack with graph and tensor scheduling abstractions. Its architecture can be studied without assuming its cost model fits analog arrays.
- PJRT
- OpenXLA uniform device APIA C API and plugin boundary through which frameworks access opaque, device-specific implementations. It is neither a model IR nor a kernel driver UAPI.
- ORT
- ONNX RuntimeA model runtime that can partition supported graph regions to a hardware Execution Provider while retaining other regions on fallback providers.
- EP
- Execution ProviderAn ONNX Runtime plugin that advertises capability, compiles claimed subgraphs, creates runtime state, and invokes the accelerator.
- Target
- Target descriptorA machine-readable description of device generation, capacities, supported formats, topology, firmware/ABI versions, errata, and calibrated capabilities used by the compiler.
- Layout
- Logical-to-physical tensor arrangementDefines dimension order, blocking, strides, sharding, bank placement, and padding. Layout conversions can dominate a theoretically cheap operator.
Host/device transport, addressing, queues, and interrupts
- PCIe
- Peripheral Component Interconnect ExpressThe host/device interconnect carrying configuration, MMIO, DMA transactions, and message-signaled interrupts.
- BAR
- Base Address RegisterPCIe-discovered address region exposing device control registers or a device-memory aperture to the host.
- MMIO
- Memory-Mapped Input/OutputCPU loads and stores directed to device registers rather than ordinary RAM.
- PIO
- Programmed Input/OutputThe CPU directly reads or writes device registers/data instead of asking a DMA engine to transfer a buffer.
- DMA
- Direct Memory AccessDevice-initiated transfer to or from host memory without the CPU copying each byte.
- IOMMU
- Input-Output Memory Management UnitTranslates and restricts device DMA addresses so a context cannot access arbitrary host memory.
- IOVA
- Input-Output Virtual AddressThe address a device places in a DMA transaction after the kernel establishes an IOMMU/DMA mapping.
- BO
- Buffer ObjectDriver-managed allocation or imported buffer represented to userspace by an opaque handle.
- Descriptor
- Fixed-format work or transfer recordContains buffer addresses, sizes, operation identifiers, dependencies, flags, and a cookie. Its byte layout is part of the device ABI.
- SQ
- Submission QueueRing where the host publishes validated commands for firmware/device consumption.
- CQ
- Completion QueueRing where the device reports terminal status, fault information, counters, and the matching cookie.
- Doorbell
- MMIO notification registerA host write telling the device that a queue producer index has advanced.
- IRQ
- Interrupt RequestKernel-visible notification that completion, fault, firmware, or health work needs service.
- MSI-X
- Message Signaled Interrupts ExtendedPCIe interrupt mechanism supporting multiple independently configured vectors.
- FLR
- Function Level ResetPCIe reset of one function without resetting the entire machine or unrelated functions.
- AER
- Advanced Error ReportingPCIe facility reporting correctable, non-fatal, and fatal link/transaction errors.
- FD
- File DescriptorPer-process Linux handle used for devices, dma-buf objects, events, and other kernel resources.
dma-buf- Shared DMA buffer frameworkLinux mechanism for exporting and importing the same buffer across device drivers.
- RO/RW/WO
- Register access modesRead-only, read/write, and write-only. RW1C means writing one clears the selected status bit.
- DRM
- Direct Rendering ManagerLinux kernel subsystem reused by
drivers/accel; here DRM does not mean digital-rights management. - TLB
- Translation Lookaside BufferHardware cache of address translations; a vendor-specific TLB programming model is not portable by default.
- SMMU
- System Memory Management UnitArm's I/O memory-management architecture used to translate and isolate device accesses.
- NUMA
- Non-Uniform Memory AccessHost architecture where memory latency/bandwidth depends on which CPU and memory node are used.
- VFIO
- Virtual Function I/OLinux framework for IOMMU-protected direct device access from a trusted userspace driver. Useful for prototypes, but production ownership and sharing still need design.
- UIO
- Userspace I/OA small Linux framework exposing simple MMIO and interrupts. It does not provide VFIO-style IOMMU isolation or a full accelerator memory model.
ioctl- Linux device-control system callTransfers a numbered request and fixed UAPI structure between userspace and the KMD. Pointer size, padding, bounds, and backward compatibility require explicit design.
- Fence
- Asynchronous completion dependencyA waitable object indicating that prior work has completed. It orders resource reuse; it is different from a CPU or DMA memory barrier.
- SG
- Scatter-gather listA sequence of physical or DMA segments representing one logical buffer so an engine can transfer non-contiguous memory without a CPU copy.
Digital compute, on-chip interconnect, memory, and implementation
- NPU
- Neural Processing UnitAn accelerator specialized for neural-network operations. It may be digital, analog, or hybrid; the name alone does not identify its arithmetic or memory architecture.
- HWA
- Hardware AcceleratorA specialized engine offloading work from a general CPU. NPU, DSP, video codec, DMA engine, and ACiM array are all possible HWAs.
- NoC
- Network-on-ChipThe internal packet or circuit-switched fabric connecting cores, tiles, SRAM banks, DMA engines, converters, and controllers. It is inside the device, unlike the PCIe host link.
- Tile
- Repeated compute and memory unitA physical or logical building block containing some combination of array, digital core, local SRAM, router, converters, and control. Its contents are architecture-specific.
- Core
- Programmable execution engineA processor that fetches instructions and manipulates state. An analog array performs MVM physics but is not itself a RISC-V-like core.
- RISC-V
- Open instruction-set architectureOften used for embedded control, scheduling, scalar work, and firmware. Too much control work can saturate cores and the NoC even when arrays are underused.
- CP
- Command ProcessorA device block or firmware core that fetches descriptors, checks dependencies, configures engines, and publishes completions.
- Control plane
- Lifecycle and policy pathOwns discovery, allocation, security, power, reset, health, version negotiation, and observability rather than tensor payload movement.
- Data plane
- Tensor and command execution pathCarries inputs, outputs, weights, partial sums, commands, and completion state through the hot path.
- DSP
- Digital Signal ProcessorA programmable digital engine efficient for vector, activation, reduction, preprocessing, and unsupported operators surrounding the analog MVM path.
- SIMD
- Single Instruction, Multiple DataOne instruction applies the same operation to several lanes. It describes data parallelism, not the complete processor schedule.
- VLIW
- Very Long Instruction WordAn ISA style encoding several operations for parallel issue, with much scheduling responsibility placed on the compiler.
- SRAM
- Static Random-Access MemoryFast volatile on-chip memory used for activations, commands, scratch data, and partial sums. It is distinct from non-volatile analog weight storage.
- DRAM
- Dynamic Random-Access MemoryHigh-capacity volatile memory, usually off-chip, holding models and tensors that do not fit locally. Moving data to and from DRAM is often expensive.
- L1
- Level-one local memory or cacheThe nearest programmable storage to a core or tile. In accelerator documentation, confirm whether L1 is software-managed SRAM, a cache, or both.
- Cache
- Hardware-managed copy of memoryExploits locality and is usually transparent to software. Scratchpad SRAM is explicitly allocated and moved by compiler/runtime, so it is not automatically a cache.
- Double buffer
- Overlap transfer and compute with two buffersWhile one buffer feeds compute, the other is filled or drained. It hides latency only when bandwidth, dependencies, and engine concurrency permit overlap.
- ASIC
- Application-Specific Integrated CircuitFixed-function or specialized silicon manufactured for a target application, unlike a reprogrammable FPGA.
- RTL
- Register-Transfer LevelA clocked digital hardware description of registers and transfers used to implement and verify controllers, buses, DMA, NoC, and digital compute.
- SystemC
- C++ system-modeling libraryUsed for functional, transaction-level, or cycle-oriented models before RTL and silicon. Model timing accuracy must be stated rather than inferred from the language.
- FPGA
- Field-Programmable Gate ArrayReconfigurable hardware used to prototype PCIe, firmware, commands, and digital engines before ASIC availability; it does not reproduce analog array physics by itself.
- ROM
- Read-Only MemoryImmutable or protected first-stage device code/data establishing reset-safe boot, identity, and firmware loading.
- PPA
- Power, Performance, and AreaThree coupled hardware implementation objectives. Improving array arithmetic alone may worsen converters, control, memory, or die area.
- DFT
- Design for TestStructures and methods that make manufactured silicon observable and testable, including scan, memory test, analog test modes, and calibration access.
- BIST
- Built-In Self-TestOn-device logic or firmware that runs known tests for memories, links, arrays, converters, or controllers and reports health.
- Bandwidth
- Data transferred per unit timeA capacity such as GB/s. Peak link bandwidth, sustained payload bandwidth, NoC bisection bandwidth, and per-array bandwidth are different measurements.
- Latency
- Time for one operation or requestState the scope and percentile: kernel, layer, model, compile, first-token, per-token, median, or p99.
- Throughput
- Completed work per unit timeExamples are inferences/s, tokens/s, or frames/s. High batching can improve throughput while worsening single-request latency.
- TOPS
- Tera Operations Per SecondA peak or measured arithmetic rate. The operation-count convention, precision, sparsity, utilization, and included digital/converter work must be disclosed.
- TOPS/W
- Operations per second per wattAn efficiency ratio whose value depends on operation counting and power boundary. Array-only peak TOPS/W is not end-to-end application efficiency.
- H2D
- Host-to-Device transferInputs, commands, constants, or firmware moving from host memory to the accelerator.
- D2H
- Device-to-Host transferOutputs, traces, or status moving from the accelerator to host memory.
- SKU
- Stock Keeping UnitA sellable board or device configuration with a defined chip count, memory, power envelope, firmware support, and qualification status.
Boot, system images, simulation, and low-level debug
- QEMU
- Machine emulator and virtualizerSystem emulation runs a guest CPU, memory, and only the devices implemented by the selected QEMU machine. A generic Arm or RISC-V
virtboard does not automatically contain an ACiM endpoint. - BSP
- Board Support PackageThe bootloader, kernel configuration, device tree, drivers, firmware, root-filesystem integration, and build metadata needed to boot a specific board.
- Bootloader
- Software that starts the host operating systemInitializes enough platform hardware to select and load the host Linux kernel, device tree, and optional initramfs. It is separate from ACiM controller firmware.
- Rootfs
- Root filesystemThe Linux userspace image containing libraries, configuration, UMD, runtime, tools, services, and applications. An initramfs is a rootfs held in memory during early boot.
- Cross-compile
- Build on one architecture for anotherFor example, build Arm64 Linux userspace on an x86 workstation or RISC-V controller firmware with a freestanding target toolchain and linker script.
- JTAG
- Board-level debug and programming interfaceLets a debugger halt a supported target, inspect registers and memory, and load or flash images through a tool such as OpenOCD. It is a bring-up path, not the normal application transport.
remoteproc- Linux remote-processor frameworkPlatform mechanism for loading firmware, powering, starting, stopping, and reporting crashes for an on-SoC remote core. It is relevant when the ACiM controller is a heterogeneous SoC processor rather than a PCIe card controller.
- Co-simulation
- Several models executing one protocolA host or QEMU machine model forwards MMIO, DMA, queues, and interrupts to a functional, cycle, RTL, or analog model so the complete software chain runs before silicon.
- ELF
- Executable and Linkable FormatA structured binary containing loadable segments, symbols, relocations, and an entry point. Linux executables, kernel modules, and controller firmware may all use ELF but target different machines and loaders.
- DTB
- Device Tree BlobBinary hardware description passed to Linux on many embedded systems. It declares platform devices, MMIO ranges, interrupts, clocks, resets, IOMMUs, reserved memory, and relationships such as remote processors.
Analog CIM, numerical formats, model execution, and accuracy
- ACiM
- Analog Compute-in-MemoryPerforms arithmetic through analog behavior inside or adjacent to memory arrays to reduce weight movement. AiCM, AIMC, analog CIM, and analog in-memory computing are overlapping spellings.
- MVM
- Matrix-Vector MultiplicationComputes y = Wx for one vector. In ACiM, weights map to cell conductances, inputs drive rows, column currents sum products, and converters plus digital reconstruction finish the operation.
- VMM
- Vector-Matrix MultiplicationOften used in analog literature for the same array operation viewed with a row-vector convention. Confirm tensor orientation instead of assuming VMM and MVM imply different hardware.
- GEMM
- General Matrix MultiplicationComputes C = alpha AB + beta C for matrices. A compiler normally decomposes GEMM into tiled MVMs, batches, partial sums, and digital accumulation.
- MAC
- Multiply-AccumulateMultiplies two values and adds the product into an accumulator. Marketing may count one MAC as one or two operations, so TOPS comparisons need a declared convention.
- Crossbar
- Intersecting row and column arrayMemory cells at intersections encode weights; driven rows and sensed columns exploit Ohm's and Kirchhoff's laws for parallel multiply and accumulation.
- WL / BL
- Wordline and bitlineRow and column conductors used to select, drive, program, and sense memory cells. Parasitic resistance and capacitance make position and array size matter.
- Conductance
- Electrical weight representationThe inverse of resistance, measured in siemens. Devices encode a weight through one or more conductance states, often using differential cells for signed values.
- DAC
- Digital-to-Analog ConverterConverts encoded digital inputs into voltage, current, pulse width, or bit-serial drive for an array. Some designs replace a conventional multibit DAC with simpler drivers and more cycles.
- ADC
- Analog-to-Digital ConverterDigitizes analog column results. Resolution, sampling rate, sharing, range, and calibration can dominate ACiM area, energy, latency, and accuracy.
- ENOB
- Effective Number of BitsThe useful converter resolution after measured noise and distortion. It is commonly lower than the nominal ADC bit count and varies with speed, range, and PVT.
- SNR
- Signal-to-Noise RatioThe ratio of desired signal power to noise power, usually in decibels. SNR helps characterize a path but does not directly equal model accuracy.
- Partial sum
- Incomplete dot-product resultProduced when a tensor is split across rows, columns, bit slices, arrays, or chiplets. It must be accumulated with declared precision and placement.
- NVM
- Non-Volatile MemoryRetains state without power. Flash, RRAM, and PCM are different NVM technologies with different programming, variation, endurance, density, and sensing behavior.
- NOR / NAND
- Flash memory organizationsNOR favors random access and simpler compute mapping; NAND favors density but uses string/page organization and more complex access. A roadmap claim does not make their ACiM backends interchangeable.
- RRAM
- Resistive Random-Access MemoryAlso called ReRAM; stores data through programmable resistance. Analog levels face device variability, nonlinear programming, drift, defects, and endurance limits.
- PCM
- Phase-Change MemoryStores state through material phases with different resistance. Multi-level programming, drift, write asymmetry, and temperature sensitivity shape its compute use.
- Write-verify
- Closed-loop cell programmingApplies a programming pulse, reads the cell, compares against a target, and repeats. It improves weight placement but costs time, energy, and endurance.
- Calibration
- Measured correction of hardware behaviorEstimates converter offsets, gains, array transfer functions, drift, defects, or per-tile scales and stores parameters used by compiler, firmware, or runtime.
- Drift
- Time-dependent state changeStored conductance or circuit response moves after programming. Monitoring, compensation, refresh, retraining, or fallback must be included in lifecycle cost.
- PVT
- Process, Voltage, and TemperatureManufacturing and operating conditions that change timing, analog transfer behavior, converter quality, retention, and calibration validity.
- SPICE
- Transistor-level circuit simulationModels electrical behavior and corners at high fidelity but limited scale. It complements, rather than replaces, architectural and task-level simulation.
- Quantization
- Map values to a finite set of levelsDefines precision, scale, zero point, rounding, clipping, and sometimes per-channel parameters. Analog error is additional to digital quantization error.
- INT8 / INT4
- 8-bit and 4-bit integer formatsDescribe logical tensor values, not automatically ADC ENOB or cell bits. A physical implementation may use several cells, slices, reads, or accumulations per logical value.
- ANA8
- Vendor analog-precision labelNot an industry-standard numeric type. Treat it as a product claim until input encoding, weight representation, accumulation, ENOB, calibration, and validation are specified.
- FP32 / BF16
- Floating-point formatsFP32 has 32 bits; bfloat16 keeps an FP32-like exponent with fewer fraction bits. A floating-point baseline must be named when reporting quantized or analog accuracy loss.
- PTQ
- Post-Training QuantizationConverts a trained model using calibration data without full retraining. It is fast but may struggle with sensitive layers, outliers, or analog nonidealities.
- QAT
- Quantization-Aware TrainingModels quantization effects during training or fine-tuning so weights adapt. Hardware-aware training may additionally inject measured analog behavior.
- QDQ
- Quantize/Dequantize operatorsGraph nodes that make scale, zero point, rounding, and precision transitions explicit. They describe numerical intent, not necessarily physical converter operations.
- Scale / ZP
- Quantization scale and zero pointMap real values to integer codes. Scale controls step size; zero point represents real zero in an asymmetric integer range.
- Clip / saturate
- Limit values to a representable rangeClipping is the chosen range restriction; saturation is output behavior at the limits. Both can create systematic model error and must be traced per layer.
- ECC
- Error-Correcting CodeRedundant coding that detects or corrects digital bit corruption. It does not generally correct continuous analog compute error or converter nonlinearity.
- YOLO
- You Only Look OnceFamily of single-stage object-detection models used here as the first end-to-end workload.
- NMS
- Non-Maximum SuppressionPostprocessing that removes redundant overlapping detections.
- mAP
- mean Average PrecisionObject-detection accuracy metric computed over classes and intersection-over-union criteria.
- LLM
- Large Language ModelTransformer model with large parameter and runtime-state requirements; referenced by the companion scale-out design.
- KV cache
- Transformer key/value cacheRuntime state retained for prior tokens so attention need not recompute them. Its capacity, bandwidth, update pattern, and placement differ from static model weights.
- Attention
- Content-dependent token mixingTransformer operation using query, key, and value tensors. Linear projections fit MVM arrays; softmax, dynamic KV access, reductions, and communication need other engines.
- Batch
- Requests processed togetherBatching can raise array utilization and throughput but consumes memory and may increase queueing and tail latency.
- p95 / p99
- Tail-latency percentilesThe time below which 95% or 99% of requests complete. They expose contention, calibration, fallback, and recovery stalls hidden by an average.
- FPS
- Frames Per SecondSteady-state image/video throughput. It must be paired with batch size, latency distribution, power, and task accuracy.
Open-source licensing, software inventory, and provenance
- SPDX
- Software Package Data ExchangeAn open standard for machine-readable software, component, license, security, and provenance information. SPDX records facts; it does not grant rights or replace license text and legal review.
- SBOM
- Software Bill of MaterialsAn inventory of components and relationships in a product or build. SPDX is one format for an SBOM, but an SPDX file may also describe other lifecycle information.
- SPDX ID
- SPDX license identifierA standardized short name such as
Apache-2.0. A source header likeSPDX-License-Identifier: Apache-2.0labels the applicable license. - Expression
- SPDX license expressionA machine-readable combination using operators such as
AND,OR, andWITH. The operator changes obligations and choice, so it is not decorative metadata. - Permissive
- License family with limited downstream conditionsCommon examples include Apache-2.0, MIT, and BSD licenses. Exact notice, attribution, patent, and redistribution terms still differ.
- Copyleft
- License conditions tied to redistribution or modificationSome licenses require corresponding source or same-license terms in defined circumstances. Scope varies, so architecture labels are not a legal conclusion.
- Apache-2.0
- Apache License 2.0A permissive license with copyright, notice, and express patent-license provisions. Preserve required notices and inspect separately licensed dependencies and assets.
- MIT / BSD
- Permissive license familiesGenerally short licenses centered on retaining copyright and permission notices, but variants differ and must be identified precisely.
- GPL / LGPL
- GNU copyleft license familiesGPL and Lesser GPL have different conditions and version choices. Kernel modules, userspace libraries, linking, modification, and distribution require component-specific review.
- NOTICE
- Distribution notice fileMay collect attribution required by Apache-licensed or third-party material. It does not replace the
LICENSEfile and should not be discarded during packaging. - Provenance
- Origin and transformation historyRecords repository, commit, author or supplier, license, patches, generator, model/dataset source, and build inputs so a shipped artifact can be reproduced and audited.
- Transitive dep.
- Dependency of a dependencyCode or data pulled indirectly into a build. Top-level repository licensing does not automatically describe the full dependency closure.
- Upstream
- Original or authoritative project lineContributing reusable changes upstream reduces long-term divergence, but product schedules and target confidentiality may require a maintained downstream boundary.
- Fork / vendor
- Maintained copy versus imported snapshotA fork has independent history; vendoring copies a dependency into the product tree. Both need ownership, update, security, license, and compatibility plans.
- DCO / CLA
- Contribution attestation or agreementA Developer Certificate of Origin is a contributor attestation; a Contributor License Agreement is a separate legal agreement. Project policy determines which is used.
- Model license
- Terms for weights, data, and outputsModel weights, training datasets, tokenizer assets, demo media, and code can have different licenses and usage restrictions; track each artifact independently.
- Hash / signature
- Integrity digest versus authenticity proofA cryptographic hash detects changed bytes; a digital signature binds a digest to a trusted key. Neither alone establishes license compliance.
- Repro build
- Reproducible buildA controlled process that produces byte-identical or verifiably equivalent outputs from declared sources, tools, dependencies, environment, and configuration.
Do not confuse
Similar words describe different contracts
Driver decision
Does an Analog CIM board need a user-mode driver (UMD) and kernel-mode driver (KMD)?
Terms here
UMD = User-Mode Driver
KMD = Kernel-Mode Driver
HAL = Hardware Abstraction Layer
UAPI = kernel/userspace contract
Best for: compiler and runtime CI
Best for: protocol exploration
Best for: mailbox or Programmed I/O (PIO) loopback
Best for: multi-process deployment
Subsystem choice: evaluate Linux drivers/accel before inventing a private device
framework. The accelerator subsystem exposes /dev/accel/accel* and reuses DRM infrastructure. A
small out-of-tree character driver is acceptable for early board enablement, but its UAPI should still be
designed as if it must survive old userspace and new kernels.
End-to-end architecture
The board is one backend beneath a stable compiler, artifact, and runtime
The simulator and hardware must consume the same executable semantics. Only the HAL transport changes. This keeps model correctness separate from PCIe, firmware, and analog variation bugs.
Terms here
API = callable source interface
ABI = binary layout and behavior
HAL = transport-neutral hardware interface
Firmware = device-resident control software
Model and application
PyTorch / ONNXGraph, tensors, shapes, QDQ, preprocessing
CompilerSupport, partition, lower, schedule, package
YOLO applicationCapture, policy, decode, NMS, results
Host userspace
Framework pluginONNX Runtime (ORT) Execution Provider (EP) capability and fallback boundary
RuntimeArtifact load, session, buffers, queues, events
User-Mode Driver (UMD) / HALHardware Abstraction Layer: discover, query, allocate, submit, wait, trace
Stable Linux user-space API (UAPI): ioctl + mmap + poll/eventfd + dma-buf
Host kernel
Kernel-Mode Driver (KMD) contextsOwnership, lifetime, permissions, Application Binary Interface (ABI)
Memory managerPin/import, IOMMU map, Input-Output Virtual Address (IOVA), fence
PCI driverBAR, DMA, MSI-X, reset, AER, firmware
PCIe contract: configuration + BAR MMIO + DMA + MSI-X
Device control
Endpoint + boot ROMIdentity, reset, mailbox, firmware load
Firmware schedulerValidate, queue, dispatch, complete, recover
Trace and healthCycles, bytes, stalls, saturation, faults
Device data plane
DMA + SRAMDirect Memory Access and Static RAM stage inputs, outputs, commands, and partial sums
Analog MVMMatrix-Vector Multiplication: program, encode, integrate, ADC, reconstruct
Digital enginesElementwise, resize, activation, reduction, fallback
target + program + constants + commands + debug mapcaps + contexts + buffers + queues + waits + faultsSQ/CQ + mailbox + opcodes + status + traceWhole-chain verification
Test one contract at a time before asking the complete board to run a model
QEMU is useful, but it is not an analog Compute-in-Memory (CIM) simulator by default. Use it to boot a supported host or controller CPU and its software. Add a custom device model, fake transport, or co-simulation bridge when the Kernel-Mode Driver (KMD), firmware, queue protocol, and analog model must run together.
Terms here
BSP = bootloader, kernel, device tree, and board support
rootfs = Linux userspace files
HAL = transport-neutral device interface
JTAG = low-level hardware debug port
Co-simulation = coordinated execution of two or more simulator models
- Host bootBoot the existing x86-64 or AArch64 Linux installation. Do not prepare an SD card unless the host platform itself requires one.
- InstallInstall the signed KMD, udev rules, controller firmware under the configured firmware search path, UMD/runtime, compiler, and diagnostics.
- DiscoverPCIe enumerates the endpoint. The KMD claims BARs, configures DMA masks and MSI-X, creates an IOMMU-protected context, and exposes a device node.
- Boot deviceIf a programmable controller exists, the KMD obtains firmware with the Linux firmware API, verifies compatibility, transfers it through BAR or DMA boot windows, starts it, and waits for READY. A fixed RTL controller instead publishes its ROM/logic ABI directly.
- RunThe application loads a compiled
.acxartifact and tensors through the runtime. The model image is usually session data, not permanently flashed firmware. - RecoverCollect host and device evidence before reset; then use context reset, Function Level Reset (FLR), secondary-bus reset, or power cycle according to fault scope.
- Build BSPCross-compile the bootloader, Linux kernel, Device Tree Blob (DTB), root filesystem, KMD, UMD/runtime, application, and board diagnostics.
- Deploy host imageWrite the vendor-defined boot image to SD, eMMC, UFS, or network boot media. Verify the physical target before using any imaging tool.
- Boot hostCapture bootloader and early kernel serial output. Confirm clocks, regulators, reset GPIOs, IOMMU, reserved memory, PCIe or on-chip interconnect, and device-tree bindings.
- Boot controllerUse Linux
remoteproc, a platform KMD, or a boot ROM to load the controller Executable and Linkable Format (ELF) image and create messaging/queue resources. - Feed inputThe host application receives camera, network, sensor, or storage input, preprocesses it, and submits buffers through the same runtime/HAL contract used in simulation.
- Field updateVersion and update host BSP, controller firmware, model artifact, and calibration independently, with an atomic rollback policy for compatible sets.
| Question | PCIe card | Embedded host SoC | Engineering rule |
|---|---|---|---|
| Make QEMU first? | Optional for host driver and protocol tests | Useful when QEMU supports the selected host CPU/board | Start with native functional tests; add QEMU where CPU/OS behavior matters. |
| What does QEMU simulate? | Host CPU/Linux plus only implemented virtual devices | Host or controller CPU and modeled peripherals | A generic QEMU machine does not contain the custom ACiM endpoint or analog physics. |
| Flash an SD card? | Normally no | Often yes, if SD/eMMC is the board boot source | The boot source comes from the board design and BSP, not from ACiM itself. |
| Build a host binary? | Yes: application, compiler tools, runtime/UMD, diagnostics, and possibly KMD module | Cross-compile when the target architecture differs from the development machine. | |
| Build a device kernel? | Build controller firmware and compute microcode/kernels if the architecture has programmable engines | Reserve "Linux kernel" for the host OS; call device payloads firmware or compute kernels. | |
| How does input reach ACiM? | Application buffer to UMD/KMD to PCIe DMA | Application buffer through shared memory, DMA, NoC, or PCIe | The runtime owns live buffer bindings; the compiler artifact owns logical dataflow. |
Name and version the deployable artifacts
| Artifact | Built for | Loaded by | Typical deployment | Required identity |
|---|---|---|---|---|
| Host boot image | Host CPU | Boot ROM / bootloader | SD, eMMC, UFS, SPI flash, or network; embedded only | BSP version, DTB, kernel config, rootfs hash |
| KMD | Host kernel and CPU | Kernel module loader or built into Linux | .ko, package, initramfs, or kernel image | Kernel release, UAPI major/minor, device IDs |
| Runtime / UMD / app | Host userspace CPU | Dynamic loader and service manager | Rootfs package, container, or development tree | Build ID, UAPI range, artifact ABI range |
| Controller firmware, if present | Device RISC-V/Arm/controller CPU; omitted for fixed RTL control | KMD, remoteproc, boot ROM, or JTAG in the lab | Firmware directory, dedicated partition, or RAM load | Firmware ABI, register-map version, image hash, entry point |
| Compute program | Device digital engines and ACiM arrays | Runtime and controller firmware | Inside .acx or a versioned kernel package | Opcode ISA, array geometry, precision, scratch requirements |
| Model artifact | Named accelerator target | Runtime session | Application data partition or model store | Compiler provenance, target ID, ABI, graph and constant hashes |
| Calibration / health profile | Chip, board, voltage, and temperature domain | Firmware/runtime under controlled policy | Protected per-device state with factory/default fallback | Chip serial, profile version, conditions, age, integrity hash |
The top-level build, boot, and test flow
1. Freeze contractsOne version setRegister map, queues, UAPI, firmware ABI, artifact ABI, memory map, fault and trace schema.
2. Build hostBSP + KMD + UMDCompile for the host CPU; retain symbols, manifests, reproducible presets, and test utilities.
3. Build deviceFirmware + kernelsLink the controller memory map; emit ELF, binary, map file, symbols, and image metadata.
4. EmulateHAL + QEMU + co-simProve CPU software, queues, DMA semantics, faults, reset, and numerical reference before a board.
5. Bring up transportEnumerate + bootPower, clock, reset, PCIe/interconnect, BAR, IOMMU, DMA, MSI-X, firmware READY.
6. Correlate workloadOne MVM to full modelCompare tensors, counters, timestamps, energy, temperature, calibration, and recovery.
Ten-stage verification ladder
Do not advance because a demo looks plausible. Advance when the stated output, timeout, error behavior, and evidence bundle pass automatically. Every later stage reruns the earlier contract tests.
0Contract lintGenerated headers agree; ABI sizes, IDs, endianness, reserved fields, and compatibility tests pass.
1Golden modelNative host functional HAL matches reference tensors and injects saturation, defects, drift, and timeouts.
2Queue modelFake PCIe/MMIO/DMA backend proves ownership, wraparound, ordering, interrupt, reset, and malformed descriptors.
3Host OS in QEMUBSP, KMD lifecycle, UAPI, service startup, permissions, and fake/custom device probe pass.
4Controller ISA simFirmware boots on a matching CPU/machine model; GDB reaches READY and queue unit tests complete.
5Host-device co-simKMD and firmware exchange real descriptors while the analog backend supplies deterministic MVM results and faults.
6RTL / FPGARegister, DMA, interrupt, boot, queue, and reset behavior correlates with the protocol model under stress.
7Board transportEnumeration, firmware boot, loopback DMA, MSI-X, IOMMU isolation, teardown, and recovery pass on silicon.
8One analog MVMRaw ADC, reconstructed output, calibration ID, temperature, and error distribution match the acceptance envelope.
9Model + operationsSubgraph and full model pass accuracy, latency, throughput, power, soak, reset, and support-bundle gates.
Choose the QEMU boundary deliberately
Fastest correctness and continuous integration loop
Generic virt alone cannot probe the real endpoint
CPU emulation does not reproduce analog variation
Highest pre-silicon fidelity; also highest integration cost
| Environment | It can prove | It cannot prove alone | Required exit evidence |
|---|---|---|---|
| Native functional model | Graph semantics, quantization, command construction, error policy | Kernel behavior, real timing, PCIe, analog distributions | Golden tensor diff and deterministic fault tests |
| QEMU host | Host architecture, Linux boot, KMD/UAPI lifecycle, packages | Unimplemented ACiM device and real DMA/interrupt timing | Automated boot/probe/open/submit/reset test |
| QEMU controller | Firmware CPU code, exceptions, memory map, modeled IRQ and timer behavior | Custom engines and analog physics unless explicitly modeled | Firmware unit tests, READY handshake, GDB-readable fault state |
| Cycle/RTL co-sim | Protocol ordering, contention, queue and engine timing, RTL behavior | Unmodeled package/board effects and silicon variation | Trace correlation and bounded cycle deltas |
| Real board | Physical link, DMA, interrupts, converters, arrays, power and thermal behavior | Broad corner coverage from one device | Population-aware correctness, performance, soak, and recovery reports |
# 1. Fast host loop: compiler/runtime/fake HAL
cmake --preset host-debug
cmake --build --preset host-debug
ctest --preset host-debug --output-on-failure
./build/host-debug/bin/acim-run --hal functional --model smoke.acx
# 2. Embedded host Linux in QEMU. This boots the host BSP only.
qemu-system-aarch64 -M virt -cpu cortex-a72 -m 4096 -nographic \
-kernel out/host/Image -initrd out/host/rootfs.cpio.gz \
-append "console=ttyAMA0 earlycon"
# A KMD can probe ACiM only after implementing a custom/fake endpoint, for example:
# -device acim-pcie,backend=unix:/tmp/acim-model.sock
# "acim-pcie" above is a proposed project device model, not built into QEMU.
# 3. Device controller firmware. The machine must match the real controller map.
qemu-system-riscv64 -M acim-controller -nographic -bios none \
-kernel out/device/acim-fw.elf -S -gdb tcp::1234
riscv64-unknown-elf-gdb out/device/acim-fw.elf \
-ex "target remote :1234"
# "acim-controller" is also a proposed custom machine name. Until it exists,
# run firmware libraries as host tests or use the vendor ISA/RTL simulator.
Embedded image safety: use Buildroot, Yocto, or the vendor BSP to produce deployable images,
and use the board vendor's imaging procedure for SD/eMMC. Record the output image hash and confirm the target
device path before writing. The files in a Buildroot output/images/ directory are deployment
artifacts; intermediate build directories are not.
Board boot and capability handshake
1Power and resetRails, clocks, reset, refclk, link training, boot straps; UART emits a ROM identifier.
2EnumerateHost finds endpoint or platform device; IDs, BAR sizes, IOMMU, DMA width, MSI-X, and AER match the specification.
3Start control planeKMD/remoteproc loads and starts compatible firmware when present; otherwise it verifies the fixed ROM/RTL control ABI and optional microcode.
4NegotiateFirmware reports READY, boot epoch, ABI, capabilities, memory/array health, calibration state, trace clock, and fatal status.
5Publish deviceKMD enables queues and userspace node; UMD queries caps, creates context, verifies artifact compatibility, and starts a session.
| Checkpoint | Host evidence | Device evidence | Timeout action |
|---|---|---|---|
| Physical discovery | lspci -nnvv or platform-device sysfs, link width/speed, AER | ROM UART banner or JTAG halt state | Inspect power, clocks, reset, straps, refclk, lanes, DTB |
| KMD probe | Driver-bound sysfs path, BAR resources, DMA mask, MSI-X vectors, IOMMU group | Mailbox visible, ROM state readable | Check IDs, BAR mapping, endian/access width, IOMMU and module log |
| Firmware READY | Requested image/hash, transfer bytes, boot command, deadline and response | Last boot stage, exception registers, crash buffer, firmware ABI | Halt with JTAG/GDB before reset; dump mailbox and load memory |
| Userspace open | Device node, permissions, UAPI query, context and queue IDs | Context table and queue generation | Compare UMD/KMD ABI, udev rules, health and reset epoch |
One inference request and completion
1. ApplicationPrepare inputCapture/preprocess tensor; assign run, request, graph, and correlation IDs.
2. Runtime / UMDBind buffersLoad
.acx, select compatible profile, allocate/import Buffer Objects (BOs), create dependencies.3. KMDMap and submitValidate handles/ranges; map scatter-gather pages to IOVA; publish SQ entry; ring MMIO doorbell.
4. FirmwareValidate and scheduleCheck generation, context, opcode, bounds, program, health, deadline, and dependency state.
5. ACiM data planeTransfer and computeDMA to SRAM; DAC/input encoding; array MVM; ADC; reconstruction; digital operators and partial sums.
6. CompletionReturn resultDMA output; write CQ; raise MSI-X; KMD wakes poll/event; UMD resolves event; app consumes tensor.
| Boundary | Control path | Data path | Ordering proof |
|---|---|---|---|
| App to UMD | API call, session, event, timeout | Tensor views or imported dma-buf | Buffer lifetime and dependency test |
| UMD to KMD | ioctl, mmap, poll/eventfd | Buffer handles resolved to pinned/imported pages | UAPI conformance, process-exit and malicious-input tests |
| KMD to device | SQ, CQ, doorbell, mailbox, MSI-X | DMA over IOVA mappings | Publish descriptor before doorbell; write result before CQ and interrupt |
| Firmware to engines | Command processor, dependencies, watchdog | SRAM/NoC streams, array inputs/outputs, partial sums | Cycle trace with command and tile IDs |
| Analog to digital | Profile, calibration, range and retry policy | DAC code, cell current/charge, ADC code, reconstructed value | Raw/reconstructed sample capture and golden-model correlation |
Capture one causally connected log, not five unrelated files
1. IdentifyCreate run manifestRun/correlation IDs, git commits, hashes, ABIs, serial, clocks, voltage, temperature, calibration.
2. Arm hostTracing before submitApp JSONL, Tracy, UMD events, KMD tracepoints/ftrace, kernel journal, PCIe AER and IOMMU faults.
3. Arm deviceUART + trace ringFirmware stages, queue events, DMA, engine cycles, counters, ADC flags, watchdog and crash buffer.
4. Run workloadKnown request IDsWarm-up separately; execute fixed inputs; preserve first fault; avoid reset until evidence is drained.
5. CorrelateJoin clocks and IDsUse synchronization markers and an affine clock transform; align host, firmware, engine, and instrument timelines.
6. DiagnoseFind first divergenceCompare expected/observed state, tensors, queue positions, bytes, cycles, faults, power and temperature.
support/run-20260712T103000Z-7f3a/
manifest.json # versions, hashes, ABIs, topology, serial, environment
host/app.jsonl # request and graph correlation IDs
host/runtime.jsonl # sessions, buffers, submissions, waits, fallback
host/tracy.tracy # host spans and optional imported device zones
host/kernel.dat # trace-cmd/ftrace events: acim, irq, iommu, sched
host/journal-kernel.txt # probe, firmware, reset, AER and IOMMU messages
host/pci.txt # config space, link, BAR and AER snapshot
device/uart.txt # ROM and early firmware boot
device/trace.bin # timestamped firmware/engine trace schema
device/crash.bin # preserved first-fault context
device/counters.json # DMA, NoC, SRAM, array, ADC, DSP, stalls, energy
model/artifact-manifest.json # target, graph, constants, compiler and debug-map hashes
model/input.npz # reproducible non-sensitive test input
model/output-device.npz # observed tensors or selected checkpoints
analysis/timeline.json # normalized clock domain and decoded events
analysis/tensor-diff.json # golden comparison and first divergent operation
# Proposed SDK command: implementation should gather data without manual copying.
acim-smi support collect --run-id 7f3a --include-trace --output support/
# Useful host evidence while reproducing a failure.
lspci -nnvv -s "$ACIM_BDF"
journalctl -k --since "$RUN_START"
trace-cmd record -e acim -e irq -e iommu ./acim-smoke --run-id 7f3a
# Device evidence should be read through a versioned KMD/firmware interface.
acim-smi trace snapshot --run-id 7f3a
acim-smi health show --json
Clock correlation: a host timestamp and device cycle counter are different clocks. Emit
repeated synchronization pairs around the run, fit host_ns = a * device_cycles + b, report fit
error, and keep raw timestamps. A single offset hides drift and makes a long trace look causally wrong.
Diagnose from the first failed boundary
| Observed symptom | First layer to inspect | Evidence to compare | Focused experiment |
|---|---|---|---|
| No device enumeration | Board power/reset/PCIe or platform DTB | Rails, clocks, reset, refclk, link state, boot straps, root-port AER | Hold controller in ROM; reduce link speed/width; inspect endpoint with JTAG |
| Enumerates but no device node | KMD probe | IDs, BAR resources, DMA mask, MSI-X, IOMMU group, firmware request | Bind driver manually in a lab; enable dynamic debug and probe tracepoints |
| Device node exists but open fails | UMD/KMD contract | Permissions, UAPI major/minor, device health, reset generation | Run capability query as the service user; compare expected ioctl sizes |
| Submit succeeds but firmware never starts | SQ publication and doorbell | Producer/consumer indices, descriptor bytes, barriers, MMIO readback, generation | Submit one no-op command; poll rather than interrupt; inspect queue memory on both sides |
| Device completes but host never wakes | CQ and MSI-X path | CQ entry visibility, interrupt cause/mask/vector, KMD handler and wakeup | Use interrupt self-test; switch to CQ polling; verify write-before-interrupt ordering |
| DMA corruption or IOMMU fault | Buffer mapping | IOVA, scatter/gather segments, length/alignment, direction, cache ownership | Use one coherent page and loopback; add guard patterns; reduce transfer size |
| Stable transport but wrong tensor | Compiler/artifact/analog boundary | Layout, quantization, scale/zero point, program/profile ID, calibration, ADC saturation, remap | Run one MVM with fixed operands; capture raw ADC and reconstructed values; bypass fusion |
| Correct but slow | Timeline and counters | Host queue, H2D, SRAM/NoC, DAC, MVM, ADC, reconstruction, DSP, D2H and postprocess time | Sweep batch/shape; force resident weights; disable retries; compare full and partial utilization |
| Intermittent after many runs | Lifetime, thermal, drift, and reset | Epoch, queue wrap, leaked mappings, temperature, voltage, calibration age, corrected faults | Run soak with periodic known-answer tests and fault injection; preserve the first failing bundle |
Compile and deploy flow
Do not make MLIR responsible for PCIe
MLIR selects and encodes legal hardware work. The runtime resolves physical buffers and live device state. The KMD protects host resources. Firmware owns device scheduling. Keeping these decisions separate permits the same artifact to run in simulation and on a board.
Terms here
ONNX = model graph format
MLIR = compiler lowering infrastructure
ORT = ONNX Runtime
EP = framework backend plugin
1. ImportONNX graphValidate opset, infer shapes, normalize layouts and constants.
2. AnalyzeTarget supportMatch operators, precision, tensor bounds, calibration, and fallback.
3. PartitionAnalog islandsCluster resident dense or convolution work; minimize analog/digital crossings.
4. LowerDevice programsMap arrays, digital kernels, transfers, dependencies, and scratch memory.
5. Package
.acx artifactCode, constants, profiles, commands, ABI, hashes, and graph debug map.6. ExecuteHAL backendSimulator or UMD/KMD/firmware uses identical buffer and command semantics.
| Decision | Compiler owns | Runtime owns | Device owns |
|---|---|---|---|
| Operator legality | Static support and diagnostic reason | Capability/firmware compatibility check | Descriptor validation before dispatch |
| Memory | Logical buffers, lifetimes, offsets, alignment | Buffer objects, imports, bindings, reuse | SRAM allocation, DMA windows, bounds check |
| Analog mapping | Array slices, precision, profile ID, alternatives | Select live profile and healthy placement | Program ID, calibration table, remap, saturation policy |
| Scheduling | Dependencies and legal overlap | Batching, queue and latency policy | Resource arbitration, watchdog, completion order |
| Failure | Fallback-capable variants | Retry, rebind, digital fallback, session loss | Fault isolation, abort, reset and health update |
First model dataflow
A YOLO demo should prove the whole hybrid path, not only one analog convolution
This placement is a proposed starting point, not a claim about an iHW or Mythic product. The exact YOLO version, supported operators, array geometry, digital engines, precision, and SRAM capacity must decide the final partition. Keep host fallback available until each device segment passes differential tests.
Terms here
BO = Buffer Object
H2D = Host-to-Device transfer
D2H = Device-to-Host transfer
NMS = detection deduplication
mAP = detection accuracy metric
HostCapture + preprocessDecode, resize/letterbox, normalize, quantize, pack layout.
PCIeInput DMAImported or allocated input BO; one H2D transfer per frame.
Device analogBackbone ConvResident weights, calibrated array MVM, ADC and reconstruction.
Device digitalNeck transformsActivation, resize, concat, routing, unsupported operators.
Device mixedHead projectionsAnalog eligible convolution/linear plus digital decode preparation.
PCIeOutput DMAReturn compact logits/boxes, not intermediate feature maps.
HostDecode + NMSDequantize, box transform, confidence filter, NMS, annotate.
Partition rule: one analog operator surrounded by host fallback can be slower than CPU/GPU execution because every boundary moves a feature map over PCIe. Prefer the largest accuracy-safe contiguous device island, keep weights resident across frames, and transfer only input and final output once the device path is validated.
| YOLO stage | Preferred placement | Compiler proof | Runtime/board proof |
|---|---|---|---|
| Input preprocessing | Host initially; device later if a validated image engine exists | Exact resize, padding, color order, normalization, quantization | Golden tensor immediately before first device operator |
| Conv + static weights | Analog when dimensions, precision, and accuracy are legal | Weight slicing, sign mapping, bias, scale, activation fusion, array utilization | Program/verify result, calibration profile, saturation and error counters |
| Activation / elementwise | Digital vector engine or firmware kernel | Quantization domain and rounding are explicit | Cycle and numerical checkpoints at each partition boundary |
| Resize / concat / route | Digital engine; avoid host round-trip | Layout and memory plan fit SRAM and DMA rules | No hidden copy; measured bytes and stall reason |
| Detection head | Hybrid; target profile selects analog-eligible projections | Tensor shapes and output scale remain compatible with decode | Layer error distribution plus end-to-end mAP on full validation set |
| Decode / NMS | Host for the Minimum Viable Product (MVP), device only with a measured benefit | Model-specific anchors/strides and thresholds packaged as metadata | Deterministic postprocess and identical evaluation configuration |
T_frame = T_pre + T_H2D + T_queue + T_device + T_D2H + T_postReport each term, median, and tail latency.FPS_steady <= 1 / max(stage_time)Frames Per Second (FPS): with pipelining, the slowest non-overlapped stage limits throughput.coverage = device_ops / supported_candidate_opsAlso report bytes crossing every partition, not only operator count.accuracy_delta = mAP_device - mAP_referenceUse the same dataset, preprocessing, decode, and thresholds.Platform requirements
Freeze a host profile and device profile before writing the KMD
PCIe generation, lane width, BAR sizes, DMA address width, queue count, and memory capacity are target facts, not values to infer from a demo. Encode them in machine-readable target and capability records.
Terms here
IOMMU = DMA translation and isolation
IOVA = device-visible mapped address
FLR = PCIe function reset
AER = PCIe error reporting
- CPU and OSDocument x86-64 and/or AArch64, distribution, kernel range, page size, endianness, and 32-bit userspace policy.
- PCIe rootRequired link generation and width, 64-bit BAR support, MSI-X, AER, hot reset/FLR behavior, and link retraining policy.
- IOMMUEnable Intel Virtualization Technology for Directed I/O (VT-d), AMD-Vi, or an Arm System Memory Management Unit (SMMU) for isolation. Qualify IOVA width, page sizes, and IOMMU group topology.
- MemoryLocked-memory limits, pinned-memory budget, scatter/gather limits, alignment, Non-Uniform Memory Access (NUMA) node, and optional hugepage policy.
- BuildCompiler, CMake/Ninja, kernel headers, Dynamic Kernel Module Support (DKMS) or in-tree build, packaging, symbols, sanitizers, and reproducible toolchain image.
- Accessudev rules, device group, least-privilege permissions, firmware search path, module signing, and secure-boot policy.
- Observability
lspci, sysfs, debugfs or tracefs, dynamic debug, ftrace, perf, AER counters, and support-bundle command. - Media interopOptional dma-buf import for camera/display pipelines; define synchronization and supported exporters explicitly.
- IdentityVendor/device/subsystem IDs, revision, serial, board Stock Keeping Unit (SKU), capability bits, hardware ABI, and manufacturing state.
- PCIe endpointBAR type/size, DMA address width, maximum payload/read request, MSI-X vectors, FLR, AER, and power states.
- BootROM state, mailbox, firmware image format, authenticity check, load window, entry point, timeout, rollback, and recovery mode.
- QueuesSubmission/completion entry formats, count, depth, alignment, ownership, generation, doorbells, interrupt moderation, and reset.
- MemorySRAM banks, address map, DMA engines, segment limits, alignment, coherency, ECC, bandwidth counters, and reserved regions.
- ComputeAnalog macro geometry, DAC/ADC modes, accumulators, digital engines, supported operators, precision, and scheduling constraints.
- ReliabilityRow/column/tile fault map, temperature/voltage sensors, watchdog, saturation flags, ECC, calibration age, and reset domains.
- DebugTrace SRAM, timestamp domain, correlation ID, performance counters, crash dump, firmware log, and safe register snapshot.
PCIe endpoint implementation
Start with a small versioned control BAR, DMA rings, and MSI-X
The layout below is a proposed v0 example for RTL, firmware, KMD, simulator, and documentation to implement from one register specification. It is not a disclosed register map for any company. Generate C, C++, SystemVerilog, simulator, and documentation definitions from the same schema.
Terms here
BAR = device address region
MMIO = register access by CPU load/store
DMA = bulk device/host transfer
MSI-X = multi-vector PCIe interrupts
RO/RW/WO/RW1C = register access modes
readl/writel + cpu_to_leXX/leXX_to_cpuBASE_LO, BASE_HI, then QUEUE_ENABLEwritel(...); readl(DEVICE_STATUS);submission.generation == device.generationspec/registers.yaml: owner + access + side_effectmajor mismatch -> -EPROTOCommand and queue ABI
Descriptors carry handles and bounded references, not trust
A KMD resolves process-owned buffer handles into IOVAs. Firmware accepts only ranges registered in the context table. Keep fixed-width fields, natural alignment, explicit reserved fields, and static size checks. Add a new ioctl or opcode for new semantics instead of silently changing an old one.
Terms here
SQ = Submission Queue
CQ = Completion Queue
ABI = binary command contract
IOVA = mapped DMA address
Doorbell = MMIO queue notification
/* include/hw/acim_queue_v0.h - generated from the protocol schema */
struct acim_sq_entry_v0 {
__le16 opcode; /* EXECUTE, PROGRAM, VERIFY, COPY, TRACE, ABORT */
__le16 flags; /* interrupt, priority class, retry permission */
__le32 command_bytes; /* bounded command-buffer length */
__le64 context_id; /* firmware-created context, never a pointer */
__le64 command_iova; /* IOVA already registered for this context */
__le64 user_cookie; /* opaque value returned in completion */
__le64 deadline_ns; /* relative or zero; clock semantics are specified */
__le32 generation; /* rejects work submitted before reset */
__le32 crc32; /* optional transport corruption check */
};
struct acim_cq_entry_v0 {
__le64 user_cookie;
__le32 status; /* OK, INVALID, FAULT, TIMEOUT, RESET, RETRY */
__le32 fault_code;
__le64 device_cycles;
__le64 completed_bytes;
__le32 generation;
__le32 reserved0;
};
static_assert(sizeof(struct acim_sq_entry_v0) == 48);
static_assert(sizeof(struct acim_cq_entry_v0) == 40);/* include/uapi/linux/acim_accel.h - no C pointers or implicit padding */
#define ACIM_IOCTL_QUERY_CAPS _IOWR('A', 0x00, struct acim_query_caps)
#define ACIM_IOCTL_CREATE_CONTEXT _IOWR('A', 0x01, struct acim_create_context)
#define ACIM_IOCTL_CREATE_BO _IOWR('A', 0x02, struct acim_create_bo)
#define ACIM_IOCTL_IMPORT_DMABUF _IOWR('A', 0x03, struct acim_import_dmabuf)
#define ACIM_IOCTL_CREATE_QUEUE _IOWR('A', 0x04, struct acim_create_queue)
#define ACIM_IOCTL_SUBMIT _IOWR('A', 0x05, struct acim_submit)
#define ACIM_IOCTL_WAIT _IOWR('A', 0x06, struct acim_wait)
#define ACIM_IOCTL_QUERY_FAULT _IOWR('A', 0x07, struct acim_query_fault)
#define ACIM_IOCTL_RESET_CONTEXT _IOWR('A', 0x08, struct acim_reset_context)
struct acim_submit {
__u32 size; /* must equal the known v0 structure size */
__u32 flags;
__u64 context_handle;
__u64 queue_handle;
__u64 command_bo_handle;
__u64 command_offset;
__u64 command_bytes;
__u64 user_cookie;
__u64 reserved[4]; /* userspace writes zero; kernel validates zero */
};| Object | UMD representation | KMD proof | Firmware proof | Lifetime ends |
|---|---|---|---|---|
| Context | Opaque handle tied to an open device FD | Owner process/file, credentials, generation, quotas | Valid context table entry and allowed engine mask | FD close, explicit destroy, fatal reset |
| Buffer object | Handle, size, usage, optional CPU map | Allocation/import, pages, DMA direction, IOMMU map, reference count | IOVA range belongs to context and command access is in bounds | All dependent fences signal, then unmap/unpin |
| Queue | Handle plus event or pollable FD | Depth, ring memory, MSI-X vector, context, reset epoch | Base, size, owner, head/tail and generation are valid | Quiesce, drain/abort, disable, free IRQ/ring |
| Artifact/program | Loaded module and binding table | Header, ABI, size, hashes, BO access and firmware compatibility | Opcode, section bounds, program ID, target/profile compatibility | Last in-flight execution finishes and program is evicted |
| Fence/event | Monotonic sequence or opaque event | Queue/context membership, timeout, cancellation, wakeup | Exactly one terminal CQ record per accepted submission | Waiter releases it after completion or device loss |
UMD, KMD, and firmware split
Put mechanisms at the lowest layer that can enforce them
Terms here
Runtime = model execution orchestration
UMD = userspace device library
KMD = privileged host driver
Firmware = device-side scheduler and control
| Responsibility | Runtime / UMD | KMD | Firmware | Do not do |
|---|---|---|---|---|
| Discovery | Enumerate device nodes and select compatible target | Bind PCI IDs, publish capabilities and health | Report hardware/firmware ABI and engine inventory | Infer capabilities from board name |
| Artifact | Parse semantic metadata, choose variants, bind buffers | Validate top-level sizes, compatibility and access | Validate every command and program section used | Trust compiler-generated bytes because they are signed by the same process |
| Memory | Choose usage, reuse, import, copy and synchronization | Allocate/pin/import, map to IOMMU, enforce context and quota | Use only registered IOVA windows and declared access | Expose physical addresses to userspace or firmware commands |
| Submission | Batch commands, choose queue, attach trace and deadline | Validate handles and publish protected descriptors | Arbitrate resources, execute and complete | Let userspace rewrite a descriptor after validation |
| Interrupt | Wait through poll/eventfd and drain completions | Own MSI-X, acknowledge cause, schedule completion work, wake waiters | Write CQ before MSI-X and moderate according to contract | Depend on a userspace process to mask a fatal interrupt |
| Analog state | Select compatible characterization and retry/fallback policy | Serialize privileged programming/reset and expose health | Apply calibration, remap, program/verify, monitor saturation/drift | Treat calibration files as ordinary unvalidated model constants |
| Recovery | Cancel work, rebuild session, recompile/rebind or fall back | Quiesce DMA, reset context/function, coordinate AER and device loss | Abort engines, capture fault, reset domains and increment generation | Continue using handles from before reset |
Tenstorrent lesson to adapt: Tenstorrent publishes separate tt-kmd and
tt-umd repositories; its UMD requires the KMD and handles IOMMU/host-memory configuration. Reuse
the separation, testing habits, packaging ideas, and observability patterns. Do not copy its Tensix-specific
address spaces, TLB model, queue protocol, or ioctl ABI into an Analog CIM device whose hardware contract is different.
Kernel-mode driver (KMD) skeleton
Probe in dependency order; remove in reverse after quiescing DMA
The skeleton shows order and failure ownership, not drop-in production code. Select current managed PCI APIs for the supported kernel range, implement every unwind path, and test probe failure after each resource acquisition.
Terms here
KMD = Kernel-Mode Driver
IRQ = Interrupt Request
DMA = Direct Memory Access
Probe = Linux device initialization callback
static int acim_probe(struct pci_dev *pdev,
const struct pci_device_id *id)
{
struct acim_dev *adev;
const struct firmware *fw = NULL;
int ret, i;
ret = pcim_enable_device(pdev);
if (ret)
return ret;
ret = pcim_iomap_regions(pdev, BIT(0), "acim");
if (ret)
return ret;
adev = devm_kzalloc(&pdev->dev, sizeof(*adev), GFP_KERNEL);
if (!adev)
return -ENOMEM;
adev->bar0 = pcim_iomap_table(pdev)[0];
pci_set_drvdata(pdev, adev);
ret = acim_read_and_validate_caps(adev); /* ROM-visible; no DMA yet */
if (ret)
return ret;
if (adev->caps.dma_bits != 32 && adev->caps.dma_bits != 64)
return -EPROTO;
ret = dma_set_mask_and_coherent(&pdev->dev,
DMA_BIT_MASK(adev->caps.dma_bits));
if (ret)
return dev_err_probe(&pdev->dev, ret, "unsupported DMA mask\n");
pci_set_master(pdev); /* device may now initiate DMA */
ret = acim_alloc_admin_rings(adev); /* dma_alloc_coherent */
if (ret)
goto err_clear_master;
ret = pci_alloc_irq_vectors(pdev, 1, adev->caps.max_vectors,
PCI_IRQ_MSI | PCI_IRQ_MSIX);
if (ret < 0)
goto err_free_rings;
adev->nr_vectors = ret;
for (i = 0; i < adev->nr_vectors; ++i) {
ret = request_threaded_irq(pci_irq_vector(pdev, i),
acim_irq, acim_irq_thread, 0,
"acim", adev);
if (ret)
goto err_free_irqs;
}
ret = request_firmware(&fw, adev->caps.firmware_name, &pdev->dev);
if (ret)
goto err_free_irqs;
ret = acim_boot_firmware(adev, fw); /* reset, load, hash, start, ABI */
release_firmware(fw);
if (ret)
goto err_free_irqs;
ret = acim_register_accel_or_chardev(adev); /* userspace opens only now */
if (ret)
goto err_quiesce;
return 0;
err_quiesce:
acim_quiesce_and_reset(adev);
err_free_irqs:
while (i--)
free_irq(pci_irq_vector(pdev, i), adev);
err_free_vectors:
pci_free_irq_vectors(pdev);
err_free_rings:
acim_free_admin_rings(adev);
err_clear_master:
pci_clear_master(pdev);
return ret;
}1. BlockStop new opens/submitsMark device removing or lost; wake waiters with a terminal error.
2. QuiesceMask and drainDisable queues, stop engines and DMA, synchronize IRQ handlers.
3. ReclaimFinish ownershipSignal/cancel fences, unmap streaming DMA, detach dma-buf and unpin pages.
4. ReleaseIRQ and ringsFree vectors, coherent control memory, firmware state and device node.
5. DisableClear bus masterConfirm no bus transaction remains before managed BAR/device release.
Important: do not enable processing engines or expose a device node immediately after
pci_set_master(). First validate the register ABI, allocate protected control rings, install
interrupts, boot compatible firmware, and complete a health handshake. A failed firmware boot is a detected
device that is not ready for userspace.
User-mode driver (UMD) skeleton
Expose a transport-neutral HAL to the runtime
The application should not issue ioctls directly. The UMD translates stable SDK objects into Linux UAPI objects and can later support VFIO, simulator, Windows, or an SoC transport without changing model execution.
Terms here
UMD = User-Mode Driver
HAL = Hardware Abstraction Layer
API = callable source interface
ioctl = Linux device-control system call
AcimStatus acimEnumerate(AcimDeviceInfo *items, size_t *count);
AcimStatus acimOpen(uint32_t index, const AcimOpenOptions *, AcimDevice **out);
AcimStatus acimQueryCaps(AcimDevice *, AcimCaps *out);
AcimStatus acimCreateContext(AcimDevice *, const AcimContextDesc *, AcimContext **out);
AcimStatus acimCreateBuffer(AcimContext *, const AcimBufferDesc *, AcimBuffer **out);
AcimStatus acimImportDmaBuf(AcimContext *, int fd, const AcimImportDesc *, AcimBuffer **out);
AcimStatus acimMapBuffer(AcimBuffer *, AcimMapMode, void **ptr);
AcimStatus acimFlushBuffer(AcimBuffer *, size_t offset, size_t bytes);
AcimStatus acimLoadExecutable(AcimContext *, const void *acx, size_t bytes,
AcimExecutable **out);
AcimStatus acimCreateQueue(AcimContext *, const AcimQueueDesc *, AcimQueue **out);
AcimStatus acimSubmit(AcimQueue *, const AcimSubmission *, AcimEvent **out);
AcimStatus acimWait(AcimEvent *, uint64_t timeout_ns, AcimCompletion *out);
AcimStatus acimQueryFault(AcimContext *, AcimFaultReport *out);
AcimStatus acimResetContext(AcimContext *);
void acimReleaseEvent(AcimEvent *);
void acimReleaseQueue(AcimQueue *);
void acimReleaseExecutable(AcimExecutable *);
void acimReleaseBuffer(AcimBuffer *);
void acimReleaseContext(AcimContext *);
void acimClose(AcimDevice *);immutable_caps != live_stateone owner per mutable queue tailO_CLOEXEC + PID generation checkWAIT_TIMEOUT != DEVICE_HANGstatus + structured detailsame artifact, buffers, events, errorsDMA and interrupt dataflow
Map only for the device lifetime of the transfer, then complete ownership
Use coherent DMA for small long-lived queue/control structures and streaming mappings or managed buffer objects for payloads. The DMA address returned by the kernel is a device bus address, not a CPU physical address.
Terms here
DMA = Direct Memory Access
IOMMU = address translation and isolation
IOVA = device-visible DMA address
MSI-X = completion/fault notification
1. Create/importBuffer objectKMD allocates or imports pages and records owner, size, usage and direction.
2. MapIOMMU / DMA APICreate IOVA or scatter/gather mapping; check every mapping result.
3. PublishValidated descriptorWrite command and ring fields, transfer ownership, then ring doorbell.
4. ExecuteDMA + enginesH2D stage, analog/digital compute, D2H result; enforce range and direction.
5. CompleteCQ + MSI-XDevice writes completion before interrupt; KMD acknowledges and wakes waiter.
6. ReclaimFence then unmapCPU reads result only after completion/sync; unmap, detach, or retain for reuse.
/* KMD producer: descriptor fields before ownership and MMIO notification */
sq[idx].command_iova = cpu_to_le64(cmd_iova);
sq[idx].command_bytes = cpu_to_le32(cmd_bytes);
sq[idx].user_cookie = cpu_to_le64(cookie);
sq[idx].generation = cpu_to_le32(adev->generation);
dma_wmb();
WRITE_ONCE(sq[idx].owner, ACIM_OWNER_DEVICE);
writel(new_tail, adev->bar0 + ACIM_SQ_DOORBELL(qid));
/* KMD completion path: ownership before completion fields */
if (READ_ONCE(cq[idx].owner) == ACIM_OWNER_HOST) {
dma_rmb();
complete_submission(le64_to_cpu(cq[idx].user_cookie),
le32_to_cpu(cq[idx].status));
}
/* Before reset/teardown, force a posted MMIO write to reach the endpoint. */
writel(ACIM_CONTROL_QUIESCE, adev->bar0 + ACIM_DEVICE_CONTROL);
readl(adev->bar0 + ACIM_DEVICE_STATUS);
Ordering boundary: coherent memory does not remove the need for barriers. dma_wmb()
and dma_rmb() order shared DMA memory; they do not themselves order arbitrary MMIO. Use the kernel
MMIO accessors and the hardware protocol's readback rule. MSI/MSI-X is preferred because PCI ordering ensures
the device's preceding memory writes arrive before the interrupt-generating write.
First inference sequence
One correlation ID follows the frame across every boundary
The sequence assumes weights were programmed and verified during session creation. Do not erase/program flash weights for every frame. A reset or changed placement/calibration epoch invalidates the resident program binding.
Terms here
ORT = ONNX Runtime
BO = Buffer Object
SQ/CQ = command/completion rings
IRQ = Interrupt Request
Application / ONNX Runtime (ORT)
Runtime / User-Mode Driver (UMD)
Kernel-Mode Driver (KMD)
Firmware
Direct Memory Access (DMA) + ACiM engines
01
Run(frame, correlation_id) -> bind executable and input
02
CREATE/IMPORT BO + validate context ->
03
SUBMIT handles, offsets, sizes, deadline ->
04
Validate, map IOVA, publish SQ, doorbell ->
05
Validate command, reserve SRAM/engines, start DMA ->
06
H2D, analog MVM, ADC/reconstruct, digital stages, D2H
07
<- CQ record then MSI-X
08
<- IRQ ack, drain CQ, signal event/fence
09
<- completion, output BO, counters, analog status
10
Decode/NMS; release or recycle event and buffers
Horizontal direction shows request or completion ownership; empty lanes remain active but do not own that transition.
Firmware implementation
Boot establishes trust; the scheduler validates every submission again
Terms here
ROM = immutable first-stage boot code
Firmware = device-resident scheduler/control code
ABI = host/device binary contract
Generation = reset epoch rejecting stale work
ROMReset-safe stateDMA off, IRQ masked, mailbox available, immutable identity readable.
KMDLoad imageRequest firmware, validate host-side container, stage image and command ROM.
ROMAuthenticate + startVerify image according to product security policy, initialize SRAM and engines.
FirmwarePublish ABIVersion, build hash, capabilities, generation, health, queue requirements.
KMDEnable userspaceNegotiate compatible ABI, run admin loopback, then register the device node.
for (;;) {
wait_for_doorbell_or_admin_event();
while (sq_has_entry()) {
SqEntry e = sq_acquire();
Validation v = validate_entry(e, device_generation, context_table);
if (!v.ok) {
cq_complete(e.cookie, STATUS_INVALID, v.fault_code);
continue;
}
Command cmd = map_and_validate_command(v.context, e.command_iova,
e.command_bytes);
if (!cmd.ok) {
cq_complete(e.cookie, STATUS_INVALID, cmd.fault_code);
continue;
}
Reservation r = reserve_resources(cmd.engines, cmd.sram, cmd.deadline);
if (!r.ok) {
defer_or_complete_busy(e, r);
continue;
}
TraceScope trace(e.cookie, cmd.graph_correlation_id);
Result result = execute_dma_analog_digital(cmd, r);
release_resources(r);
cq_publish_before_msix(e.cookie, result);
}
service_health_watchdog_and_calibration_events();
}AcceptedDescriptor ownedGeneration and context valid.
ValidatedAll ranges legalOpcode, buffers, artifact and profile checked.
ReservedResources heldSRAM, DMA, array and digital engine assigned.
RunningWatchdog armedTrace and performance counters active.
CompletingWrites visibleOutput and CQ committed before MSI-X.
TerminalOK or faultExactly one completion and resources released.
Analog-specific device contract
PCIe moves bytes; the analog lifecycle determines whether those bytes are correct
A digital NPU can often load deterministic code and weights. An Analog CIM board also needs a controlled program, verify, characterize, bind, monitor, and refresh lifecycle. These operations are privileged and should not be hidden inside an ordinary inference command.
Terms here
MVM = Matrix-Vector Multiplication
DAC = Digital-to-Analog Converter
ADC = Analog-to-Digital Converter
PVT = Process, Voltage, Temperature
1. PackageWeight programLogical tensor, slices, sign method, target cells, tolerances, hash.
2. ProgramDevice operationErase/program pulse policy and wear budget are device-specific.
3. VerifyReadback qualityDetect bad cell/row/column, retry within limits, record residual error.
4. CalibrateConverters + pathsGain/offset, ADC range, temperature bucket, digital correction.
5. BindLogical program IDCompiler placement maps to healthy physical resources and profile epoch.
6. MonitorInference healthSaturation, drift, ECC/fault, temperature and accuracy sentinels.
| Artifact / state | Required fields | Owner | Invalidated by |
|---|---|---|---|
| Target descriptor | Array geometry, supported precision/modes, converter limits, digital ops, SRAM and alignment | Hardware team; compiler consumes | Hardware revision or incompatible firmware ABI |
| Characterization profile | Chip/lot/device ID, temperature/voltage bucket, gain/offset/noise, bad resources, measurement method, timestamp | Silicon/test team; runtime verifies | Age, PVT excursion, repair/remap, recalibration policy |
| Program manifest | Model/weight hashes, physical placement, slice/sign scheme, programming algorithm, verify tolerance, wear count | Compiler + privileged service | Reprogram, remap, failed verify, reset if state is volatile |
| Live health map | Unavailable rows/columns/tiles/converters, saturation, thermal, calibration age, topology epoch | Firmware + KMD | Any new fault, reset, thermal transition or service action |
| Executable binding | Artifact hash, program IDs, physical resources, profile hash, device generation, fallback variant | Runtime/UMD | Any mismatch in the fields above |
Fail closed: if the runtime cannot prove that the resident program, physical placement, calibration profile, live fault map, firmware ABI, and executable agree, it must rebind/reprogram or choose a digital fallback. Silently running with a stale analog profile creates an accuracy failure that ordinary DMA and queue tests will not catch.
Fault, reset, and security
Every failure has an owner, a containment boundary, and a terminal event
Terms here
AER = PCIe Advanced Error Reporting
ECC = Error-Correcting Code
FLR = Function Level Reset
Fail closed = reject work when compatibility or safety is unknown
| Failure | Detect | Contain | Recover | Evidence retained |
|---|---|---|---|---|
| Malformed ioctl | KMD size, flags, reserved fields, overflow and handle checks | Reject before allocation or MMIO | Return stable -EINVAL/-E2BIG/-ENOENT | Rate-limited audit counter, command ID |
| Illegal IOVA | IOMMU fault plus firmware range check | Stop offending context/queue; do not reset unrelated contexts first | Abort context, signal all fences, reset context or function if needed | IOVA, access type, context, command, engine |
| Execution timeout | Firmware watchdog and host deadline | Stop queue, then engine/reset domain | Queue abort -> context reset -> function reset escalation | Last command, trace tail, engine status, temperature |
| Firmware panic | Fatal mailbox/IRQ, heartbeat loss | Mask new work and stop DMA | Capture bounded dump, reset, reload, increment generation | Build hash, panic code, logs, registers, in-flight IDs |
| PCIe AER | PCI core AER callbacks and counters | error_detected() quiesces without new I/O | Recover, slot reset and reinitialize, or mark permanently lost | AER severity, requester, TLP header where available |
| Analog saturation/drift | ADC saturation, sentinel inference, profile age, temperature | Stop affected program/tile; preserve other healthy capacity | Change range/profile, recalibrate, remap, retry or digital fallback | Array/tile, inputs summary, profile, PVT, before/after error |
| Uncorrectable array fault | Program verify, health scan, accuracy sentinel | Remove row/column/tile from live resource map | Rebind spare/replica or recompile placement; increment topology epoch | Fault location, first seen, wear, remap and accuracy result |
offset + bytes <= BO.sizevalidate -> publish immutable descriptorsmallest effective reset domaincookie + generation + terminal statusdebug aperture off by defaultmeasured boot metadata in support bundleProfiling and correlation
A fast array is irrelevant if the frame stalls at a boundary you cannot see
Terms here
H2D = Host-to-Device transfer
D2H = Device-to-Host transfer
Correlation ID = identifier joining one request across layers
Tail latency = high-percentile slow-request latency
| Trace field | Origin | Purpose | Required clock relation |
|---|---|---|---|
| graph_node_id | Compiler debug map | Relate device work to original ONNX node/subgraph | Identifier only |
| correlation_id | Application/runtime | Join frame/request across host, KMD, firmware and engines | Identifier only |
| command_id / cookie | UMD/KMD | Queue ownership, wait and completion diagnosis | Identifier only |
| host_submit_ns | KMD monotonic clock | Host queue and interrupt latency | CLOCK_MONOTONIC nanoseconds |
| device_start/end_cycles | Firmware hardware timer | DMA, analog and digital stage duration | Publish timer frequency and reset epoch |
| profile/topology epoch | Firmware/runtime | Explain numerical or performance changes after calibration/remap | Monotonic per device generation |
Minimum profiler output: one line per command with model/artifact hash, graph node, context, queue, correlation ID, input/output bytes, queue delay, H2D, compute stages, D2H, interrupt/wakeup, analog profile, saturation/retry count, status, and reset generation. Aggregate only after raw correlation works.
Implementation repository
The skeleton is reasonable; make code, artifacts, data, and loaders traceable
A directory tree is only the first layer of the design. Every owned source path should produce a named build target or validated data file, identify who loads it, state how long it remains valid, and name the smallest test that proves the boundary. The separate repository blueprint covers CMake, compilers, CI, GoogleTest, and contribution mechanics in greater depth; this section maps those mechanics onto ACiM board boot and model execution.
Terms here
RTL = Register-Transfer Level hardware
UAPI = kernel/userspace interface files
CI = Continuous Integration
Build target = named output in the build graph
Loader = component that places an artifact into its execution context
Generated code = mechanical output from the protocol schema
Repository verdict: a monorepo is reasonable for the first end-to-end vertical slice because protocol, simulator, compiler, runtime, KMD, firmware, tools, and tests can change atomically. Split a component only when ownership, access control, release cadence, or build tooling requires it. A split repository must consume a versioned protocol package and run the same cross-version compatibility suite before release.
acim-stack/
|-- CMakeLists.txt
|-- CMakePresets.json
|-- .github/
| `-- workflows/ # fast, hardware, nightly, release lanes
|-- cmake/
| `-- toolchains/ # host and controller cross builds
|-- configs/
| |-- targets/chip-rev.yaml # arrays, SRAM, ISA, precision, costs
| |-- boards/pcie-card.yaml # IDs, BARs, DMA, MSI-X, clocks
| |-- boards/integrated-soc.yaml # MMIO, DMA, SMMU, IRQ, power domains
| |-- models/yolo.yaml # preprocessing, shapes, accuracy gates
| `-- test-matrices/ # simulator, FPGA, board, PVT lanes
|-- spec/ # authoritative binary contracts
| |-- registers.yaml
| |-- queue.yaml
| |-- firmware_abi.yaml
| |-- artifact.schema.json
| |-- calibration.schema.json
| `-- trace.schema.json
|-- tools/codegen/ # schema validation and emitters
|-- generated/ # optional distributable generated output
| |-- include/hw/
| |-- rtl/
| |-- firmware/
| `-- docs/
|-- include/
| |-- uapi/linux/acim_accel.h # stable userspace/kernel contract
| `-- acim/ # supported SDK and HAL interfaces
|-- compiler/
| |-- import/ dialect/ transforms/
| `-- target/ scheduling/ packaging/
|-- runtime/umd/
| |-- device.cpp buffer.cpp queue.cpp artifact.cpp
| `-- backends/kmd_linux.cpp fake.cpp cosim.cpp
|-- driver/kmd/
| |-- pci.c platform.c uapi.c context.c memory.c queue.c irq.c
| `-- firmware.c reset.c trace.c health.c Kbuild
|-- firmware/
| |-- boot/ mailbox/ scheduler/ dma/
| `-- analog/ digital/ health/ trace/ tests/
|-- kernels/
| |-- analog/ # array programs and descriptors
| `-- digital/ # DSP/vector/scalar fallback kernels
|-- sim/
| |-- functional/ analog/ cycle/
| `-- pcie_model/ platform_model/ # transport, reset and fault models
|-- platforms/
| |-- linux-pcie/ # udev, module config, packages
| |-- linux-soc/ # DTB, clocks, power, platform package
| `-- embedded/board-name/ # Buildroot/Yocto and boot scripts
|-- apps/
| |-- acim-smoke/
| `-- acim-yolo/
|-- models/
| |-- manifests/ quantization/
| `-- golden/ # small redistributable test tensors
|-- calibration/
| `-- fixtures/ # synthetic/non-production device profiles
|-- tools/
| |-- acim-info/ acim-fw/ acim-memtest/
| `-- acim-trace/ acim-health/ acim-support-bundle/
|-- scripts/
| |-- build/ test/ package/
| `-- ci/ # orchestration, not duplicated test logic
|-- tests/
| |-- unit/ contract/ smoke/ integration/
| |-- differential/ fault-injection/ end-to-end/
| `-- performance/ soak/ qualification/
|-- packaging/
`-- docs/spec/ is authoritativeCI regenerates headers and fails on a diff. Every register includes access, owner, reset value, side effects and revision.generated/ is mechanicalGenerate into the build tree by default. Keep a source-tree mirror only when separate RTL/KMD builds or source packages require it; CI must regenerate with no diff. Generator version and schema hash travel with the output.uapi/ is append-only in behaviorRun ABI layout checks on 32-bit and 64-bit builds; explicit reserved fields replace compiler padding.configs/ is validated product dataA chip target, board, model, and test matrix have schemas, units, revisions, and compatibility tests. They are not informal command-line flags.platforms/ owns boot integrationKeep PCIe packages, integrated-SoC resources, and embedded BSP files outside runtime logic. The same UMD API should work after any supported platform reaches READY.Transport models come before the boardModel PCIe BAR or integrated MMIO side effects, ring wrap, ordering, interrupts, reset epochs, malformed commands and DMA faults.
tools/ are release deliverablesEnumeration, firmware, memory test, trace, health and support bundle are necessary to debug customer systems.models/ stores provenance, not arbitrary datasetsRecord model/dataset versions, preprocessing, quantization statistics, artifact hash, board profile and expected metrics. Keep only permitted small golden inputs in Git.calibration/ contains test fixtures onlyModel quantization data and per-chip analog calibration are different. Production chip profiles belong in the provisioning/service store, not source control.out/ is ignored build outputNever mix source with binaries, firmware, model artifacts, board images, traces, or test results. Package them from a reproducible preset and manifest.Build output is a versioned handoff, not another source directory
out/<preset>/ directory or CI artifact storeout/linux-x86_64-release/
|-- host/
| |-- bin/acim-run
| |-- bin/acim-smi
| `-- lib/libacim_runtime.so
|-- kmd/acim_accel.ko
|-- device/
| |-- acim-fw.elf # optional programmable controller
| |-- acim-fw.bin
| |-- acim-fw.map
| `-- acim-fw.sym
|-- generated/protocol-manifest.json
|-- models/yolo-int8.acx
|-- packages/acim-sdk.tar.zst
`-- evidence/build-manifest.json
out/edge-board-release/
|-- images/sdcard.img
|-- images/Image
|-- images/board.dtb
|-- images/rootfs.ext4
`-- evidence/bsp-manifest.jsonHost code
acim-run, acim-smi, and libacim_runtime.so execute on the Linux host CPU. They never execute inside the analog array.Privileged host code
acim_accel.ko executes in the Linux kernel and owns PCIe, IOMMU, DMA mappings, interrupts, reset, and firmware loading.Device control code
acim-fw.elf/bin executes on a device controller only when the hardware has a programmable CPU. Keep symbols and the link map for debug.Model executable data
yolo-int8.acx contains target-specific commands, constants, weight placement, kernel references, metadata, and debug maps; it is loaded by the runtime.Embedded host imageThe SD/eMMC image contains the host BSP and userspace. It is separate from device firmware and may be absent for a workstation PCIe card.
EvidenceEvery package records source commits, dirty state, compiler/tool versions, schema hashes, target/profile IDs, ABI ranges, and artifact hashes.
1. Select inputs
spec/ + configs/Protocol schemas, chip target, board profile, model manifest, toolchain preset.2. Generate
tools/codegen/Validate schemas; emit C/C++/firmware/RTL definitions, docs, hashes, and compatibility fixtures.3. Build hostSDK + UMD + KMDCMake/Ninja builds userspace; Kbuild builds the module; package symbols and manifests.
4. Build deviceFirmware + kernelsCross-compile, link to the controller memory map, emit ELF/bin/map/symbols and size report.
5. Compile modelONNX to
.acxUse target and calibration inputs to lower, schedule, place, package, and preserve debug mapping.6. Package + proveInstallable setRun contract/smoke tests; assemble host, device, model, optional BSP, and evidence manifests.
| Source owner | Named output | Loader / consumer | Validity lifetime | Smallest proof |
|---|---|---|---|---|
spec/ + tools/codegen/ | acim_protocol_codegen: generated headers, RTL package, docs, protocol manifest | Compiler for KMD, UMD, firmware, simulator and RTL | Protocol version | Regenerate-with-no-diff plus cross-language encode/decode fixture |
driver/kmd/ | Kbuild modules: acim_accel.ko and debug symbols; or built-in object | Linux module loader or kernel image link | Host boot/module load | Probe, query caps, remove, reprobe, and leak check |
firmware/ + kernels/ | acim_firmware, when present: controller ELF/bin/map and kernel package | KMD, remoteproc, boot ROM, or JTAG lab flow | Device boot epoch; kernels may be session resident | READY handshake plus no-op/admin queue command |
runtime/umd/ | acim_runtime: runtime library and HAL backends | Application dynamic loader | Host process/session | Same buffer/queue test against fake, co-sim, and KMD backends |
compiler/ | acim-compile: .acx executable artifact | Runtime validates and extracts command/program payloads for firmware | Target/profile compatibility; then session binding | Inspect, validate, deserialize, and execute one golden MVM |
platforms/embedded/ | edge_board_image: bootloader, kernel, DTB, rootfs and media image | Host boot ROM/bootloader | Embedded host boot/update slot | QEMU-supported BSP boot or physical serial boot to SDK health check |
calibration/fixtures/ plus external provisioning store | Test fixture or production per-device calibration/health profile | Provisioning tool, runtime and firmware | Chip, PVT range, age and calibration generation | Integrity, serial/profile match, known-answer operation, stale-profile rejection |
Boot flow: code is loaded, then capabilities and mutable state are negotiated
1. Host boot
platforms/Embedded only: bootloader, Linux, rootfs and DTB. A workstation PCIe flow starts at KMD probe on its supported OS.2. KMD probe
pci.c or platform.cReads PCI BAR/caps or platform MMIO/DT resources; creates device state; configures DMA, IOMMU/SMMU, IRQ/MSI-X, reset, power and trace.3. Device control
driver/kmd/firmware.cWorks with firmware/boot/; loads firmware when present, checks image/ABI, then starts the controller or verifies fixed RTL control.4. Admin queues
driver/kmd/memory.cWorks with queue.c; allocates rings and exchanges IOVAs, depth, generation, vectors, heartbeat and trace clock.5. Publish UAPI
driver/kmd/uapi.cWorks with health.c; exposes the device only after READY and returns capabilities, health, calibration generation and ABI.6. Open runtime
runtime/umd/device.cppQueries caps; creates context/queue; records run identity; rejects incompatible SDK, firmware or target.| Boot step | Executing code | Data entering | State produced | Failure evidence |
|---|---|---|---|---|
| Host boot | Bootloader and Linux BSP | Boot image, DTB, rootfs configuration | PCIe root or platform bus, IOMMU/SMMU, filesystems, services | Bootloader/earlycon/kernel serial log |
| Device discovery | PCI or platform KMD probe and reset modules | PCI config/BARs or DT/MMIO resources, generated board constants and live caps | Kernel device object, mapped MMIO, DMA mask, IRQ/MSI-X vectors | Kernel tracepoint, AER/platform fault, IOMMU/SMMU and probe log |
| Control-plane start | Boot ROM, KMD firmware loader, optional controller firmware | Firmware bytes/hash, entry point, compatibility range | READY, boot epoch, firmware build/ABI, crash area | Mailbox stage, ROM UART, JTAG/GDB, crash buffer |
| Queue negotiation | KMD queue/memory plus firmware scheduler | Ring IOVAs, size, generation, IRQ/MSI-X vector, context limits | Enabled admin/data queues and heartbeat | Queue indices, descriptor dump, IRQ cause, timeout stage |
| Userspace open | UMD/HAL and KMD UAPI | Requested API version, process identity and policy | Context, Buffer Object namespace, queue/event handles | errno, UAPI trace, negotiated capabilities and reset epoch |
Model flow: separate immutable weights, per-device calibration, session state, and request tensors
1. Offline compile
compiler/Inputs: ONNX, target, and model-quantization statistics. Output: commands, constants, placement and debug map in .acx.2. Validate artifact
artifact.cppChecks hashes, target, ABI, tensor contract, required features, scratch size and fallback variants.3. Program session
firmware/analogCombines immutable weights with chip calibration/health; programs, verifies and remaps arrays; returns program handle.4. Submit request
buffer.cpp + queue.cppPer-request input BO and descriptor reference resident program, output BO, deadline and correlation ID.5. Execute graphDMA + ACiM + digitalMoves activations, performs MVM/ADC/reconstruction and digital operators; weights remain resident when legal.
6. Complete + inspectCQ + runtime + appReturns output tensor, status, counters and trace IDs; compares selected checkpoints with the golden model.
| Data class | Examples | Owner | Lifetime | Must invalidate when |
|---|---|---|---|---|
| Executable code | KMD, host runtime, controller firmware, digital microcode | Packaging/KMD/boot ROM | Module, process, boot epoch or kernel version | ABI, security, target or firmware generation is incompatible |
| Immutable model data | Graph commands, weights, scales, constants, debug map | Compiler artifact and runtime | Artifact version; may become session resident | Artifact hash, target, layout or precision contract changes |
| Per-device state | Calibration, defect map, remap, temperature/voltage bucket | Manufacturing/service plus firmware | Chip/profile generation and valid PVT range | Serial mismatch, age/temperature threshold, recharacterization or fault update |
| Per-session state | Program handle, resident placement, context, queues, mapped BOs | Runtime, KMD and firmware | Session and reset generation | Reset, firmware restart, device loss, remap or calibration generation changes |
| Per-request data | Input/output tensors, descriptor, deadline, dependencies, correlation ID | Application/runtime and queue owners | Submission through terminal completion | Completion, abort, timeout or reset releases ownership safely |
| Evidence data | Logs, traces, counters, raw ADC samples, tensor checkpoints | Support/profiling tools | Run bundle and retention policy | Never silently reused as current calibration or executable input |
Staged delivery
Reach the board through nine evidence-producing milestones
Each milestone is a vertical slice with owned code, a reproducible operation, named boundary data, a measurable exit gate, and an explicit capability that it unlocks. Transport semantics remain common while an integrated SoC uses MMIO/DMA/SMMU/IRQ and a PCIe device uses BAR/DMA/IOMMU/MSI-X.
Terms here
Protocol v0 = first versioned hardware/software contract
FPGA = Field-Programmable Gate Array prototype
MVM = Matrix-Vector Multiplication
Exit gate = evidence required before depending on this milestone
Unlock = new work that becomes technically meaningful after the gate passes
Qualification = evidence required before product release
Two plans with different jobs: the ten-stage verification ladder answers where and at what fidelity a test runs, from native model through silicon. These nine delivery milestones answer which dependency-ordered code slice the team implements. For example, the queue slice should pass first in the fake backend, then QEMU/co-simulation, FPGA, and finally silicon without changing its descriptor semantics or whether the physical transport is integrated or PCIe.
host CI | fake | co-simintegrated SoC | PCIe | FPGAcalibrated simulator | siliconplatform matrix | qualificationDependency graph: each exit artifact becomes the next milestone's input
M1 | ContractProtocol packageRegisters, UAPI, queues, faults, trace schema, fixtures, and compatibility rules.
feeds simulator + KMD + firmware + RTLM2 | Virtual deviceTransport behaviorMMIO/BAR state, DMA address spaces, interrupts, reset, ordering, and injected faults.
feeds physical probe expectationsM3 | BootREADY deviceIdentity, mapped resources, firmware/control ABI, capabilities, health, and reset epoch.
feeds memory and queue setupM4 | MemoryVerified data pathBO ownership, IOVA mappings, exact payloads, cache rules, DMA counters, and fault cleanup.
feeds trusted ring memoryM5 | CommandsReliable execution pathSQ/CQ ownership, doorbells, IRQ/MSI-X, timeout, abort, and one terminal completion.
feeds analog command transport
Transport foundation
Exact bytes + deterministic completionrequired before numerical correlation
Compute foundation
M6 | One MVMCalibrated operationResident weight program, chip profile, raw ADC, reconstructed output, error distribution, and mitigation cost.
feeds legal analog placementM7 | SubgraphHybrid device islandPartition, SRAM schedule, boundary tensors, analog/digital trace, fallback, and resident program handle.
feeds complete model assemblyM8 | Full modelQualified workloadAccuracy, placement, latency percentiles, throughput, energy, thermal behavior, and recovery evidence.
feeds product regression baselineM9 | OperationsRelease evidencePackages, compatibility matrix, isolation, updates, rollback, recovery, health, and support bundles.
feeds release qualificationSequence graph: code, commands, tensors, and evidence cross different owners
Build / compiler / CI
Host runtime / KMD
Device control
DMA + ACiM + digital
Evidence store
M1
Generate protocol package -> compile host consumers
ABI/layout report
M2
Submit conformance suite -> virtual device
Deterministic protocol/fault trace
M3
Probe resources, reset, load/start control plane ->
Boot stages and image/ABI hashes
M3
<- READY, caps, health, calibration and reset generation
Cold-boot and failure bundle
M4
Register BO/IOVA + submit loopback command ->
Mapping, ownership and command record
M4
DMA pattern H2D/D2H -> verify guards -> unmap
Pattern, guard, counter and fault result
M5
Publish SQ + doorbell + deadline ->
Descriptor and ordering trace
M5
<- CQ status + IRQ/MSI-X + wake event
One terminal status per cookie
M6
Program weights + bind chip profile ->
Program/verify, placement and profile IDs
M6
Start MVM -> DAC/array/ADC/reconstruct
Raw values, error, cycles and energy
M7
Load and dispatch hybrid subgraph ->
Partition and SRAM schedule
M7
Execute analog + digital island ->
Boundary tensors and fallback trace
M8
Dispatch declared validation and performance scenarios ->
Run manifest, placement and scenario IDs
M8
Execute full graph -> return detections and counters
mAP, latency, power, thermal, recovery
M9
Install/update qualification set -> exercise supported host
Compatibility and support bundle
M9
Release gate accepts or rejects the evidence set
Each row is an ownership handoff. The evidence lane stores machine-readable records keyed by build, artifact, device, reset generation, and run/correlation IDs.
Detailed implementation and exit gates
Phase A
Contracts and virtual hardware
Entry: architecture decisions are documented. Exit: host software can exercise the real command semantics and faults without silicon.
Freeze protocol v0
Create one machine-readable contract before KMD, firmware, simulator, and RTL implementations diverge.
host CI | no board
Implement
spec/, tools/codegen/, generated C/C++/firmware/RTL definitions, Linux UAPI structures, version rules, ownership, endianness, reset values, and memory ordering.ExerciseRegenerate every consumer; encode and decode golden capability, mailbox, SQ, CQ, fault, and trace fixtures across languages; reject unknown major versions and nonzero reserved fields.
Data and evidenceSchema and generator hashes, generated diff, structure sizes/offsets, round-trip fixtures, compatibility matrix, and malformed-input results.
Exit gateRegeneration is clean; all consumers agree bit-for-bit; old/new combinations behave exactly as declared; fuzzed lengths, enum values, overflow, and alignment fail safely.
Unlocks milestone 2: simulator, RTL, KMD, and firmware teams can implement against the same stable contract in parallel.
Build the transport and device model
Represent the control registers, host/device memory, queues, interrupts, reset, and faults before a board is required.
native fake | QEMU/co-sim
Implement
sim/pcie_model/ and/or sim/platform_model/, fake/co-sim HAL backends, MMIO side effects, DMA address spaces, IRQ/MSI-X, posted-write behavior, reset epochs, firmware mailbox, and fault injection.ExerciseRun the real UMD queue library against the model; wrap rings, fill queues, reorder notifications, drop/duplicate interrupts, issue malformed descriptors, reset in flight, and simulate illegal DMA.
Data and evidenceRegister snapshots, queue memory, DMA transactions, interrupt events, command/fault timelines, deterministic seeds, and expected KMD/firmware status codes.
Exit gateThe same conformance suite passes for every transport enabled by the target profile; every accepted command has one terminal completion; every injected fault is bounded and reproducible.
Unlocks milestone 3: physical discovery and boot can be compared with a known protocol model instead of debugged from intuition.
Phase B
Device and transport bring-up
Entry: protocol/model tests pass. Exit: one physical device moves exact data and completes queued work across reset and interrupt corner cases.
Discover, reset, and boot the device
Establish identity, resources, firmware/control-plane readiness, health, and a repeatable teardown path.
integrated SoC | PCIe FPGA/board
Implement
driver/kmd/platform.c and/or pci.c, reset.c, firmware.c, board/DTB integration, resource mapping, SMMU/IOMMU setup, IRQ/MSI-X allocation, health handshake, and safe remove.ExerciseCold boot; read IDs/caps; map MMIO/BARs; start optional controller firmware; negotiate ABI and reset generation; publish the device node; open/close; remove/reprobe; force boot timeout and incompatible firmware.
Data and evidenceDTB/platform resources or PCI config, register map version, firmware hash/ABI, READY stages, health/calibration generation, serial log, KMD trace, AER or platform fault state.
Exit gateCold boot and controlled failure are deterministic; no device node appears before READY; teardown leaves no mapping/IRQ; repeated bind/reset succeeds, targeting 1000 cycles before qualification.
Unlocks milestone 4: the host can safely allocate device-visible memory and test data movement without analog compute.
Pass DMA and shared-memory loopback
Prove byte-exact ownership transfer through every supported buffer type, size, alignment, and mapping mode.
SMMU/IOMMU | DMA | SRAM
Implement
driver/kmd/memory.c, firmware DMA handlers, BO allocation/import, IOVA registration, coherent admin memory, streaming payload mappings, cache synchronization, scatter/gather limits, and acim-memtest.ExerciseLoop deterministic patterns H2D and D2H across boundary sizes, page crossings, unaligned requests, maximum segments, concurrent buffers, wrong direction, unmapped IOVA, context exit, timeout, and reset.
Data and evidenceBO handle, physical pages/scatterlist, IOVA, direction, bytes, pattern hash, guard regions, cache-operation trace, DMA counters, bandwidth, IOMMU/SMMU fault and cleanup record.
Exit gatePayload and guards are exact for the complete matrix; illegal access is contained; every map has one unmap after completion/device loss; bandwidth is explained by transfer and memory counters.
Unlocks milestone 5: queue failures can be investigated without wondering whether descriptor or completion memory is corrupt.
Pass queues and completion interrupts
Prove command ownership, ordering, wakeup, timeout, abort, and reset semantics under concurrency.
IRQ | MSI-X | polling
Implement
driver/kmd/queue.c, irq.c, runtime/umd/queue.cpp, firmware scheduler, SQ/CQ ownership barriers, doorbells, moderation, poll/event integration, watchdog, abort, and generation checks.ExerciseWrap and fill rings; run multiple queues/processes; switch polling/interrupt mode; delay, lose, and duplicate interrupts; complete out of order where legal; close during work; timeout, abort, reset, and submit stale entries.
Data and evidenceProducer/consumer indices, descriptor bytes, command cookie, queue/context/generation, doorbell and IRQ timestamps, CQ status, watchdog state, wakeup latency, and first-fault trace.
Exit gateEvery accepted cookie receives exactly one terminal status; no deadlock or queue corruption occurs at wrap/full; stale generations never execute; polling and interrupt paths return identical results.
Unlocks milestone 6: analog numerical debugging can rely on a trustworthy command, transfer, and completion path.
Phase C
Analog compute and model correlation
Entry: transport is exact and recoverable. Exit: one operation, one subgraph, and one full model meet declared numerical and performance gates.
Execute one resident MVM
Make the smallest analog operation observable from programmed weights through reconstructed output.
analog simulator | calibrated device
Implement
compiler/target/, kernels/analog/, firmware programming/verify/remap, analog simulator, range/profile selection, raw ADC capture, reconstruction, saturation/retry reporting, and golden reference.ExerciseCompile a tiny
.acx; program and verify a known weight block; bind the correct chip profile; run fixed vectors and corner values; force bad rows, saturation, stale calibration, remap, retry, and digital fallback.Data and evidenceWeight/program hash, physical placement, defect/remap table, calibration ID/PVT, DAC codes, raw ADC codes, reconstructed values, golden accumulations, error distribution, cycles, energy, and temperature.
Exit gateProgram verify succeeds; error distribution stays inside the predeclared budget across required devices/PVT; first numerical divergence is identifiable; all mitigation cost is visible in cycles, arrays, and digital work.
Unlocks milestone 7: compiler partitioning and SRAM scheduling can combine trusted analog and digital operations.
Execute a YOLO subgraph
Prove one contiguous hybrid island with resident weights, local activation scheduling, and explicit fallback boundaries.
simulator parity | one device
Implement
compiler/transforms/, scheduling/memory planning, artifact loading, analog Conv/GEMM placement, digital activation/route operators, fallback, layer debugger, boundary tensor capture, and graph-to-command mapping.ExerciseCompile a fixed Conv/activation/route island; program weights once; run one H2D input and one D2H output; sweep supported shapes; disable fusion to isolate layers; force digital fallback and one SRAM-pressure case.
Data and evidencePartition report, target/artifact hash, resident program handle, SRAM allocation timeline, boundary tensors/distributions, command trace, fallback reason, H2D/D2H bytes, per-stage cycles, and saturation/retry counts.
Exit gateEvery boundary tensor meets its error budget; SRAM lifetimes do not overlap illegally; simulator/device placement agrees; fallback is numerically equivalent; repeated warm runs reuse weights and avoid unexplained transfers.
Unlocks milestone 8: full-model accuracy and end-to-end latency become meaningful rather than hiding a broken layer or excessive host fallback.
Qualify end-to-end YOLO
Measure the complete deployed workload, including preprocessing, fallback, postprocessing, power, thermal behavior, and recovery.
validation set | performance lab
ImplementVersioned model/run manifests, complete application pipeline, validation harness, golden host backend, latency/power collection, profiler correlation, cold/warm scenarios, stream/queue policy, soak, and model-level recovery.
ExerciseRun the declared validation dataset; compare host, simulator, ACiM, and fallback variants; measure cold load, first inference, steady state, throughput, queue depth, thermal soak, calibration refresh, timeout, reset, and model rebind.
Data and evidenceDataset/model/preprocess versions, detections, mAP and per-class deltas, fallback/operator placement, latency percentiles, FPS, energy/frame, clocks, temperature, transfer bytes, profiler timeline, and recovery bundles.
Exit gatePre-agreed accuracy and SLO thresholds pass on the complete pipeline; all host/off-chip work is disclosed; stage times explain end-to-end latency; independent reruns reproduce metrics inside declared variance.
Unlocks milestone 9: product hardening can preserve a known-good workload while platform, update, isolation, and recovery behavior expands.
Phase D
Product operations and supportability
Entry: the target model meets numerical/performance gates. Exit: supported systems can install, isolate, update, recover, and produce actionable evidence.
Harden the product path
Turn the working board stack into a maintainable platform with explicit compatibility, failure, and service behavior.
platform/kernel matrix | qualification
Implement
packaging/, platform integration, permissions, multi-process isolation, dma-buf where required, power states, AER or on-chip fault handling, reset escalation, firmware/model updates, rollback, diagnostics, profiler, and support-bundle tools.ExerciseClean install/uninstall; old/new ABI matrix; concurrent users; process death; suspend/resume; device removal; link/interconnect faults; firmware panic; calibration/profile update; interrupted update; rollback; thermal fault; and field-style reproduction from one command.
Data and evidencePackage and image hashes, compatibility matrix, permissions/IOMMU groups, update state, reset/fault timeline, resource-leak report, performance regression result, dependency/build provenance, health snapshot, and complete support bundle.
Exit gateSupported platforms pass clean install, isolation, update/rollback, soak, recovery, and regression thresholds; failures never leave stale DMA or silent corruption; one bundle identifies the first failed boundary.
Unlocks release qualification: release approval can be based on a versioned evidence set rather than a successful laboratory demo.
One developer entry point: provide a repository wrapper such as
scripts/test/run.py --lane <contract|dma|queue|mvm|yolo-e2e> --backend <fake|cosim|fpga|silicon>.
It should select CTest/KUnit/firmware/model tests, capture the build and run manifests, and return a non-zero
status on a failed gate. The wrapper orchestrates existing test tools; it must not hide their raw reports.
Verification matrix
Separate transport correctness, numerical correlation, and product qualification
| Test layer | Representative cases | Pass condition | Automation |
|---|---|---|---|
| Schema and ABI | Generated register diff, struct sizes/offsets, reserved bits, unknown versions, endian conversion | RTL/C/C++/docs agree; old UMD works with new KMD inside declared compatibility | Every change, 32/64-bit cross-build |
| KMD UAPI | Fuzz ioctls, invalid handles, overflow, race/close, fork, quotas, permissions, compat mode | No crash/leak; stable errno; no unauthorized MMIO/DMA | KUnit, kselftest, syzkaller-style corpus |
| DMA and queues | All sizes/alignment, scatterlists, ring wrap/full, interrupt storms/loss, timeout, concurrent queues | Exact data; one terminal completion; no mapping remains after fence/device loss | Simulator, FPGA and silicon parity suite |
| Firmware | Malformed descriptor, illegal IOVA, unsupported opcode, resource exhaustion, watchdog, panic, reset mid-command | Bounded fault, captured evidence, generation change, clean reinitialization | Firmware unit tests plus host fault injection |
| Analog operation | Program/verify, bad rows/columns, PVT, saturation, drift, calibration mismatch, remap and retry | Declared tolerance/distribution; stale profile rejected; fallback works | Statistical simulator and device population |
| YOLO correctness | Boundary tensors, layer distributions, full validation set, host/device/fallback variants | Predetermined mAP and per-layer error budgets with identical preprocessing/postprocess | Nightly model corpus and release run |
| Performance | Cold/warm start, single stream, throughput, queue depth, overlap, thermal soak, recovery | No unexplained gap between stage times and end-to-end latency; tails within the Service-Level Objective (SLO) | Scenario harness with raw trace archive |
| PCIe reliability | AER correctable/non-fatal/fatal, link retrain, FLR/hot reset, remove during work | Callbacks quiesce correctly; recover or report permanent device loss without stale DMA | Lab qualification and platform matrix |
Information to obtain
Questions that must be answered by the chip, board, and firmware teams
Public demos cannot answer these details. They are the minimum inputs for turning the architecture into code without guessing at hidden hardware behavior.
| Area | Required questions | Deliverable | Blocks |
|---|---|---|---|
| PCIe | IDs/revisions? Gen/lanes? BAR sizes? DMA width/segments? MSI-X count? FLR/AER? posted-write readback register? | PCIe compliance and register specification | KMD probe, DMA, interrupt and reset |
| Boot/firmware | ROM states? Load path? Authentication? Firmware ABI? mailbox? heartbeat? crash dump? rollback? | Boot protocol and firmware image format | Safe device publication and field update |
| Memory/queues | SRAM map/banks? coherent regions? DMA engines? queue format/depth? ownership? ordering? timeout/abort? | Memory map, queue ABI and timing model | UMD/KMD/firmware and simulator |
| Analog macro | Array size? weight/input bits? sign and slicing? DAC/ADC modes? accumulation? saturation? programming endurance/time? | Machine-readable target profile | Compiler legality, mapping, scheduling and cost |
| Calibration | Factory vs field data? profile format? PVT buckets? gain/offset/noise? bad resources? refresh triggers? authenticity? | Characterization schema and lifecycle | Accuracy, binding, recovery and qualification |
| Digital engines | ISA/microcode? RISC-V? vector ops? resize/concat/activation/NMS? precision/layout? local memory and synchronization? | Digital backend and firmware kernel ABI | Contiguous YOLO device partition |
| YOLO demo | Exact model/export/opset? dataset? preprocessing? quantization/retraining? operator placement? off-chip work? measured silicon? | Reproducible model card and run manifest | Honest baseline and software reproduction |
| Product policy | Single/multi-process? threat model? secure boot? dma-buf? virtualization? power states? supported kernels/platforms? | Platform requirements and security model | Stable product UAPI and qualification plan |
Primary references and evidence limits
Kernel behavior is sourced; hardware details remain proposed until disclosed
dma_wmb()/dma_rmb() ownership example and the distinction between shared DMA memory and MMIO ordering./dev/accel/accel*.error_detected, reset, resume, permanent failure and interrupt behavior during recovery.request_firmware() lifecycle and synchronous/asynchronous image loading APIs.virt machineGeneric virtual platform capabilities, including its PCIe host bridge; it is not a model of a private ACiM controller or endpoint.-s -S, remote debugging, breakpoints, registers, and memory inspection for emulated CPU bring-up.tt-kmdPublic KMD repository with character device, PCIe, memory, interrupt, message queue, telemetry and device-specific modules.tt-umdPublic UMD repository, KMD dependency, IOMMU/host-memory setup, simulator tests, logging and profiling patterns.
Evidence boundary: no public source identified in this study discloses an iHW production UMD,
KMD, firmware ABI, PCIe register map, operator placement, or complete YOLO run manifest. Every concrete
ACIM_* name, offset, structure, and repository path on this page is an engineering proposal to be
replaced or confirmed by the actual hardware contract.