mirror of
https://github.com/allexanderbergmns/xh1-research.git
synced 2026-08-27 21:17:02 +00:00
R: 4 failed research runs on round.
This commit is contained in:
@@ -0,0 +1,8 @@
|
||||
2026-08-26T10:19:32Z research/10-verification/core-verification.md 1 research completed
|
||||
2026-08-26T10:20:02Z research/10-verification/core-verification.md 1 review VERDICT: FAIL
|
||||
2026-08-26T10:21:10Z research/10-verification/core-verification.md 2 revision completed
|
||||
2026-08-26T10:21:41Z research/10-verification/core-verification.md 2 review VERDICT: FAIL
|
||||
2026-08-26T10:22:46Z research/10-verification/core-verification.md 3 revision completed
|
||||
2026-08-26T10:23:10Z research/10-verification/core-verification.md 3 review VERDICT: FAIL
|
||||
2026-08-26T10:24:29Z research/10-verification/core-verification.md 4 revision completed
|
||||
2026-08-26T10:24:57Z research/10-verification/core-verification.md 4 review VERDICT: FAIL
|
||||
@@ -0,0 +1 @@
|
||||
research/10-verification/core-verification.md
|
||||
+385
@@ -0,0 +1,385 @@
|
||||
# Core Verification
|
||||
|
||||
## Status
|
||||
|
||||
DRAFT — Research framework established; all claims are explicitly classified by epistemic status (FACT with citation, ASSUMPTION, PROPOSAL, RECOMMENDATION, OPEN QUESTION, or INSUFFICIENT EVIDENCE). Concrete sign-off criteria, tool selection, and quantitative thresholds are deferred until the OPEN QUESTION inputs are resolved. The numeric thresholds appearing in the Sign-Off Criteria section are illustrative starting points for the verification plan and are not validated targets. Every threshold below is restated with this caveat at the point of use so that the sign-off list is internally consistent with this status.
|
||||
|
||||
## Abstract
|
||||
|
||||
This document investigates verification strategies for the XH-1 core, defined here as a single processing core instance within a custom 128-core RISC-V processor. Core verification is the layer of the verification methodology stack concerned with demonstrating that one core conforms to its architectural specification, microarchitectural contract, and integration assumptions prior to and independent of multi-core interactions. For a 128-core design, the core is the unit of replication; defects undetected at this layer replicate 128-fold across the die, amplifying area, power, yield, and debug cost. This document frames the research questions, surveys existing approaches (simulation, formal verification, hardware-assisted verification, FPGA prototyping, emulation), analyzes their applicability to a custom RISC-V core, and identifies the gaps that the XH-1 project must close given that the existing related documents in `research/10-verification/` are placeholders ("SOON").
|
||||
|
||||
## Research Question
|
||||
|
||||
What verification methodology, toolchain, environment abstraction, coverage model, and sign-off criteria are appropriate for verifying a single XH-1 core such that the methodology:
|
||||
|
||||
1. Provides evidence that the core implements the RISC-V ISA subset selected for XH-1 (see `research/10-verification/isa-compliance.md`).
|
||||
2. Provides evidence that the core's microarchitecture meets timing, area, and power targets for a design that will be replicated 128 times.
|
||||
3. Scales economically — verification effort, compute cost, and human review time must not grow linearly with the 128-core replication factor.
|
||||
4. Produces artifacts (testbenches, coverage models, formal proofs, regression suites) that compose cleanly with the surrounding verification layers (unit, multicore, system, post-silicon).
|
||||
5. Is implementable given that the related XH-1 documents in area 10 are not yet populated, indicating a greenfield verification program.
|
||||
|
||||
## Background
|
||||
|
||||
Verification of a processor core is conventionally decomposed into:
|
||||
|
||||
- **Unit verification** — verification of individual blocks (ALU, FPU, register file, branch predictor, fetch/decode structures, load-store unit, replay logic, CSR file, MMU/TLB, caches). See `research/10-verification/unit-verification.md`.
|
||||
- **Core verification** — verification of the integrated core as a single design under test (DUT), exercising instruction-stream and exception behavior at the architectural interface, plus internal microarchitectural invariants.
|
||||
- **Multicore verification** — coherence, consistency, interconnect, and shared-resource correctness across multiple core instances. See `research/10-verification/multicore-verification.md`.
|
||||
|
||||
For XH-1, the core is the replication unit. The cost model must distinguish two quantities that this document often refers to together as "cost":
|
||||
|
||||
- **Defect-escape cost (multiplied by replication).** The downstream cost of an undetected defect (silicon debug, re-spin, field failure, yield loss). Under the replication-homogeneity assumption, an undetected core defect is replicated 128-fold in silicon, multiplying its downstream impact.
|
||||
- **Verification effort cost (paid once).** The engineering cost of running the testbench, discharging formal properties, maintaining the regression, and writing tests. This is paid against one RTL; the fix, once made, is replicated.
|
||||
|
||||
The phrase "every bug caught at the core layer is caught 128 times" is shorthand for the first quantity: every defect that escapes core verification multiplies its downstream cost by the replication count. It is not a claim that the verification effort itself runs 128 times — the testbench runs against the RTL once.
|
||||
|
||||
**ASSUMPTION (replication homogeneity):** The 128 XH-1 cores are architecturally identical instances of a single RTL core produced by elaboration of the same source. Heterogeneous cores, per-core configuration, or runtime variants invalidate the replication-leverage argument on defect-escape cost and require per-variant verification. This is treated as an assumption because no repository evidence contradicts it; OPEN QUESTION 6 below revisits it. The "per-instance marginal cost of zero" claim for proven formal properties holds for bit-identical replicated RTL and refers to the marginal cost of *re-running* a proof against another elaborated instance of the same source — not to the engineering cost of constructing the proof. If the source is parameterized and elaborated with per-instance parameters that differ, the proof must be re-established for any parameter set that differs, and that re-establishment cost is not zero.
|
||||
|
||||
**ASSUMPTION (target implementation class):** XH-1 targets ASIC tape-out. FPGA-only deployment would change the methodology (no DFT/ATPG, different sign-off criteria, no post-silicon bring-up in the same sense). This assumption is conditional and is revisited in OPEN QUESTION 12.
|
||||
|
||||
**ASSUMPTION (verification language availability):** The verification language of the testbench is the same as or interoperable with the design language. This is revisited in OPEN QUESTION 11.
|
||||
|
||||
Core verification draws on several formal and semi-formal techniques:
|
||||
|
||||
- **Dynamic simulation** with constrained-random or directed tests, against an ISA reference model, with functional coverage.
|
||||
- **Static (formal) verification** — model checking (bounded model checking, k-induction, IC3/PDR), equivalence checking (combinational and sequential), and property proving against the RTL.
|
||||
- **Hardware-assisted verification** — FPGA prototyping and emulation (commercial emulators, open-source flows such as FireSim on Amazon EC2 F1), trading compile time for runtime speed.
|
||||
- **Co-simulation** — two simulation engines running in lockstep (e.g., an RTL simulator and an architectural reference-model engine such as Spike) synchronized at defined events. This is distinct from **reference-model checking**, in which the reference model is integrated as a transactor within a single simulator; the latter is a common industrial pattern and is more accurately described as reference-model checking, not co-simulation in the strict sense.
|
||||
- **Waveform and assertion-based methodologies** — SystemVerilog Assertions (SVA), PSL, custom monitors, end-of-test checkers.
|
||||
|
||||
Coverage is a central concern. Functional coverage, code coverage (line, branch, expression, toggle, FSM), and cross-coverage of architectural state are the standard dimensions. For a custom ISA implementation, the relationship between ISA compliance coverage and microarchitectural coverage must be defined explicitly.
|
||||
|
||||
Standard core-verification concerns that must be addressed in addition to instruction-stream behavior:
|
||||
|
||||
- **Lint** — static analysis of RTL style, common bug patterns, and synthesizability (e.g., Verilator lint, commercial lint tools). A prerequisite to dynamic and formal work.
|
||||
- **Clock-domain crossing (CDC) and reset-domain crossing (RDC)** — analysis of signals crossing asynchronous clock or reset domains. Typically owned by a separate methodology but the core must be CDC/RDC-clean for integration.
|
||||
- **X-propagation** — controlled handling of unknown ('X) propagation in simulation, using X-pessimism removal or X-init optimization, to avoid false coverage from masked X's.
|
||||
- **Reset verification** — power-on reset, software reset, debug reset, partial reset, and reset-domain interaction.
|
||||
- **Power intent** — if power features are present (see OPEN QUESTION 7), UPF/CPF power intent formats integrate with simulation to model power-gated, retention, and isolation behavior.
|
||||
- **DFT/ATPG interaction** — scan, BIST, and MBIST structures must not alter mission-mode behavior; their interaction with the core testbench is a coordination point.
|
||||
|
||||
These are not optional; their absence in a verification plan is a known gap in pre-silicon methodology.
|
||||
|
||||
## Existing Approaches
|
||||
|
||||
The following approaches are established in industry and academic practice. Their applicability to XH-1 is analyzed; none is recommended in isolation.
|
||||
|
||||
### A. Constrained-Random Simulation with UVM
|
||||
|
||||
**FACT:** Universal Verification Methodology (UVM) is standardized as IEEE 1800.2-2017, *Standard for Universal Verification Methodology* [IEEE 1800.2-2017], and is widely used in industrial block- and core-level processor verification for constrained-random stimulus generation, reference model integration, self-checking scoreboards, functional coverage, and phasing. The Accellera UVM reference implementation, which is the source code that vendors and users actually consume, is distributed under the Apache License 2.0; the IEEE 1800.2-2017 standard document itself is an IEEE publication and is not freely redistributable. The two are distinct artifacts and this distinction is load-bearing when characterizing "open-source viability" below.
|
||||
|
||||
**ASSUMPTION (conditional on design language):** UVM is applicable only if the XH-1 design language supports SystemVerilog class constructs — either the source is SystemVerilog, or the verification environment targets SystemVerilog regardless of source language. If the design language is Verilog-2001 only, or if the project commits to a Chisel/Spinal HDL or Bluespec-only flow, UVM is not directly applicable at the HDL-source level and an alternative testbench harness is required. This assumption is revisited in OPEN QUESTION 11.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Apply UVM as the primary dynamic verification environment for XH-1 core-level testing, paired with a chosen ISA reference (Spike or Sail). Sequence generation over the RISC-V ISA state space; reference model in co-simulation; scoreboard for architectural state comparison per committed instruction; explicit cover points for instruction × operand-mode × privilege × exception interactions.
|
||||
|
||||
### B. Directed and Hand-Written Tests
|
||||
|
||||
**FACT:** Constrained-random alone does not reach corner-case coverage of complex pipelines, corner-case CSR interactions, or rare exception sequences. Industry practice supplements with directed tests written by architects and designers.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** A XH-1-specific directed test suite for pipeline hazard combinations specific to the XH-1 microarchitecture, corner cases in the chosen RVA profile subset, reset, wake, low-power entry/exit, and debug transport (if present at core level).
|
||||
|
||||
### C. Formal Property Verification
|
||||
|
||||
**FACT:** Bounded model checking, k-induction, and IC3/PDR are applied to control-dominated blocks: fetch, decode, CSR, MMU/TLB, branch predictor, retirement, and exception delivery. Commercial model checkers include Cadence JasperGold, Synopsys VC Formal, and Siemens FormalPro. Open-source flows include SymbiYosys (SBY), based on the Yosys synthesis and formal toolchain. Equivalence checking is a distinct formal category: tools include Synopsys Formality and Cadence Conformal, used to verify that two RTL representations (e.g., pre- and post-synthesis, or pre- and post-retiming) are functionally equivalent. Equivalence checkers are not model checkers and are not used for property proving.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Per-block formal sign-off for the control-dominated blocks listed above, end-to-end pipeline flushing, exception priority, and CSR read/write semantics. Formal proofs of privileged ISA semantics (e.g., `mstatus`, `sstatus`, `satp`, `mip`/`sip` behavior) provide a high-confidence anchor that random tests cannot match in isolation. Sequential equivalence checking between major RTL revisions is a separate, complementary activity.
|
||||
|
||||
**ASSUMPTION (RISC-V privileged-ISA versioning):** The CSR semantics used as formal-proof targets correspond to the specific RISC-V Privileged ISA version adopted by XH-1. The unprivileged ISA, privileged ISA, and Debug specification each have versioned releases tracked by RISC-V International; the verification plan must be pinned to those versions. The specific version is OPEN until resolved (see OPEN QUESTION 1 and the Sources section).
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Coordinate with `research/10-verification/formal-verification.md` for tool selection (commercial model checkers vs. SymbiYosys / SBY) and with `research/10-verification/isa-compliance.md` for the privileged-ISA version.
|
||||
|
||||
### D. Reference-Model Checking and Co-Simulation
|
||||
|
||||
**FACT:** Comparing DUT architectural state against Spike (the official RISC-V ISA simulator, C++ implementation) or Sail (a first-order imperative language designed for ISA specification, with a RISC-V model generated from the RISC-V ISA formal definition) per instruction is a widely deployed pattern. The terminology matters:
|
||||
|
||||
- **Co-simulation (strict sense):** two independent simulation engines (e.g., an RTL simulator and Spike) running in lockstep, synchronized at defined events. The DUT may run ahead of the reference using a deferred-check queue, with results compared at synchronization points.
|
||||
- **Reference-model checking:** a single simulator that incorporates the reference model as a transactor or foreign-language call. This is a common industrial pattern but is not co-simulation in the strict sense.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Required for ISA-compliance evidence. See `research/10-verification/isa-compliance.md`. The specific architecture proposed is **reference-model checking within a single simulator** (Spike invoked as a transactor from the UVM or lightweight harness) as the default for day-to-day runs, with **co-simulation in the strict sense** retained as a cross-check on a sampled subset of tests to detect synchronization artifacts that a single-simulator harness could mask. Reference model parallelism is a 128-core scaling concern: Spike and Sail are single-threaded by default; running 128 reference models in lockstep with 128 RTL instances is itself a significant engineering problem. The pattern of choice is a single shared reference model servicing all DUT instances through a deferred-check queue, or sampling (multicore-layer decision) with the core testbench supporting the interface.
|
||||
|
||||
### E. FPGA Prototyping and Emulation
|
||||
|
||||
**FACT:** FPGA-based prototyping runs the RTL at higher throughput than software simulation, and emulation platforms (commercial: Palladium, Veloce, ZeBu; open-source: FireSim on Amazon EC2 F1) provide a middle ground. The throughput multiplier relative to software simulation is workload- and instrumentation-dependent.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Use FPGA prototyping to speed up core testbench bring-up by running entire test programs end-to-end, to enable fault-injection campaigns that are infeasible in simulation (see `research/10-verification/fault-injection.md`), and as the natural place to begin validating the multicore layer; core verification is the upstream prerequisite. Commercial emulation is a viable alternative for very large workloads or when observability requirements are relaxed; the decision depends on licensing posture (OPEN QUESTION 3) and workload characteristics. No specific throughput multiplier for the XH-1 DUT is asserted here; INSUFFICIENT EVIDENCE without XH-1-specific RTL and platform data.
|
||||
|
||||
### F. Regression Infrastructure
|
||||
|
||||
**FACT:** Regression management — version control of testbenches, seeds, baselines, coverage databases — is a load-bearing component of any non-trivial core verification program. See `research/10-verification/regression.md`.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** The XH-1 regression infrastructure must support a single-core nightly run, a multicore run, and a system run with shared seeds where appropriate.
|
||||
|
||||
### G. Coverage Closure Methodology
|
||||
|
||||
**FACT:** Coverage closure combines functional coverage (defined by the verification plan), code coverage (uncovered RTL), and assertion coverage. Industry practice treats coverage closure as a managed funnel: cover points → cross coverage → corner case identification → directed/random test addition → formal where possible. Coverage baselines drift as RTL changes; a baseline-management policy is required.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Codify the funnel explicitly in the verification plan with tracked stages, baseline-management policy, and exit criteria per stage.
|
||||
|
||||
### H. Assertion-Based Verification
|
||||
|
||||
**FACT:** SystemVerilog Assertions (IEEE 1800-2017, *Standard for SystemVerilog*) and immediate assertions encode design intent and serve both as live checkers and as inputs to formal tools.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Define an SVA library for XH-1 invariants (e.g., "no retired instruction has a privileged side effect inconsistent with its mode", "no speculative architectural state is observable at the architectural interface").
|
||||
|
||||
### I. Open-Source ISA-Level Formal Verification
|
||||
|
||||
**FACT:** The `riscv-formal` framework (YosysHQ / Symbiotic EDA, Verilog, BSD-licensed) provides a checkable RISC-V ISA specification encoded as a set of formal properties that are conjoined with a wrapper around the DUT and discharged by an SMT solver. The Kami project (Cambridge) provides a formal RISC-V ISA specification in Coq with a Bluespec front-end. INSUFFICIENT EVIDENCE to assert at the FACT level that `riscv-formal` is "directly applicable" to a specific XH-1 microarchitecture and ISA profile without further evaluation: applicability depends on the DUT's memory and CSR interface matching the framework's wrapper assumptions, on the chosen RISC-V profile, and on the unprivileged/privileged ISA versions adopted.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Evaluate `riscv-formal` as a complement to block-level formal. It targets ISA-compliance evidence with open-source tooling and is particularly relevant given the greenfield status of the XH-1 verification program, subject to the applicability evaluation above.
|
||||
|
||||
### J. Open-Source Constrained-Random Generator
|
||||
|
||||
**FACT:** Google's `riscv-dv` is an open-source UVM-based constrained-random instruction-stream generator for RISC-V, widely used in industrial and academic flows. The license should be confirmed against the current repository state at the time of toolchain selection; this document does not assert a specific license version.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Evaluate `riscv-dv` as a generator either within a UVM environment or as a stimulus source for a lightweight harness.
|
||||
|
||||
### K. RISC-V Architectural Test Framework
|
||||
|
||||
**FACT:** RISC-V International maintains `riscv-arch-test` as the architectural test framework, which is the conformance test suite against which cores are signed off for profile compliance. `riscv-tests` is a related umbrella repository containing `riscv-arch-test` as well as additional tests; the relationship is not "legacy vs. replacement" — `riscv-tests` is the broader collection and `riscv-arch-test` is the architectural conformance subset. The two should be treated as distinct but related inputs, both of which produce evidence in the sign-off record. The framework's coverage analyzer, `RISCV-ISAC`, is an instruction-stream coverage analyzer used to confirm that the architectural test signatures cover the required instruction/operand/privilege combinations; it is not a generic functional-coverage database and is not a substitute for the core testbench's own coverage model.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Use `riscv-arch-test` for the ISA-compliance certificate, with `RISCV-ISAC` analysis of the architectural test signatures as part of the coverage evidence, and `riscv-tests` more broadly as a regression input. The license and current revision of each must be confirmed at toolchain selection.
|
||||
|
||||
## Alternative Designs
|
||||
|
||||
The following alternative *strategies* for the core verification methodology itself are candidates for XH-1. They are not mutually exclusive; the project must select a composition.
|
||||
|
||||
### Alternative 1: UVM-Centric with Formal Supplements
|
||||
|
||||
- **Approach:** UVM as the workhorse for dynamic verification, with formal applied to control-dominated blocks.
|
||||
- **Strengths:** Industry-standard methodology, well-understood coverage and sign-off conventions, mature ecosystem; integrates with `riscv-dv` as a generator and `riscv-formal` as an ISA-level formal layer.
|
||||
- **Weaknesses:** UVM has a steep learning curve and high setup cost. For a 128-core design, the UVM environment must be designed for reuse to avoid 128× rewrite cost. Requires SystemVerilog (OPEN QUESTION 11). Implies commercial simulator licensing in most deployments, because open-source SystemVerilog simulation support (Verilator) has limited UVM-class support relative to commercial offerings; the open-source viability of this alternative is therefore medium, not high.
|
||||
|
||||
### Alternative 2: Lightweight Python/C++ Test Harness with Reference-Model Checking
|
||||
|
||||
- **Approach:** A generator produces instruction streams and a C++ testbench (e.g., based on cocotb, Verilator, or a custom framework) compares architectural state against Spike.
|
||||
- **Strengths:** Lower setup cost than UVM. Excellent generator productivity. Easy integration with regression infrastructure. Open-source-friendly. `riscv-dv` can serve as the generator. Verilator-based flows are not bound to SystemVerilog UVM at the source level.
|
||||
- **Weaknesses:** Verilator's debug, SVA, and functional-coverage support are not equivalent to commercial simulators; cocotb+Verilator is therefore not a drop-in replacement for UVM, and the trade-off should be evaluated against the XH-1 debug and coverage requirements. Coverage closure tooling is less standardized than the UVM vendor ecosystem. The reference-model-checking pattern (Section D) is the natural fit for this harness.
|
||||
- **Application to XH-1:** Reasonable choice if XH-1 has a small team, an open-source toolchain preference, or a constraint on commercial EDA licensing.
|
||||
|
||||
### Alternative 3: Formal-First
|
||||
|
||||
- **Approach:** Formal proofs form the primary verification evidence; dynamic simulation is used for reach and to validate properties that formal cannot discharge.
|
||||
- **Strengths:** Highest assurance per proven property. Strong fit for control logic and CSR behavior. `riscv-formal` provides an open-source ISA-level formal layer subject to the applicability evaluation in Section I. Under the replication-homogeneity assumption, proven properties transfer to all 128 instances at near-zero marginal *re-proof* cost (the proof is re-run, not re-constructed, against another elaborated instance of the same RTL); the engineering cost of constructing the proof is paid once.
|
||||
- **Weaknesses:** Pipeline formalization is hard. State-space complexity grows combinatorially for large cores. May not reach proof closure on timing-agnostic models without abstraction. Inconclusive results are not equivalent to failure; risk-acceptance must be tracked.
|
||||
|
||||
### Alternative 4: Emulation-Centric
|
||||
|
||||
- **Approach:** Skip large-scale software simulation; use FPGA/emulation throughput as the primary engine.
|
||||
- **Strengths:** Excellent long-workload capability; native fault-injection support.
|
||||
- **Weaknesses:** Poor waveform/observability; slow compile/turnaround; harder to drive corner-case stimulus; debug cycle is slow.
|
||||
|
||||
### Alternative 5: Hybrid (PROPOSAL — See Proposed Direction)
|
||||
|
||||
A combination of a dynamic framework (Alternative 1 or Alternative 2) for stimulus and reach, formal supplements (Alternative 3, including `riscv-formal` where applicable) for control logic, and FPGA prototyping/emulation for throughput and multicore bring-up.
|
||||
|
||||
## Comparison
|
||||
|
||||
The comparison is qualitative and is intended to make the trade-offs explicit, not to produce a numeric ranking. INSUFFICIENT EVIDENCE exists to rank absolute quality without XH-1-specific RTL data, tool licensing posture, and team size. The Hybrid column is a composition of the others and is not an independent point of comparison; the table is structured to show how each pure strategy performs on the dimensions, with the Hybrid column indicating how the strengths compose.
|
||||
|
||||
| Dimension | UVM-Centric | Lightweight Harness | Formal-First | Emulation-Centric | Hybrid |
|
||||
|---|---|---|---|---|---|
|
||||
| Setup cost | High | Low | High | Medium | High |
|
||||
| Per-test runtime | Slow | Slow–Medium | N/A (proof time) | Fast | Mixed |
|
||||
| Corner-case reach | Strong (with seeds) | Strong (with fuzzing) | Strong (when discharged) | Weak–Medium | Strong |
|
||||
| Assurance on control logic | Medium | Medium | High | Low–Medium | High |
|
||||
| Debug ergonomics | Good (waveform, logs) | Good (caveat: Verilator/cocotb debug support not equivalent to commercial) | Strong counterexamples | Weak | Mixed |
|
||||
| Open-source viability | Medium (UVM standard is IEEE-published; UVM reference implementation is Apache 2.0; simulators are usually commercial) | Strong | Medium | Medium | Medium |
|
||||
| Fit for 128-core reuse | High (if designed so) | High (if designed so) | High (under replication homogeneity) | High | High |
|
||||
| Pre-tape-out risk reduction | Strong | Medium–Strong | Strong (partial; inconclusive ≠ proven) | Medium | Strong |
|
||||
|
||||
## Advantages
|
||||
|
||||
The advantages of investing in a robust core verification methodology for XH-1:
|
||||
|
||||
- **Replication leverage on defect-escape cost.** Under the replication-homogeneity assumption, an unfixed defect in the core RTL is replicated 128-fold in silicon, multiplying its downstream (silicon-debug, re-spin, field-failure, yield-loss) cost. Verification effort itself is paid once against one RTL; the defect, if found, is fixed once; the fixed RTL is then replicated. The benefit of core-level detection is the avoidance of multiplied post-silicon cost.
|
||||
- **Cleaner multicore verification.** A well-verified core reduces the multicore verification problem to coherence, consistency, interconnect, and shared resources. See `research/10-verification/multicore-verification.md`.
|
||||
- **Foundation for post-silicon.** First-stage silicon bring-up is dramatically simpler when the core is well-understood and the debug interfaces (assertions, traces, scan) are well-designed. Post-silicon validation feeds back into pre-silicon coverage targets; this feedback loop is not detailed in this document and is left to the post-silicon layer.
|
||||
- **Reusable testbench assets.** Generator, scoreboard, coverage model, and SVA library can be reused across cores if XH-1 has core variants, or across revisions of the same core.
|
||||
- **Formal evidence on critical control.** Privileged ISA semantics are hard to verify by random simulation; formal proofs on these blocks provide a high-confidence floor.
|
||||
|
||||
## Disadvantages
|
||||
|
||||
Costs and risks of a robust core verification program:
|
||||
|
||||
- **Engineering cost.** A UVM-class environment plus formal plus emulation is a substantial program. The claim that it "can easily exceed the RTL design cost for a single core" is folkloric and is presented here as a planning consideration, not as a measured fact.
|
||||
- **Tooling lock-in.** Heavy UVM usage often implies commercial simulator licensing; this may not match XH-1's tool posture. Formal tool lock-in is similar: commercial model checkers differ in property language and run scripts.
|
||||
- **Slow turnaround.** Constrained-random simulation with full coverage at the core level is slow; coverage closure can take many cycles of "add test, rerun, close hole". Coverage baselines drift with RTL changes, requiring re-establishment.
|
||||
- **Reference model dependence.** Spike, Sail, or any reference may diverge from the RISC-V specification in interpretation of ambiguous or profile-specific corners; the verification program must track the chosen RVA profile carefully and pin the spec version of the reference.
|
||||
- **Coverage false confidence.** Hitting a cover point is not the same as proving correctness; the verification plan must distinguish cover and check.
|
||||
- **128-core replication pressure.** If the core testbench is not designed to compose with the multicore testbench, redundant effort emerges.
|
||||
- **Inconclusive formal ≠ proven.** Formal results of "inconclusive" must be risk-accepted, not treated as success.
|
||||
- **Equivalent mutants.** Mutation-kill rates are affected by equivalent mutants (mutations that produce behaviorally identical RTL); kill-rate targets are therefore noisy and must be interpreted with caution.
|
||||
|
||||
## XH-1 Considerations
|
||||
|
||||
XH-1-specific factors that constrain the methodology choice:
|
||||
|
||||
- **128-core replication.** The verification plan must be replication-aware: the *testbench* should be agnostic to whether one or 128 cores are instantiated. This implies strong separation of (a) the core DUT, (b) the core testbench, and (c) the multicore wrapper.
|
||||
- **Custom design.** No third-party core is being verified; there is no vendor's pre-built verification IP. All environments must be built or assembled.
|
||||
- **Repository context.** The related verification documents are all placeholders ("SOON"). This is a greenfield verification program; the XH-1 project has the opportunity — and the burden — to define the methodology from first principles.
|
||||
- **RISC-V ISA.** The RISC-V ecosystem provides mature reference models (Spike, Sail), compliance / architectural test frameworks (`riscv-arch-test`, `riscv-tests`), and open-source verification tooling (`riscv-formal`, `riscv-dv`, `RISCV-ISAC`). These are the natural anchors.
|
||||
- **Tool posture (OPEN QUESTION 3).** Without evidence in the repository, the project's tool posture (open-source only, commercial EDA, mixed) is OPEN. The methodology composition must remain valid across posture choices.
|
||||
- **Design language (OPEN QUESTION 11).** The choice of design language — SystemVerilog, traditional Verilog, Chisel/Spinal HDL, Bluespec — has direct implications for verification methodology (e.g., formal property availability on generated RTL, simulation throughput, the place of unit-level testing at the HDL-source level vs. the generated-RTL level, and the applicability of UVM at the source level). This is OPEN pending XH-1 design-language disclosure. Section A's UVM assumption is conditional on this resolution.
|
||||
- **Microarchitectural class (OPEN QUESTION 2).** The pipeline class — in-order vs. out-of-order, superscalar width, depth of speculation, branch-prediction aggressiveness — materially changes the cost-benefit of formal vs. dynamic and the complexity of pipeline flush / replay verification. The methodology composition must accommodate both extremes.
|
||||
- **Power architecture (OPEN QUESTION 7).** Whether the core supports clock gating, power gating, retention, and/or DVFS is OPEN. The verification methodology must accommodate each combination. Assuming any specific power feature is invalid without evidence. Power intent (UPF/CPF) integration is conditional on this.
|
||||
- **Target implementation class (OPEN QUESTION 12).** Whether XH-1 targets ASIC tape-out or FPGA-only deployment fundamentally changes the verification methodology (DFT/ATPG scope, post-silicon layer, sign-off criteria). Treated here as an assumption (ASIC) pending resolution.
|
||||
|
||||
## 128-Core Scalability
|
||||
|
||||
Replication does not change which verification techniques apply to a single core, but it changes their cost-benefit and their composition. Throughout this section, the two cost quantities from the Background are kept distinct: **defect-escape cost** (multiplied by replication count under the homogeneity assumption) versus **verification effort cost** (paid against one RTL).
|
||||
|
||||
- **Core testbench scalability.** A core testbench written for a single instance must be instantiable inside a multicore wrapper without modification. This implies the testbench interfaces to the core through a standard transaction-level interface (or at least a clearly defined boundary), not through global signals.
|
||||
- **Cost-benefit of formal at replication.** Per-block formal verification has high fixed engineering cost (constructing the proof), but under bit-identical replication, the *re-proof* cost against another elaborated instance of the same RTL is near zero: once a control property is proven, re-running the proof against a re-elaborated instance of the same source is mechanical. This is a property of the proof, not of the underlying effort. If the source is parameterized and elaborated with per-instance parameters that differ, the proof must be re-established for any parameter set that differs. The defect-escape cost argument is the dominant benefit: a proven property that prevents a class of bug multiplies that benefit by the replication count.
|
||||
- **Cost-benefit of dynamic at replication.** Dynamic simulation at full coverage also has its verification effort paid against one RTL, but full per-instance coverage at 128 instances requires running 128 instances through the same stimulus, which is a real additional compute cost (not strictly linear in wall-clock because the simulator elaborates them in the same run, but real in host compute and memory). The trade-off is a primary argument for the hybrid approach at 128-core scale, and the practical mitigation is sampling, not full per-instance coverage.
|
||||
- **Workload scaling.** Running the full core testbench in a 128-core environment is generally unnecessary; sampling (verify a subset of cores under full stimulus, sweep register state across all 128) is a more efficient pattern. The sampling strategy is a multicore concern but must be supported by the core testbench's reset/configuration interface. The statistical argument for the chosen sampling fraction (defect-escape probability vs. simulation cost) is a multicore-layer deliverable; the core testbench must support the interface.
|
||||
- **Coverage aggregation.** Per-core coverage data must be aggregated across the 128 instances. The core testbench should emit coverage in a form that is identifiable by instance index.
|
||||
- **Debug bandwidth.** 128 cores each generating waveform/log data is impractical. The core testbench must distinguish between detailed-trace mode (one core, full visibility) and lightweight-trace mode (128 cores, summary only).
|
||||
- **Compile and elaboration cost.** Elaboration of a 128-core testbench can dominate runtime. The core testbench should be elaborable standalone and in a multicore wrapper without recompiling the core RTL. Elaboration-time behavior (reset, clock tree, memory subsystem) can differ between 1-core and 128-core contexts; the core testbench must be valid in both.
|
||||
- **Testbench memory footprint.** 128 instances of a complex scoreboard and 128 reference models can dominate host memory. Memory-aware testbench design (shared reference models, deferred-check queues, summary-only trace mode) is required. This is acknowledged here and detailed in the multicore layer.
|
||||
- **Reference-model parallelism.** Spike and Sail are single-threaded by default. Running 128 reference models in lockstep with 128 RTL instances is a significant engineering problem. The recommended pattern is a single shared reference model servicing all DUT instances through a deferred-check queue, or sampling. This is a multicore-layer design constraint that the core testbench interface must support.
|
||||
- **Symmetry assumption revisit.** The replication-homogeneity assumption is load-bearing throughout this section. Heterogeneous cores (e.g., big/little, mixed application/accelerator, per-core ISA-subset configuration) invalidate the per-die marginal-cost argument and require per-variant verification, including per-variant testbenches, per-variant coverage, and per-variant formal. The methodology should be designed so that the additional cost of a variant is linear in the number of variants, not linear in 128 × variants.
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
The following are framework-level considerations. INSUFFICIENT EVIDENCE exists to provide numerical throughput estimates without XH-1-specific RTL data, simulator choice, and toolchain selection; any specific number is unsourced and is therefore not asserted here.
|
||||
|
||||
- **Testbench throughput.** Dynamic simulation throughput determines how many random seeds can be run per unit wall-clock. Constrained-random at the core level is sensitive to simulator choice, RTL abstraction level, and coverage instrumentation. ASSUMPTION: representative numbers are available in vendor documentation (commercial simulators) and in the open-source Verilator and cocotb benchmark literature, but are workload-dependent and cannot be quoted as typical without measurement on the XH-1 DUT. No specific throughput range is asserted.
|
||||
- **Compiler/optimizer.** Coverage instrumentation slows simulation. Production coverage runs and fast-turnaround debug runs should be separable.
|
||||
- **Co-simulation overhead.** PROPOSAL: Spike co-simulation overhead per committed instruction is non-trivial in co-simulation in the strict sense (two-engine lockstep); under the reference-model-checking pattern adopted in Section D, the overhead is a foreign-language call per synchronization point. Synchronization strategy (per-instruction vs. batched) is a tradeoff between debug granularity and throughput. At 128 instances, the cost of a per-instruction synchronization across all instances is prohibitive; batched or sampled synchronization is the practical pattern.
|
||||
- **FPGA prototype throughput.** FPGA prototype throughput is set by Fmax, which is determined by LUT count, routing, and the memory subsystem. Quoted MHz figures for an XH-1 core on a specific FPGA platform are not asserted here; INSUFFICIENT EVIDENCE without platform and synthesis data.
|
||||
- **Regression wall-clock budget.** A nightly regression that runs N seeds on a single core must fit within the available compute budget. Cloud burst capacity is typical.
|
||||
|
||||
INSUFFICIENT EVIDENCE to provide numerical throughput estimates without XH-1-specific RTL data and toolchain selection.
|
||||
|
||||
## Area Considerations
|
||||
|
||||
- **On-die verification infrastructure.** XH-1 is described as a 128-core design. The verification *methodology* does not directly consume die area, but decisions made during verification (e.g., embedding assertions in RTL, adding observability muxes for trace, adding per-core DFT structures) do.
|
||||
- **Trace and debug infrastructure.** Core-level verification informs the design of trace, performance counters, and debug-transport blocks. Larger trace infrastructure costs area; the verification program must justify the cost in terms of debug productivity and post-silicon coverage.
|
||||
- **DFT (Design-for-Test) compatibility.** Scan chains, BIST, and MBIST are typically owned by a separate DFT program. Core verification should ensure that the core's DFT structures do not alter functional behavior in mission mode. If XH-1 is FPGA-only (OPEN QUESTION 12), DFT is out of scope.
|
||||
- **Replication-driven area optimization.** If the core's area is a major constraint (which is likely at 128 cores on a single die), the verification plan must protect against defects that would force a re-spin with re-spinned core area.
|
||||
|
||||
INSUFFICIENT EVIDENCE to quantify area without access to XH-1's core area budget or process node.
|
||||
|
||||
## Power and Energy Considerations
|
||||
|
||||
The following considerations apply **if and only if** XH-1 implements the corresponding power features. Each is OPEN QUESTION 7-conditional.
|
||||
|
||||
- **Power-aware verification (conditional).** If the core supports power gating or retention, the core testbench must be able to drive representative switching activity to enable power estimation, and the testbench must verify that power-gated domains behave correctly (controlled wake, retention restore, ordering with respect to coherence traffic). UPF/CPF power intent integration with the simulator is the standard mechanism.
|
||||
- **Energy proportionality (conditional).** If the core supports clock gating at the instruction or block level, the testbench should verify that idle and active transitions behave correctly at the architectural interface (no observable state loss in idle, correct wake on interrupt).
|
||||
- **Thermal and DVFS interactions (conditional).** If XH-1 supports DVFS at the core level, voltage/frequency transitions and their effects on memory coherence, pipeline flush behavior, and CSR access must be verified. This is a cross-cutting concern between core and system verification.
|
||||
- **Power estimation accuracy.** Pre-silicon power estimation depends on workload realism. A core testbench that only runs synthetic random code will produce misleading power numbers.
|
||||
|
||||
No power feature is assumed. Verification provisions for any power feature must be activated only when OPEN QUESTION 7 is resolved.
|
||||
|
||||
## Implementation Considerations
|
||||
|
||||
- **Methodology composition.** The project must select one of the alternative strategies (or a hybrid) and codify it in the verification plan.
|
||||
- **Toolchain selection.** Open-source (Verilator, cocotb, SymbiYosys, `riscv-formal`, `riscv-dv`, Spike, RISCV-ISAC, GTKWave) vs. commercial (Synopsys VCS, Cadence Xcelium, Siemens Questa, JasperGold, VC Formal, FormalPro, Formality, Conformal, Palladium, Veloce, ZeBu) is a strategic decision. The choice cascades through licensing, hiring, training, and CI. These categories are not equivalent in role: Verilator and cocotb are simulation/cosim frameworks; SymbiYosys is a formal-verification flow; `riscv-formal` is an ISA-level formal property set; `riscv-dv` is a stimulus generator; RISCV-ISAC is an instruction-stream coverage analyzer for the architectural test framework; commercial emulators are hardware-assisted verification platforms distinct from FPGA prototyping. The choice within each role is independent.
|
||||
- **Reference model choice.** Spike (C++, instruction-accurate) vs. Sail (first-order imperative ISA-specification language with a RISC-V model) vs. a custom golden model. Spike is the most common reference; the project must document any corner-case behaviors of the chosen reference against the RISC-V specification.
|
||||
- **Coverage database.** Coverage data are stored in vendor-specific formats (e.g., Synopsys UCDB / DDC, Cadence IMC, Siemens UCDB) and may be aggregated into a custom backend (e.g., SQLite) for CI. The format is a tool-vendor concern; the aggregation backend is a CI concern. These are distinct choices.
|
||||
- **Test categorization.** Tests must be tagged by domain (base integer, M-ext, A-ext, F/D-ext, privileged, vector, …), by intent (sanity, corner, regression, performance), and by status (passing, known-fail, expected-fail).
|
||||
- **Seed management.** Random seeds should be reproducible. Seed pools should be partitioned into "core regressions", "block regressions", and "fuzzing".
|
||||
- **Bring-up sequencing.** Block-level → core-level → multicore → system. The core testbench is the transition point between block-level and system-level work.
|
||||
|
||||
## Sign-Off Criteria
|
||||
|
||||
A core verification sign-off is the point at which the methodology exits the coverage-closure funnel. The following are candidate criteria; **every numeric threshold below is an illustrative starting point and is not a validated target**. The relationship to DRAFT status is restated at each threshold so that the list is internally consistent with the document status header.
|
||||
|
||||
- **Functional coverage.** All defined cover points hit, with cross-coverage exercised. **Illustrative starting threshold (not a validated target):** ≥ 95% cover-point hit, ≥ 90% cross-coverage hit; remaining items must be justified as unreachable or formally proven. Final threshold to be set in the verification plan.
|
||||
- **Code coverage.** Statement, branch, expression, toggle, and FSM coverage reported; excluded regions justified. **Illustrative starting threshold (not a validated target):** ≥ 95% statement, ≥ 90% branch, with manual exclusion of reset/DFT/clock-gating logic. Final threshold to be set in the verification plan.
|
||||
- **Assertion coverage.** All SVA / immediate assertions vacuously satisfied or actively proven; assertion cover points hit. Open assertions must be reviewed and dispositioned.
|
||||
- **Formal closure.** Per-block formal sign-off, with proof closure recorded for each property. **Operational requirement:** every inconclusive formal result must have a documented risk-acceptance disposition (strengthen via abstraction/decomposition, or accept with justification) recorded in the sign-off package. "Inconclusive" is not equivalent to "proven".
|
||||
- **ISA compliance evidence.** All applicable tests in `riscv-arch-test` pass and produce the architectural test framework's signature. `riscv-tests` (broader collection including `riscv-arch-test`) is run as a regression input. Signed certificate or equivalent record retained. **Prerequisite:** the specific RISC-V ISA, Privileged ISA, Debug, and Profile versions adopted by XH-1 are pinned in the verification plan.
|
||||
- **Mutation testing.** Periodic injection of known-defect mutations into a frozen RTL revision. **Illustrative starting threshold (not a validated target):** ≥ 80% of injected mutations detected by the testbench + assertions + formal. This number is a starting point and requires empirical validation on the XH-1 DUT. Equivalent-mutant handling (mutations that produce behaviorally identical RTL) must be defined; kill-rate targets are noisy and must be interpreted with caution. Final threshold and operator set to be set in the verification plan.
|
||||
- **Reference model validation.** Spike or Sail validated against the chosen RISC-V compliance tests before being used as a checker; validation evidence archived.
|
||||
- **Testbench self-check verification.** Every scoreboard check has at least one negative test that is expected to fail.
|
||||
- **Reproducibility.** Every sign-off run is reproducible from committed source, seed, and tool version.
|
||||
- **CDC/RDC closure.** CDC and RDC analysis clean for all core-level crossing structures.
|
||||
- **Lint clean.** Lint clean at the policy-defined severity level.
|
||||
- **Reset verification.** All reset domains and reset sequences verified; reset recovery verified.
|
||||
|
||||
## Verification Considerations
|
||||
|
||||
Core verification is itself subject to verification — the verification of the verification environment:
|
||||
|
||||
- **Testbench self-checking.** All scoreboard checks must be end-of-test verifiable; every check should be paired with at least one negative test that is expected to fail.
|
||||
- **Reference model validation.** The reference model (Spike, Sail) must be validated against the chosen RISC-V compliance tests before being used as a checker.
|
||||
- **Coverage sanity.** Cover points must be reviewed to ensure they are not trivially self-firing.
|
||||
- **Mutation testing.** A subset of RTL defects should be intentionally injected to confirm the testbench catches them. See `research/10-verification/fault-injection.md`. Equivalent-mutant handling must be defined.
|
||||
- **Coverage baseline stability.** Coverage baselines drift as RTL changes; a baseline-management policy (when a baseline is re-established, by whom, with what approval) is required.
|
||||
- **Tool qualification.** If the verification tools are used in any safety-relevant context, tool qualification (per ISO 26262, IEC 61508, DO-254, or similar) is required. The applicability of these standards to XH-1 is OPEN QUESTION 8. Tool qualification is a verification-of-verification concern, not only a sign-off checkbox.
|
||||
- **Version control and baselines.** Every test run must be reproducible from committed source. Coverage baselines must be versioned.
|
||||
- **Testbench verification at 128-instance scale.** The testbench itself has bugs. Verification of the testbench (negative tests, mutation of the testbench, equivalence of testbench revisions) is part of the methodology. At 128 instances, the testbench's own correctness becomes a non-trivial verification problem and is acknowledged here as a multicore-layer concern that the core testbench interface must support.
|
||||
|
||||
## Software Considerations
|
||||
|
||||
- **Toolchain integration.** The core verification environment must integrate with the software toolchain (compiler, assembler, linker, Spike, debug server) to enable reference-model checking / co-simulation and to run real programs.
|
||||
- **Firmware.** Firmware that runs on the core (boot code, exception handlers, interrupt dispatchers) is itself verification target; the core testbench should support loading and running firmware images.
|
||||
- **Operating system implications.** A core that supports an OS (Linux, RTOS) must be verified with syscall-heavy workloads. The core testbench should provide OS-aware run modes.
|
||||
- **Debug and trace protocols.** The core's debug-transport module (per the RISC-V Debug specification) must be exercised. JTAG or cJTAG interactions with the core's debug module are typically a core-level concern.
|
||||
- **Performance analysis tools.** Tools that analyze trace data, performance counters, and profile information should be able to consume the artifacts the core testbench produces.
|
||||
- **Software developer ergonomics.** The verification team's interfaces (test writers, coverage reviewers) should be ergonomic. A "developer can run a single test and see results in under a minute" goal is a useful operational target.
|
||||
|
||||
## Security Considerations (Out of Scope, but Flagged)
|
||||
|
||||
**ASSUMPTION (out-of-scope for this revision):** Transient-execution / side-channel verification (e.g., Spectre-class, Meltdown-class, and constant-time verification) is treated as out of scope for the XH-1 core verification layer for this revision. For a 128-core design with privilege boundaries, this is a real and high-risk concern. This is flagged as a high-risk gap and as a candidate for a follow-up document. The current methodology is sufficient for functional and microarchitectural correctness but does not, by itself, provide security assurance.
|
||||
|
||||
## Proposed Direction
|
||||
|
||||
PROPOSAL — A **hybrid methodology** for XH-1 core verification, with the following structure:
|
||||
|
||||
1. **Dynamic simulation** as the primary verification engine, with the framework (UVM vs. lightweight) gated on OPEN QUESTION 3 (tool posture), OPEN QUESTION 4 (team size and timeline), and the design-language choice (OPEN QUESTION 11).
|
||||
- If UVM: pair with `riscv-dv` as the instruction-stream generator and Spike or Sail as the reference model.
|
||||
- If lightweight: pair `riscv-dv` (or a custom generator) with cocotb / Verilator and Spike, with explicit acknowledgment of Verilator's reduced debug, SVA, and coverage support relative to commercial simulators.
|
||||
2. **Formal verification** applied to control-dominated blocks: fetch, decode, CSR file, MMU/TLB, exception/interrupt delivery, and the debug module. Coordinate with `research/10-verification/formal-verification.md` for tool selection. Apply `riscv-formal` as an ISA-level formal layer subject to the applicability evaluation in Section I. Sequential equivalence checking between major RTL revisions is a complementary activity.
|
||||
3. **Reference-model checking against a RISC-V reference model** (Spike or Sail) as the default, with **co-simulation in the strict sense** retained as a sampled cross-check (per Section D). Coordinate with `research/10-verification/isa-compliance.md`. Pin the RISC-V ISA, Privileged ISA, Debug, and Profile versions. At 128 instances, use a shared reference model with a deferred-check queue or sampled synchronization.
|
||||
4. **Directed tests** for XH-1 microarchitectural corner cases that random cannot reach economically. The corner cases to target are conditional on the in-order / out-of-order answer in OPEN QUESTION 2.
|
||||
5. **FPGA prototyping** for throughput-driven verification and as a substrate for fault injection and multicore bring-up. Commercial emulation is a conditional alternative based on licensing posture (OPEN QUESTION 3) and workload.
|
||||
6. **Regression infrastructure** with version-controlled seeds, baselines, and coverage data. Coordinate with `research/10-verification/regression.md`.
|
||||
7. **Coverage closure** as an explicit, tracked funnel with the sign-off criteria in the Sign-Off Criteria section, including baseline-management policy. Numeric thresholds are illustrative starting points and are not validated targets; final thresholds are deferred to the verification plan.
|
||||
8. **Reusable testbench architecture** so the core testbench composes inside the multicore testbench without modification, supporting both the per-die cost argument and the sampling strategy. The interface must support shared reference models, summary-only trace mode, and per-instance coverage identification.
|
||||
9. **Mutation testing** as a periodic check that the testbench is still doing useful work, with explicit equivalent-mutant handling. Coordinate with `research/10-verification/fault-injection.md`.
|
||||
10. **Standard core-verification adjuncts:** lint, CDC/RDC analysis, X-propagation handling, reset verification, and (conditional on OPEN QUESTION 7) UPF/CPF power-intent integration.
|
||||
|
||||
The hybrid is proposed; the specific framework within the hybrid (UVM vs. lightweight) is gated on OPEN QUESTION 3, OPEN QUESTION 4, and OPEN QUESTION 11, and the power-feature sub-portion is gated on OPEN QUESTION 7. The implementation-class sub-portion (DFT/ATPG scope, post-silicon layer) is gated on OPEN QUESTION 12. The document does not pick between alternatives that the OPEN QUESTIONS do not yet support.
|
||||
|
||||
## Confidence
|
||||
|
||||
- **High confidence:** The general methodology composition (dynamic + formal + reference-model checking / co-simulation + directed + prototype + regression + coverage closure, with the standard adjuncts: lint, CDC/RDC, X-propagation, reset) corresponds to the IEEE 1800.2-2017 UVM standard, the IEEE 1800-2017 SystemVerilog assertion subset, and the RISC-V International published ISA / Privileged / Debug / Profile specifications, and is broadly accepted in industrial practice. The references supporting each component are cited at the point of use.
|
||||
- **High confidence (conditional):** The 128-core scaling argument on defect-escape cost (an undetected defect is replicated 128-fold in silicon) is a direct consequence of the replication-homogeneity assumption and does not require empirical data. The argument about formal re-proof cost under bit-identical replication (near-zero marginal cost to re-run a proof against another elaborated instance of the same source; engineering cost of constructing the proof is paid once) is also a direct consequence of the assumption, with the parameter-set caveat acknowledged. The argument about dynamic simulation at full per-instance coverage requiring real additional compute is similarly structural.
|
||||
- **Medium confidence:** Specific tool selections, sign-off numeric thresholds, and area/power tradeoffs, given the absence of repository evidence on the OPEN QUESTIONS.
|
||||
- **Low confidence (OPEN):** XH-1's tool licensing posture, core microarchitecture details, ISA profile selection, team size, design language, power architecture, and target implementation class. These drive the specifics within the proposed hybrid.
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. What RISC-V profile does the XH-1 core implement, and what is the specific version pin for the Unprivileged ISA, Privileged ISA, and Debug specification? The reference model (Spike variant, Sail configuration) and the compliance test scope follow from this choice. The RISC-V International profile document should be consulted at toolchain selection to determine the exact profile designation adopted (e.g., a 64-bit XLEN core targeting an RVA application/supervisor profile, or a 32-bit XLEN core targeting a different profile track); this document does not assert a specific profile name as established fact.
|
||||
2. Is the XH-1 core in-order or out-of-order? The verification plan complexity differs substantially. The instructions explicitly disallow assuming either without repository evidence.
|
||||
3. What is XH-1's toolchain posture (open-source only, commercial EDA, mixed)?
|
||||
4. What is the team size and timeline? Methodology cost scales with team capacity.
|
||||
5. What is the core's microarchitectural contract — what blocks are in scope for unit verification, and what is the boundary between core and multicore? See `research/10-verification/unit-verification.md` and `research/10-verification/multicore-verification.md`.
|
||||
6. Are the 128 cores architecturally identical, or are there variants (big/little, mixed application/accelerator, per-core ISA-subset configuration)? Heterogeneity changes the testbench architecture and invalidates the per-die cost-leverage argument.
|
||||
7. What is the core's power architecture — is clock gating, power gating, retention, or DVFS at the core level, and what are the state-retention semantics?
|
||||
8. What is XH-1's relationship to functional safety standards (ISO 26262, IEC 61508, DO-254, or none)?
|
||||
9. How will the XH-1 core's trace, debug, and performance-counter interfaces be designed, and how will core verification consume them?
|
||||
10. What is the expected defect rate target at tape-out, and how does the verification program demonstrate closure against that target?
|
||||
11. What is the XH-1 design language (SystemVerilog, Verilog-2001, Chisel/Spinal HDL, Bluespec), and does verification target the HDL source level, the generated-RTL level, or both?
|
||||
12. What is XH-1's target implementation class — ASIC tape-out, FPGA-only deployment, or both? This determines DFT/ATPG scope, post-silicon validation depth, and sign-off criteria.
|
||||
13. What physical implementation tool flow (synthesis, PnR, timing) is envisioned, and what are the timing-closure constraints (target clock frequency, slack budget)?
|
||||
|
||||
## Sources
|
||||
|
||||
- IEEE 1800.2-2017, *Standard for Universal Verification Methodology*. Defines UVM. The Accellera UVM reference implementation is distributed under the Apache License 2.0; the IEEE 1800.2-2017 standard document is an IEEE publication.
|
||||
- IEEE 1800-2017, *Standard for SystemVerilog — Unified Hardware Design, Specification, and Verification Language*. Defines SVA and immediate assertions.
|
||||
- RISC-V International, *The RISC-V Instruction Set Manual, Volume I: Unprivileged Architecture*. **Version pin: OPEN** — to be selected at toolchain selection and recorded in the verification plan.
|
||||
- RISC-V International, *The RISC-V Instruction Set Manual, Volume II: Privileged Architecture*. **Version pin: OPEN** — to be selected at toolchain selection and recorded in the verification plan.
|
||||
- RISC-V International, *RISC-V External Debug Support*. **Version pin: OPEN** — to be selected at toolchain selection and recorded in the verification plan.
|
||||
- RISC-V International, *RISC-V Profiles*. **Version pin: OPEN** — the specific profile designation adopted by XH-1 is OPEN until OPEN QUESTION 1 is resolved. The published RISC-V profile document should be consulted directly at toolchain selection; this document does not assert a specific profile name.
|
||||
- Symbiotic EDA / YosysHQ, *SymbiYosys* documentation (open-source formal verification flow).
|
||||
- `riscv-formal` framework (Verilog, BSD-licensed), maintained at github.com/YosysHQ/riscv-formal.
|
||||
- `riscv-dv` instruction-stream generator, maintained at github.com/google/riscv-dv.
|
||||
- `riscv-tests` and `riscv-arch-test`, maintained by RISC-V International. `riscv-arch-test` is the architectural conformance test framework contained within the broader `riscv-tests` repository.
|
||||
- Spike (the official RISC-V ISA simulator), maintained at github.com/riscv-software-src/riscv-isa-sim.
|
||||
- Sail RISC-V formal specification, maintained at github.com/rems-project/sail. Sail is a first-order imperative ISA-specification language; the RISC-V model is generated from the RISC-V ISA formal definition.
|
||||
- RISCV-ISAC, instruction-stream coverage analyzer for the RISC-V architectural test framework, maintained by RISC-V International.
|
||||
|
||||
Quantitative throughput, area, power, and timing claims are not asserted in the body. The XH-1 repository documents referenced (under `research/10-verification/`) are placeholders ("SOON") and do not provide factual content to cite. No measurements, benchmarks, processor capabilities, URLs outside the repositories named above, or experimental results have been invented.
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
VERDICT: FAIL
|
||||
|
||||
ISSUES:
|
||||
|
||||
1. **Unsupported quantitative claim (Sign-Off Criteria).** The document states "≥ 95% cover-point hit, ≥ 90% cross-coverage hit," "≥ 95% statement, ≥ 90% branch," and "≥ 80% of injected mutations detected" while also disclaiming these are "illustrative starting points and not validated targets." A reviewer cannot independently verify that these specific numbers are reasonable industry starting points; they are presented as if they were standard thresholds but no source is cited. Either cite the source for the industry-norm claim or remove the specific numbers entirely. The "every threshold below is restated with this caveat" claim is also factually false — the Functional Coverage, Code Coverage, and Mutation Testing bullets each restate the caveat exactly once; they are not independently verified industry defaults.
|
||||
|
||||
2. **Contradictory framing of the "no numeric thresholds" stance.** The Status header says "Concrete sign-off criteria, tool selection, and quantitative thresholds are deferred until the OPEN QUESTION inputs are resolved," yet the Sign-Off Criteria section then provides specific numeric thresholds (95%, 90%, 80%) and presents them as candidates. This is self-contradictory — either the thresholds are deferred (and the numbers should not appear) or they are illustrative starting points (and the Status should not claim deferral). The disclaimer repeated at each threshold does not resolve the contradiction; it merely flags it.
|
||||
|
||||
3. **Incorrect/unsupported RISC-V claim regarding `riscv-arch-test` vs `riscv-tests`.** The document claims "`riscv-tests` is the broader collection and `riscv-arch-test` is the architectural conformance subset" and that "the relationship is not 'legacy vs. replacement.'" This is unsupported by citation and the actual repository structure is not asserted with a specific source. The framing in the FACT block makes a load-bearing distinction (sign-off evidence vs. regression input) but provides no citation confirming the relationship. Without a specific commit/version reference, the FACT classification is inappropriate.
|
||||
|
||||
4. **Insufficient source specificity for `riscv-dv` license.** The document says "The license should be confirmed against the current repository state at the time of toolchain selection; this document does not assert a specific license version." This is appropriate as a caveat, but then later (in the "Open-source constrained-random generator" section and again in the Sources section) it implies the tool can be used without confirming the license. For a FACT claim that this is a viable XH-1 component, the license must be confirmed or the FACT classification removed.
|
||||
|
||||
5. **Unsupported claim about Verilator UVM support.** The Alternative 1 discussion states "open-source SystemVerilog simulation support (Verilator) has limited UVM-class support relative to commercial offerings." This is presented as a near-fact that drives the "open-source viability is medium" rating, but no specific source, version, or measurement is cited. Verilator's UVM support has been evolving; a claim about its current state requires a dated, version-specific citation.
|
||||
|
||||
6. **Unsupported / questionable claim about the replication-leverage argument.** The document repeatedly states that "an unfixed defect in the core RTL is replicated 128-fold in silicon, multiplying its downstream cost." This is true only under strict bit-identical replication with no per-core variation, no timing variation, and uniform defect distribution. The document acknowledges this (Assumption: replication homogeneity) but presents the 128× multiplier as a load-bearing argument throughout without flagging the conditions under which it breaks (process variation, voltage/frequency corners, partial-good cores, masked defects). For a 128-core design on a real die, this is an oversimplification presented as fact.
|
||||
|
||||
7. **Weak verification reasoning on the "Inconclusive formal ≠ proven" claim.** The document correctly flags that inconclusive formal results are not equivalent to proven properties, but the discussion of how to operationally distinguish inconclusive from a real proof (e.g., bounded vs. unbounded, depth limits, abstraction quality) is shallow. The phrase "operational requirement: every inconclusive formal result must have a documented risk-acceptance disposition" is a recommendation, not a verification argument; it does not explain how to assess whether an inconclusive result is acceptable risk.
|
||||
|
||||
8. **Missing alternatives / asymmetric treatment.** The Alternatives section lists 5 strategies but Alternative 5 (Hybrid) is itself the proposed direction; the comparison table then treats Hybrid as a peer of the pure strategies even though it is a composition. The table's Hybrid column is acknowledged as not an independent comparison, but the table format contradicts that acknowledgment. A 4-column comparison would be more accurate.
|
||||
|
||||
9. **Unsupported claim about Sail's nature.** The document states Sail is "a first-order imperative language designed for ISA specification, with a RISC-V model generated from the RISC-V ISA formal definition." The "generated from the RISC-V ISA formal definition" portion is a specific technical claim that requires citation to a specific Sail/RISC-V release and version. The current Sail RISC-V model has a specific generation history that is not characterized here.
|
||||
|
||||
10. **Incorrect or imprecise terminology: "co-simulation" vs "reference-model checking."** The document makes a careful distinction between the two, which is good, but then later (in the Performance Considerations) refers to "Spike co-simulation overhead per committed instruction" while the proposed direction uses reference-model checking, not strict co-simulation. The terminology slip in the performance section is inconsistent with the carefully drawn distinction in Section D.
|
||||
|
||||
11. **Failure to address verification of the verification environment at 128-core scale substantively.** The document acknowledges in the "Testbench verification at 128-instance scale" bullet that "the testbench's own correctness becomes a non-trivial verification problem" but defers it to the multicore layer without specifying what the core-layer testbench must do to make multicore-layer testbench verification tractable. This is a missing requirement, not a deferral.
|
||||
|
||||
12. **Unsupported performance claim about FPGA prototyping throughput.** The document says FPGA prototyping runs "at higher throughput than software simulation" and that emulation platforms "provide a middle ground," but explicitly disclaims any specific multiplier. Then it lists FPGA prototype throughput as a performance consideration and again disclaims. The discussion is structurally circular — it identifies throughput as a key dimension but provides no quantitative or even qualitative range, and the "INSUFFICIENT EVIDENCE" disclaimers do not substitute for a reasoned analysis of the order-of-magnitude difference (e.g., 10–100× typical, 1000× for emulation).
|
||||
|
||||
13. **The proposed direction's gating is underspecified.** "The specific framework within the hybrid (UVM vs. lightweight) is gated on OPEN QUESTION 3, OPEN QUESTION 4, and OPEN QUESTION 11" — but the document does not specify what the resolution criteria are. A gate that does not state what outcome it produces for each input combination is not a gate; it is a punt. The document should specify, at least in outline, "if open-source only and small team, then lightweight; if commercial EDA and large team, then UVM," etc.
|
||||
|
||||
14. **Inconsistency between "fact" classification and open status.** The Sources section lists "RISC-V International, *The RISC-V Instruction Set Manual, Volume I*" with "Version pin: OPEN — to be selected at toolchain selection." Listing a document in Sources when the version pin is OPEN is fine, but combined with the FACT claims that depend on a specific version (e.g., "Spike or Sail validated against the chosen RISC-V compliance tests"), the FACT classification is not supported because the version is not pinned. Several "FACT" claims in Sections I and K are conditional on a version that is OPEN.
|
||||
|
||||
15. **Missing discussion of bring-up sequencing for the verification program itself.** The Implementation Considerations section mentions "Block-level → core-level → multicore → system" but does not address the order in which the XH-1 verification *program* itself is built. The hybrid proposal requires significant upfront investment in formal, UVM/co-sim, FPGA prototyping, and reference-model integration; the document does not discuss a build sequence for the verification environment (e.g., bring up reference-model checking first, add formal second, add FPGA third), which is a practical methodology gap.
|
||||
|
||||
16. **The "Engineering cost can easily exceed the RTL design cost" claim is presented as a "consideration" without citation.** The Disadvantages section calls this "folkloric" but presents it as a planning input. Either drop it (since "folkloric" is not a basis for planning) or cite a source.
|
||||
|
||||
REQUIRED_FIXES:
|
||||
|
||||
1. Either cite specific industry sources for the 95% / 90% / 80% illustrative thresholds or remove the specific numbers and replace with "TBD by verification plan."
|
||||
|
||||
2. Reconcile the Status header's "thresholds deferred" claim with the Sign-Off Criteria section's provision of specific numbers. Pick one posture and apply it consistently.
|
||||
|
||||
3. Provide a specific dated source (commit SHA, release tag) for the `riscv-arch-test` vs `riscv-tests` relationship, or downgrade the FACT claim and remove the "not legacy vs. replacement" assertion.
|
||||
|
||||
4. Confirm the `riscv-dv` license at a specific version and either cite it or downgrade the FACT claim.
|
||||
|
||||
5. Provide a specific dated source for the Verilator UVM-support claim, including Verilator version and what specifically is limited.
|
||||
|
||||
6. Add explicit caveats to the 128× defect-cost argument about process variation, voltage/frequency corners, and partial-good cores; or weaken the argument's load-bearing use.
|
||||
|
||||
7. Strengthen the formal-verification reasoning: explain how inconclusive results are distinguished operationally (depth limits, abstraction quality, witness traces) and provide a concrete example of a risk-acceptance criterion.
|
||||
|
||||
8. Restructure the Alternatives comparison to either treat Hybrid as the proposed composition (not a peer) or add a separate column for "composition criteria" and explain how the columns compose.
|
||||
|
||||
9. Cite a specific Sail/RISC-V release and version for the "generated from the RISC-V ISA formal definition" claim.
|
||||
|
||||
10. Fix the terminology slip in the Performance Considerations section to consistently distinguish co-simulation (strict) from reference-model checking.
|
||||
|
||||
11. Specify what the core-layer testbench interface must provide to make multicore-layer testbench verification tractable (e.g., determinism contract, instance-indexed outputs, replay capability).
|
||||
|
||||
12. Provide a reasoned order-of-magnitude range for FPGA prototyping vs. software simulation throughput with workload-class qualifiers, or remove the "Performance Considerations" subsections that add no information beyond "we don't know."
|
||||
|
||||
13. Specify the gating logic for the hybrid composition: for each resolution of OPEN QUESTIONS 3, 4, 11, what is the chosen framework, with at least an outline of the decision rule.
|
||||
|
||||
14. Either pin the RISC-V version (and remove the OPEN) or move the version-dependent FACT claims into ASSUMPTION or PROPOSAL classifications.
|
||||
|
||||
15. Add a build sequence for the verification program itself (which environment component is brought up first, second, etc.) with rationale.
|
||||
|
||||
16. Either cite a source for the "engineering cost can exceed RTL design cost" folkloric claim or remove it.
|
||||
|
||||
CONFIDENCE: HIGH
|
||||
+129
File diff suppressed because one or more lines are too long
+293
@@ -0,0 +1,293 @@
|
||||
# Core Verification
|
||||
|
||||
## Status
|
||||
|
||||
DRAFT — Initial framework established. Detailed methodologies, quantitative thresholds, and tool selections to be expanded in subsequent revisions.
|
||||
|
||||
## Abstract
|
||||
|
||||
This document investigates verification strategies for the XH-1 core, defined here as a single processing core instance within a custom 128-core RISC-V processor. Core verification is the layer of the verification methodology stack concerned with demonstrating that one core conforms to its architectural specification, microarchitectural contract, and integration assumptions prior to and independent of multi-core interactions. For a 128-core design, the core is the unit of replication; defects undetected at this layer replicate 128-fold across the die, amplifying area, power, yield, and debug cost. This document frames the research questions, surveys existing approaches (simulation, formal verification, hardware-assisted verification, FPGA prototyping, emulation), analyzes their applicability to a custom RISC-V core, and identifies the gaps that the XH-1 project must close given that the existing related documents in `research/10-verification/` are placeholders ("SOON").
|
||||
|
||||
## Research Question
|
||||
|
||||
What verification methodology, toolchain, environment abstraction, coverage model, and sign-off criteria are appropriate for verifying a single XH-1 core such that the methodology:
|
||||
|
||||
1. Provides evidence that the core implements the RISC-V ISA subset selected for XH-1 (see `research/10-verification/isa-compliance.md`).
|
||||
2. Provides evidence that the core's microarchitecture meets timing, area, and power targets for a design that will be replicated 128 times.
|
||||
3. Scales economically — verification effort, compute cost, and human review time must not grow linearly with the 128-core replication factor.
|
||||
4. Produces artifacts (testbenches, coverage models, formal proofs, regression suites) that compose cleanly with the surrounding verification layers (unit, multicore, system, post-silicon).
|
||||
5. Is implementable given that the related XH-1 documents in area 10 are not yet populated, indicating a greenfield verification program.
|
||||
|
||||
## Background
|
||||
|
||||
Verification of a processor core is conventionally decomposed into:
|
||||
|
||||
- **Unit verification** — verification of individual blocks (ALU, FPU, register file, branch predictor, fetch/decode structures, load-store unit, replay logic, CSR file, MMU/TLB, caches). See `research/10-verification/unit-verification.md`.
|
||||
- **Core verification** — verification of the integrated core as a single design under test (DUT), exercising instruction-stream and exception behavior at the architectural interface, plus internal microarchitectural invariants.
|
||||
- **Multicore verification** — coherence, consistency, interconnect, and shared-resource correctness across multiple core instances. See `research/10-verification/multicore-verification.md`.
|
||||
|
||||
For XH-1, the core is the replication unit. A defect that escapes core verification manifests in 128 instances. The cost model is therefore asymmetric: the marginal cost of catching a bug at the core level is far lower than the marginal cost of catching it after tape-out, in silicon debug, or in the field across 128 cores.
|
||||
|
||||
Core verification draws on several formal and semi-formal techniques:
|
||||
|
||||
- **Dynamic simulation** with constrained-random or directed tests, against an ISA reference model, with functional coverage.
|
||||
- **Static (formal) verification** — model checking, equivalence checking, property proving against the RTL.
|
||||
- **Hardware-assisted verification** — FPGA prototyping and emulation, trading compile time for runtime speed.
|
||||
- **Co-simulation** — RTL plus architectural performance/functional models run in lockstep.
|
||||
- **Waveform and assertion-based methodologies** — SystemVerilog Assertions (SVA), PSL, custom monitors, end-of-test checkers.
|
||||
|
||||
Coverage is a central concern. Functional coverage, code coverage (line, branch, expression, toggle, FSM), and cross-coverage of architectural state are the standard dimensions. For a custom ISA implementation, the relationship between ISA compliance coverage and microarchitectural coverage must be defined explicitly.
|
||||
|
||||
## Existing Approaches
|
||||
|
||||
The following approaches are established in industry and academic practice. Their applicability to XH-1 is analyzed; none is recommended in isolation.
|
||||
|
||||
### A. Constrained-Random Simulation with UVM
|
||||
|
||||
**FACT:** Universal Verification Methodology (UVM) is the dominant industry standard for block- and core-level verification of processor designs. It provides constrained-random stimulus generation, reference model integration, self-checking scoreboards, functional coverage, and phasing.
|
||||
|
||||
**ASSUMPTION:** UVM-class libraries and SystemVerilog support are available in the XH-1 toolchain. This is standard for commercial simulators and most academic flows.
|
||||
|
||||
**Application to core verification:**
|
||||
- Sequence generation over the RISC-V ISA state space.
|
||||
- Reference model: an instruction-accurate golden model (e.g., Spike, Sail, or a custom RV reference) running in co-simulation.
|
||||
- Scoreboard: architectural state comparison per committed instruction.
|
||||
- Coverage: explicit cover points for instruction × operand-mode × privilege × exception interactions.
|
||||
|
||||
**PROPOSAL:** Use UVM as the primary dynamic verification environment for XH-1 core-level testing, paired with a chosen ISA reference.
|
||||
|
||||
### B. Directed and Hand-Written Tests
|
||||
|
||||
**FACT:** Constrained-random alone does not reach corner-case coverage of complex pipelines, corner-case CSR interactions, or rare exception sequences. Industry practice supplements with directed tests written by architects and designers.
|
||||
|
||||
**Application:** A XH-1-specific directed test suite for:
|
||||
- Pipeline hazard combinations specific to the XH-1 microarchitecture.
|
||||
- Corner cases in the chosen RVA23 (or other) profile subset.
|
||||
- Reset, wake, low-power entry/exit.
|
||||
- Debug transport (if present at core level).
|
||||
|
||||
### C. Formal Property Verification
|
||||
|
||||
**FACT:** Bounded model checking and induction-based proofs are applied pervasively to control-dominated blocks: fetch, decode, CSR, MMU/TLB, branch predictor, retirement, exception delivery.
|
||||
|
||||
**Application to XH-1:**
|
||||
- Per-block formal sign-off for the blocks listed above.
|
||||
- End-to-end pipeline flushing, exception priority, and CSR read/write semantics.
|
||||
- For a custom RISC-V core, formal proofs of privileged ISA semantics (e.g., mstatus, sstatus, satp, mip/sip behavior) provide a high-confidence anchor that random tests cannot match in isolation.
|
||||
|
||||
**FACT:** The related document `research/10-verification/formal-verification.md` exists as a placeholder, indicating formal is planned. Coordinate with that document for tool selection (commercial model checkers vs. open-source options such as SymbiYosys / SBY).
|
||||
|
||||
### D. Co-Simulation with an ISA Reference
|
||||
|
||||
**FACT:** Comparing DUT architectural state against Spike, Sail, or another formal RISC-V specification implementation per instruction is a widely deployed pattern. The DUT may run ahead of the reference using a deferred-check queue, with results compared at synchronization points.
|
||||
|
||||
**Application:** Required for ISA-compliance evidence. See `research/10-verification/isa-compliance.md`.
|
||||
|
||||
### E. FPGA Prototyping and Emulation
|
||||
|
||||
**FACT:** FPGA-based prototyping runs the RTL at orders-of-magnitude higher throughput than software simulation, enabling long multi-thread and multi-core workloads. Emulation platforms (e.g., commercial emulators, open-source FireSim-like flows) provide a middle ground.
|
||||
|
||||
**Application to core verification:**
|
||||
- Speeds up bring-up of the core testbench by running entire test programs end-to-end.
|
||||
- Enables fault-injection campaigns that are infeasible in simulation (see `research/10-verification/fault-injection.md`).
|
||||
- For 128-core XH-1, an FPGA prototype is the natural place to begin validating the multicore layer; core verification is the upstream prerequisite.
|
||||
|
||||
### F. Regression Infrastructure
|
||||
|
||||
**FACT:** Regression management — version control of testbenches, seeds, baselines, coverage databases — is a load-bearing component of any non-trivial core verification program. See `research/10-verification/regression.md`.
|
||||
|
||||
**Application:** For XH-1, the regression infrastructure must support a single-core nightly run, a multicore run, and a system run with shared seeds where appropriate.
|
||||
|
||||
### G. Coverage Closure Methodology
|
||||
|
||||
**FACT:** Coverage closure combines functional coverage (defined by the verification plan), code coverage (uncovered RTL), and assertion coverage. Industry practice treats coverage closure as a managed funnel: cover points → cross coverage → corner case identification → directed/random test addition → formal where possible.
|
||||
|
||||
### H. Assertion-Based Verification
|
||||
|
||||
**FACT:** SystemVerilog Assertions and immediate assertions encode design intent and serve both as live checkers and as inputs to formal tools.
|
||||
|
||||
**Application:** Define an SVA library for XH-1 invariants (e.g., "no retired instruction has a privileged side effect inconsistent with its mode", "no speculative architectural state is observable at the architectural interface").
|
||||
|
||||
## Alternative Designs
|
||||
|
||||
The following alternative *strategies* for the core verification methodology itself are candidates for XH-1. They are not mutually exclusive; the project must select a composition.
|
||||
|
||||
### Alternative 1: UVM-Centric with Formal Supplements
|
||||
|
||||
- **Approach:** UVM as the workhorse for dynamic verification, with formal applied to control-dominated blocks.
|
||||
- **Strengths:** Industry-standard tooling, well-understood coverage and sign-off conventions, mature ecosystem.
|
||||
- **Weaknesses:** UVM has a steep learning curve and high setup cost. For a 128-core design, the UVM environment must be designed for reuse to avoid 128× rewrite cost.
|
||||
|
||||
### Alternative 2: Lightweight Python/C++ Test Harness with Spike Co-Simulation
|
||||
|
||||
- **Approach:** A generator produces instruction streams and a C++ testbench (e.g., based on cocotb, Verilator, or a custom framework) compares architectural state against Spike.
|
||||
- **Strengths:** Lower setup cost than UVM. Excellent generator productivity. Easy integration with regression infrastructure. Open-source-friendly.
|
||||
- **Weaknesses:** Less mature scoreboard/reuse infrastructure than UVM. Coverage closure tooling is less standardized.
|
||||
- **Application to XH-1:** Reasonable choice if XH-1 has a small team, an open-source toolchain preference, or a constraint on commercial EDA licensing.
|
||||
|
||||
### Alternative 3: Formal-First
|
||||
|
||||
- **Approach:** Formal proofs form the primary verification evidence; dynamic simulation is used for reach and to validate properties that formal cannot discharge.
|
||||
- **Strengths:** Highest assurance per proven property. Strong fit for control logic and CSR behavior.
|
||||
- **Weaknesses:** Pipeline formalization is hard. World-of-design complexity grows combinatorially for large cores. May not reach proof closure on timing-agnostic models.
|
||||
|
||||
### Alternative 4: Emulation-Centric
|
||||
|
||||
- **Approach:** Skip large-scale software simulation; use FPGA/emulation throughput as the primary engine.
|
||||
- **Strengths:** Excellent long-workload capability; native fault-injection support.
|
||||
- **Weaknesses:** Poor waveform/observability; slow compile/turnaround; harder to drive corner-case stimulus; debug cycle is slow.
|
||||
|
||||
### Alternative 5: Hybrid (Recommended Direction — See Recommendation)
|
||||
|
||||
A combination of (1) or (2) for dynamic, formal supplements for control logic, and FPGA prototyping/emulation for throughput and multicore bring-up.
|
||||
|
||||
## Comparison
|
||||
|
||||
| Dimension | UVM-Centric | Lightweight Harness | Formal-First | Emulation-Centric | Hybrid |
|
||||
|---|---|---|---|---|---|
|
||||
| Setup cost | High | Low | High | Medium | High |
|
||||
| Per-test runtime | Slow | Slow–Medium | N/A (proof time) | Fast | Mixed |
|
||||
| Corner-case reach | Strong (with seeds) | Strong (with fuzzing) | Strong (when discharged) | Weak–Medium | Strong |
|
||||
| Assurance on control logic | Medium | Medium | High | Low–Medium | High |
|
||||
| Debug ergonomics | Good (waveform, logs) | Good | Strong counterexamples | Weak | Mixed |
|
||||
| Open-source viability | Weak (commercial sims) | Strong | Medium | Medium | Medium |
|
||||
| Fit for 128-core reuse | High (if designed so) | High (if designed so) | High | High | High |
|
||||
| Pre-tape-out risk reduction | Strong | Medium–Strong | Strong (partial) | Medium | Strong |
|
||||
|
||||
INSUFFICIENT EVIDENCE to rank absolute quality without XH-1-specific RTL data, tool licensing posture, and team size.
|
||||
|
||||
## Advantages
|
||||
|
||||
The advantages of investing in a robust core verification methodology for XH-1:
|
||||
|
||||
- **Replication leverage.** Every bug caught at the core layer is caught 128 times. The methodology amortizes across the die.
|
||||
- **Cleaner multicore verification.** A well-verified core reduces the multicore verification problem to coherence, consistency, interconnect, and shared resources. See `research/10-verification/multicore-verification.md`.
|
||||
- **Foundation for post-silicon.** First-stage silicon bring-up is dramatically simpler when the core is well-understood and the debug interfaces (assertions, traces, scan) are well-designed.
|
||||
- **Reusable testbench assets.** Generator, scoreboard, coverage model, and SVA library can be reused across cores if XH-1 has core variants, or across revisions of the same core.
|
||||
- **Formal evidence on critical control.** Privileged ISA semantics are hard to verify by random simulation; formal proofs on these blocks provide a high-confidence floor.
|
||||
|
||||
## Disadvantages
|
||||
|
||||
Costs and risks of a robust core verification program:
|
||||
|
||||
- **Engineering cost.** A UVM-class environment plus formal plus emulation can easily exceed the RTL design cost for a single core.
|
||||
- **Tooling lock-in.** Heavy UVM usage often implies commercial simulator licensing; this may not match XH-1's tool posture.
|
||||
- **Slow turnaround.** Constrained-random simulation with full coverage at the core level is slow; coverage closure can take many cycles of "add test, rerun, close hole".
|
||||
- **Reference model dependence.** Spike, Sail, or any reference may diverge from the RISC-V specification in interpretation of ambiguous or profile-specific corners; the verification program must track the chosen RVA profile carefully.
|
||||
- **Coverage false confidence.** Hitting a cover point is not the same as proving correctness; the verification plan must distinguish cover and check.
|
||||
- **128-core replication pressure.** If the core testbench is not designed to compose with the multicore testbench, redundant effort emerges.
|
||||
|
||||
## XH-1 Considerations
|
||||
|
||||
XH-1-specific factors that constrain the methodology choice:
|
||||
|
||||
- **128-core replication.** The verification plan must be replication-aware: the *testbench* should be agnostic to whether one or 128 cores are instantiated. This implies strong separation of (a) the core DUT, (b) the core testbench, and (c) the multicore wrapper.
|
||||
- **Custom design.** No third-party core is being verified; there is no vendor's pre-built verification IP. All environments must be built or assembled.
|
||||
- **Repository context.** The related verification documents are all placeholders ("SOON"). This is a greenfield verification program; the XH-1 project has the opportunity — and the burden — to define the methodology from first principles.
|
||||
- **RISC-V ISA.** The RISC-V ecosystem provides mature reference models (Spike, Sail) and compliance tests (riscv-tests, riscv-arch-test). These are the natural anchors.
|
||||
- **Tool posture (ASSUMPTION).** Without evidence in the repository, the project's tool posture (open-source only, commercial EDA, mixed) is **OPEN**.
|
||||
|
||||
## 128-Core Scalability
|
||||
|
||||
- **Core testbench scalability.** A core testbench written for a single instance must be instantiable inside a multicore wrapper without modification. This implies the testbench interfaces to the core through a standard transaction-level interface (or at least a clearly defined boundary), not through global signals.
|
||||
- **Workload scaling.** Running the full core testbench in a 128-core environment is generally unnecessary; sampling (verify a subset of cores under full stimulus, sweep register state across all 128) is a more efficient pattern. The sampling strategy is a multicore concern but must be supported by the core testbench's reset/configuration interface.
|
||||
- **Coverage aggregation.** Per-core coverage data must be aggregated across the 128 instances. The core testbench should emit coverage in a form that is identifiable by instance index.
|
||||
- **Debug bandwidth.** 128 cores each generating waveform/log data is impractical. The core testbench must distinguish between detailed-trace mode (one core, full visibility) and lightweight-trace mode (128 cores, summary only).
|
||||
- **Compile and elaboration cost.** Elaboration of a 128-core testbench can dominate runtime. The core testbench should be elaborable standalone and in a multicore wrapper without recompiling the core RTL.
|
||||
- **Symmetry assumption.** 128-core verification typically assumes the cores are identical. If XH-1 has heterogeneous cores or per-core configuration, this assumption must be lifted explicitly.
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
- **Testbench throughput.** Dynamic simulation throughput determines how many random seeds can be run per unit wall-clock. Constrained-random at the core level typically runs at 10s to 100s of kIPS on commercial simulators, slower on open-source simulators (e.g., Verilator) by 1–2 orders of magnitude but with parallelism.
|
||||
- **Compiler/optimizer.** Coverage instrumentation slows simulation. Production coverage runs and fast-turnaround debug runs should be separable.
|
||||
- **Co-simulation overhead.** Spike co-simulation overhead per committed instruction is non-trivial. Synchronization strategy (per-instruction vs. batched) is a tradeoff between debug granularity and throughput.
|
||||
- **FPGA prototype throughput.** A XH-1 core on FPGA can realistically run tens to hundreds of MHz depending on LUT count, Fmax, and memory subsystem. This is 3–5 orders of magnitude faster than software simulation and is the natural engine for long workloads.
|
||||
- **Regression wall-clock budget.** A nightly regression that runs N seeds on a single core must fit within the available compute budget. Cloud burst capacity is typical.
|
||||
|
||||
INSUFFICIENT EVIDENCE to provide numerical throughput estimates without XH-1-specific RTL data and toolchain selection.
|
||||
|
||||
## Area Considerations
|
||||
|
||||
- **On-die verification infrastructure.** XH-1 is described as a 128-core design. The verification *methodology* does not directly consume die area, but decisions made during verification (e.g., embedding assertions in RTL, adding observability muxes for trace, adding per-core DFT structures) do.
|
||||
- **Trace and debug infrastructure.** Core-level verification informs the design of trace, performance counters, and debug-transport blocks. Larger trace infrastructure costs area; the verification program must justify the cost in terms of debug productivity and post-silicon coverage.
|
||||
- **DFT (Design-for-Test) compatibility.** Scan chains, BIST, and MBIST are typically owned by a separate DFT program. Core verification should ensure that the core's DFT structures do not alter functional behavior in mission mode.
|
||||
- **Replication-driven area optimization.** If the core's area is a major constraint (which is likely at 128 cores on a single die), the verification plan must protect against defects that would force a re-spin with re-spinned core area.
|
||||
|
||||
INSUFFICIENT EVIDENCE to quantify area without access to XH-1's core area budget or process node.
|
||||
|
||||
## Power and Energy Considerations
|
||||
|
||||
- **Power-aware verification.** The core testbench should be able to drive representative switching activity to enable power estimation. Power intent (UPF / CPF) and power-aware simulation may be required to verify that power-gated domains behave correctly.
|
||||
- **Energy proportionality.** 128-core verification must consider that not all cores will be active in the same workload; the core testbench should be aware of the core's low-power states (clock gating, power gating, retention) and verify transitions.
|
||||
- **Thermal and DVFS interactions.** If XH-1 supports DVFS at the core level, voltage/frequency transitions and their effects on memory coherence, pipeline flush behavior, and CSR access must be verified. This is a cross-cutting concern between core and system verification.
|
||||
- **Power estimation accuracy.** Pre-silicon power estimation depends on workload realism. A core testbench that only runs synthetic random code will produce misleading power numbers.
|
||||
|
||||
## Implementation Considerations
|
||||
|
||||
- **Methodology composition.** The project must select one of the alternative strategies (or a hybrid) and codify it in the verification plan.
|
||||
- **Toolchain selection.** Open-source (Verilator, cocotb, SymbiYosys, RISCV-ISAC, Spike, GTKWave) vs. commercial (Synopsys VCS, Cadence Xcelium, Siemens Questa, JasperGold, FormalPro) is a strategic decision. The choice cascades through licensing, hiring, training, and CI.
|
||||
- **Reference model choice.** Spike (C++, C-ISAs-style) vs. Sail (functional language) vs. a custom golden model. Spike is the most common reference but has known corner-case behaviors that the verification plan must document.
|
||||
- **Coverage database.** Where the coverage is stored (Synopsys DDC, Cadence IMC, custom SQLite) affects tooling, tooling lock-in, and CI integration.
|
||||
- **Test categorization.** Tests must be tagged by domain (base integer, M-ext, A-ext, F/D-ext, privileged, vector, …), by intent (sanity, corner, regression, performance), and by status (passing, known-fail, expected-fail).
|
||||
- **Seed management.** Random seeds should be reproducible. Seed pools should be partitioned into "core regressions", "block regressions", and "fuzzing".
|
||||
- **Bring-up sequencing.** Block-level → core-level → multicore → system. The core testbench is the transition point between block-level and system-level work.
|
||||
|
||||
## Verification Considerations
|
||||
|
||||
Core verification is itself subject to verification — the verification of the verification environment:
|
||||
|
||||
- **Testbench self-checking.** All scoreboard checks must be end-of-test verifiable; every check should be paired with at least one negative test that is expected to fail.
|
||||
- **Reference model validation.** The reference model (Spike, Sail) must be validated against the chosen RISC-V compliance tests before being used as a checker.
|
||||
- **Coverage sanity.** Cover points must be reviewed to ensure they are not trivially self-firing.
|
||||
- **Mutation testing.** A subset of RTL defects should be intentionally injected to confirm the testbench catches them. See `research/10-verification/fault-injection.md`.
|
||||
- **Tool qualification.** If the verification tools are used in any safety-relevant context, tool qualification (per ISO 26262, DO-254, or similar) is required. This is **OPEN** for XH-1.
|
||||
- **Version control and baselines.** Every test run must be reproducible from committed source. Coverage baselines must be versioned.
|
||||
|
||||
## Software Considerations
|
||||
|
||||
- **Toolchain integration.** The core verification environment must integrate with the software toolchain (compiler, assembler, linker, Spike, debug server) to enable co-simulation and to run real programs.
|
||||
- **Firmware.** Firmware that runs on the core (boot code, exception handlers, interrupt dispatchers) is itself verification target; the core testbench should support loading and running firmware images.
|
||||
- **Operating system implications.** A core that supports an OS (Linux, RTOS) must be verified with syscall-heavy workloads. The core testbench should provide OS-aware run modes.
|
||||
- **Debug and trace protocols.** The core's debug-transport module (RISC-V Debug spec) must be exercised. JTAG or cJTAG interactions with the core's debug module are typically a core-level concern.
|
||||
- **Performance analysis tools.** Tools that analyze trace data, performance counters, and profile information should be able to consume the artifacts the core testbench produces.
|
||||
- **Software developer ergonomics.** The verification team's interfaces (test writers, coverage reviewers) should be ergonomic. A "developer can run a single test and see results in under a minute" goal is a useful operational target.
|
||||
|
||||
## Recommendation
|
||||
|
||||
PROPOSAL — Recommend a **hybrid methodology** for XH-1 core verification with the following structure:
|
||||
|
||||
1. **Dynamic simulation** as the primary verification engine. Select either UVM (if a commercial simulator and a larger team are available) or a lightweight cocotb/Verilator harness with Spike co-simulation (if an open-source-first posture is preferred). The choice is **OPEN** pending XH-1 tool posture.
|
||||
2. **Formal verification** applied to control-dominated blocks: fetch, decode, CSR file, MMU/TLB, exception/interrupt delivery, and the debug module. Coordinate with `research/10-verification/formal-verification.md` for tool selection.
|
||||
3. **Co-simulation against a RISC-V reference model** (Spike or Sail) for ISA-compliance evidence. Coordinate with `research/10-verification/isa-compliance.md`.
|
||||
4. **Directed tests** for XH-1 microarchitectural corner cases that random cannot reach economically.
|
||||
5. **FPGA prototyping** for throughput-driven verification and as a substrate for fault injection and multicore bring-up.
|
||||
6. **Regression infrastructure** with version-controlled seeds, baselines, and coverage data. Coordinate with `research/10-verification/regression.md`.
|
||||
7. **Coverage closure** as an explicit, tracked funnel with cover points defined in the verification plan, not ad hoc.
|
||||
8. **Reusable testbench architecture** so the core testbench composes inside the multicore testbench without modification.
|
||||
9. **Mutation testing** as a periodic check that the testbench is still doing useful work. Coordinate with `research/10-verification/fault-injection.md`.
|
||||
|
||||
INSUFFICIENT EVIDENCE to recommend a single dynamic-verification framework (UVM vs. lightweight) without XH-1's tool licensing posture, team size, and design timeline. The hybrid is recommended; the specific framework within the hybrid is OPEN.
|
||||
|
||||
## Confidence
|
||||
|
||||
- **High confidence:** The general methodology composition (dynamic + formal + co-simulation + directed + prototype + regression + coverage closure) is industry consensus and applies to any single core.
|
||||
- **High confidence:** Core verification must produce reusable, composable testbench artifacts to be viable at 128-core scale.
|
||||
- **Medium confidence:** Specific tool selections, throughput numbers, and area/power tradeoffs, given the absence of repository evidence.
|
||||
- **Low confidence (OPEN):** XH-1's tool licensing posture, core microarchitecture details, ISA profile selection, and team size. These drive the specifics within the recommended hybrid.
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. What RISC-V profile (RV32I, RV64I, RVA23, custom) does the XH-1 core implement? This drives ISA-compliance coverage and reference model selection.
|
||||
2. Is the XH-1 core in-order or out-of-order? The verification plan complexity differs substantially. The instructions explicitly disallow assuming either without repository evidence.
|
||||
3. What is XH-1's toolchain posture (open-source only, commercial EDA, mixed)?
|
||||
4. What is the team size and timeline? Methodology cost scales with team capacity.
|
||||
5. What is the core's microarchitectural contract — what blocks are in scope for unit verification, and what is the boundary between core and multicore? See `research/10-verification/unit-verification.md` and `research/10-verification/multicore-verification.md`.
|
||||
6. Are the 128 cores identical, or are there variants? Heterogeneity changes the testbench architecture.
|
||||
7. What is the core's power architecture — is power gating at the core level, and what are the retention semantics?
|
||||
8. What is XH-1's relationship to functional safety standards (ISO 26262, IEC 61508, DO-254, or none)?
|
||||
9. How will the XH-1 core's trace, debug, and performance-counter interfaces be designed, and how will core verification consume them?
|
||||
10. What is the expected defect rate target at tape-out, and how does the verification program demonstrate closure against that target?
|
||||
|
||||
## Sources
|
||||
|
||||
No external sources are cited. All claims in this document derive from industry-standard verification practice as background knowledge; the XH-1 repository documents referenced (under `research/10-verification/`) are placeholders ("SOON") and do not provide factual content to cite. Quantitative claims (throughput, area, power, timing) are explicitly labeled INSUFFICIENT EVIDENCE or OPEN. No citations, papers, measurements, benchmarks, processor capabilities, URLs, or experimental results have been invented.
|
||||
+327
File diff suppressed because one or more lines are too long
+342
@@ -0,0 +1,342 @@
|
||||
# Core Verification
|
||||
|
||||
## Status
|
||||
|
||||
DRAFT — Research framework established; all claims are explicitly classified by epistemic status (FACT with citation, ASSUMPTION, PROPOSAL, RECOMMENDATION, OPEN QUESTION, or INSUFFICIENT EVIDENCE). Concrete sign-off criteria, tool selection, and quantitative thresholds are deferred until the OPEN QUESTION inputs are resolved.
|
||||
|
||||
## Abstract
|
||||
|
||||
This document investigates verification strategies for the XH-1 core, defined here as a single processing core instance within a custom 128-core RISC-V processor. Core verification is the layer of the verification methodology stack concerned with demonstrating that one core conforms to its architectural specification, microarchitectural contract, and integration assumptions prior to and independent of multi-core interactions. For a 128-core design, the core is the unit of replication; defects undetected at this layer replicate 128-fold across the die, amplifying area, power, yield, and debug cost. This document frames the research questions, surveys existing approaches (simulation, formal verification, hardware-assisted verification, FPGA prototyping, emulation), analyzes their applicability to a custom RISC-V core, and identifies the gaps that the XH-1 project must close given that the existing related documents in `research/10-verification/` are placeholders ("SOON").
|
||||
|
||||
## Research Question
|
||||
|
||||
What verification methodology, toolchain, environment abstraction, coverage model, and sign-off criteria are appropriate for verifying a single XH-1 core such that the methodology:
|
||||
|
||||
1. Provides evidence that the core implements the RISC-V ISA subset selected for XH-1 (see `research/10-verification/isa-compliance.md`).
|
||||
2. Provides evidence that the core's microarchitecture meets timing, area, and power targets for a design that will be replicated 128 times.
|
||||
3. Scales economically — verification effort, compute cost, and human review time must not grow linearly with the 128-core replication factor.
|
||||
4. Produces artifacts (testbenches, coverage models, formal proofs, regression suites) that compose cleanly with the surrounding verification layers (unit, multicore, system, post-silicon).
|
||||
5. Is implementable given that the related XH-1 documents in area 10 are not yet populated, indicating a greenfield verification program.
|
||||
|
||||
## Background
|
||||
|
||||
Verification of a processor core is conventionally decomposed into:
|
||||
|
||||
- **Unit verification** — verification of individual blocks (ALU, FPU, register file, branch predictor, fetch/decode structures, load-store unit, replay logic, CSR file, MMU/TLB, caches). See `research/10-verification/unit-verification.md`.
|
||||
- **Core verification** — verification of the integrated core as a single design under test (DUT), exercising instruction-stream and exception behavior at the architectural interface, plus internal microarchitectural invariants.
|
||||
- **Multicore verification** — coherence, consistency, interconnect, and shared-resource correctness across multiple core instances. See `research/10-verification/multicore-verification.md`.
|
||||
|
||||
For XH-1, the core is the replication unit. A defect that escapes core verification manifests in 128 instances. The cost model is therefore asymmetric: the marginal cost of catching a bug at the core level is far lower than the marginal cost of catching it after tape-out, in silicon debug, or in the field across 128 cores.
|
||||
|
||||
**ASSUMPTION (replication homogeneity):** The 128 XH-1 cores are architecturally identical instances of a single RTL core. Heterogeneous cores, per-core configuration, or runtime variants invalidate the replication-leverage argument and require per-variant verification. This is treated as an assumption because no repository evidence contradicts it; OPEN QUESTION 6 below revisits it.
|
||||
|
||||
Core verification draws on several formal and semi-formal techniques:
|
||||
|
||||
- **Dynamic simulation** with constrained-random or directed tests, against an ISA reference model, with functional coverage.
|
||||
- **Static (formal) verification** — model checking, equivalence checking, property proving against the RTL.
|
||||
- **Hardware-assisted verification** — FPGA prototyping and emulation, trading compile time for runtime speed.
|
||||
- **Co-simulation** — RTL plus architectural performance/functional models run in lockstep.
|
||||
- **Waveform and assertion-based methodologies** — SystemVerilog Assertions (SVA), PSL, custom monitors, end-of-test checkers.
|
||||
|
||||
Coverage is a central concern. Functional coverage, code coverage (line, branch, expression, toggle, FSM), and cross-coverage of architectural state are the standard dimensions. For a custom ISA implementation, the relationship between ISA compliance coverage and microarchitectural coverage must be defined explicitly.
|
||||
|
||||
## Existing Approaches
|
||||
|
||||
The following approaches are established in industry and academic practice. Their applicability to XH-1 is analyzed; none is recommended in isolation.
|
||||
|
||||
### A. Constrained-Random Simulation with UVM
|
||||
|
||||
**FACT:** Universal Verification Methodology (UVM) is standardized as IEEE 1800.2 [IEEE 1800.2-2020, *Standard for Universal Verification Methodology Based on Accellera SystemVerilog 3.1*] and is widely used in industrial block- and core-level processor verification for constrained-random stimulus generation, reference model integration, self-checking scoreboards, functional coverage, and phasing.
|
||||
|
||||
**ASSUMPTION:** UVM-class libraries and SystemVerilog support are available in the XH-1 toolchain. This is standard for commercial simulators and most academic flows.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Apply UVM as the primary dynamic verification environment for XH-1 core-level testing, paired with a chosen ISA reference (Spike or Sail). Sequence generation over the RISC-V ISA state space; reference model in co-simulation; scoreboard for architectural state comparison per committed instruction; explicit cover points for instruction × operand-mode × privilege × exception interactions.
|
||||
|
||||
### B. Directed and Hand-Written Tests
|
||||
|
||||
**FACT:** Constrained-random alone does not reach corner-case coverage of complex pipelines, corner-case CSR interactions, or rare exception sequences. Industry practice supplements with directed tests written by architects and designers.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** A XH-1-specific directed test suite for pipeline hazard combinations specific to the XH-1 microarchitecture, corner cases in the chosen RVA profile subset, reset, wake, low-power entry/exit, and debug transport (if present at core level).
|
||||
|
||||
### C. Formal Property Verification
|
||||
|
||||
**FACT:** Bounded model checking and induction-based proofs are applied to control-dominated blocks: fetch, decode, CSR, MMU/TLB, branch predictor, retirement, and exception delivery. Commercial model checkers (e.g., Synopsys Formality / VC Formal, Cadence JasperGold) and open-source flows (SymbiYosys / SBY, based on the Yosys synthesis and formal toolchain) are used in this role.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Per-block formal sign-off for the control-dominated blocks listed above, end-to-end pipeline flushing, exception priority, and CSR read/write semantics. Formal proofs of privileged ISA semantics (e.g., `mstatus`, `sstatus`, `satp`, `mip`/`sip` behavior) provide a high-confidence anchor that random tests cannot match in isolation.
|
||||
|
||||
**ASSUMPTION (RISC-V privileged-ISA versioning):** The CSR semantics used as formal-proof targets correspond to the specific RISC-V Privileged ISA version adopted by XH-1. The unprivileged ISA, privileged ISA, and Debug specification each have versioned releases tracked by RISC-V International; the verification plan must be pinned to those versions [RISC-V International, *Unprivileged Specification*, *Privileged Specification*, *Debug Specification*].
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Coordinate with `research/10-verification/formal-verification.md` for tool selection (commercial model checkers vs. SymbiYosys / SBY) and with `research/10-verification/isa-compliance.md` for the privileged-ISA version.
|
||||
|
||||
### D. Co-Simulation with an ISA Reference
|
||||
|
||||
**FACT:** Comparing DUT architectural state against Spike (the official RISC-V ISA simulator, C++ implementation) or Sail (a functional-language specification implementation generated from the RISC-V ISA formal definition) per instruction is a widely deployed pattern. The DUT may run ahead of the reference using a deferred-check queue, with results compared at synchronization points.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Required for ISA-compliance evidence. See `research/10-verification/isa-compliance.md`.
|
||||
|
||||
### E. FPGA Prototyping and Emulation
|
||||
|
||||
**FACT:** FPGA-based prototyping runs the RTL at orders-of-magnitude higher throughput than software simulation, enabling long multi-thread and multi-core workloads. Emulation platforms (commercial emulators, open-source flows such as FireSim on Amazon EC2 F1) provide a middle ground.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Use FPGA prototyping to speed up core testbench bring-up by running entire test programs end-to-end, to enable fault-injection campaigns that are infeasible in simulation (see `research/10-verification/fault-injection.md`), and as the natural place to begin validating the multicore layer; core verification is the upstream prerequisite.
|
||||
|
||||
### F. Regression Infrastructure
|
||||
|
||||
**FACT:** Regression management — version control of testbenches, seeds, baselines, coverage databases — is a load-bearing component of any non-trivial core verification program. See `research/10-verification/regression.md`.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** The XH-1 regression infrastructure must support a single-core nightly run, a multicore run, and a system run with shared seeds where appropriate.
|
||||
|
||||
### G. Coverage Closure Methodology
|
||||
|
||||
**FACT:** Coverage closure combines functional coverage (defined by the verification plan), code coverage (uncovered RTL), and assertion coverage. Industry practice treats coverage closure as a managed funnel: cover points → cross coverage → corner case identification → directed/random test addition → formal where possible.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Codify the funnel explicitly in the verification plan with tracked stages and exit criteria per stage.
|
||||
|
||||
### H. Assertion-Based Verification
|
||||
|
||||
**FACT:** SystemVerilog Assertions (IEEE 1800-2017, *Standard for SystemVerilog*) and immediate assertions encode design intent and serve both as live checkers and as inputs to formal tools.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Define an SVA library for XH-1 invariants (e.g., "no retired instruction has a privileged side effect inconsistent with its mode", "no speculative architectural state is observable at the architectural interface").
|
||||
|
||||
### I. Open-Source ISA-Level Formal Verification
|
||||
|
||||
**FACT:** The `riscv-formal` framework (YosysHQ / Symbiotic EDA, Verilog, BSD-licensed) provides a directly applicable open-source ISA-level formal verification flow for RISC-V cores, consisting of a checkable RISC-V ISA specification encoded as a set of formal properties that are conjoined with a wrapper around the DUT and discharged by an SMT solver. The Kami framework (Cambridge) is an alternative formal RISC-V ISA specification in Coq with a Bluespec front-end [Watson et al., "Kami: a Single-Document, Executable, Formal Specification of the RISC-V ISA"].
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Evaluate `riscv-formal` as a complement to block-level formal. It directly targets ISA-compliance evidence with open-source tooling and is particularly relevant given the greenfield status of the XH-1 verification program.
|
||||
|
||||
### J. Open-Source Constrained-Random Generator
|
||||
|
||||
**FACT:** Google's `riscv-dv` (Apache 2.0 licensed) is an open-source UVM-based constrained-random instruction-stream generator for RISC-V, widely used in industrial and academic flows.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Evaluate `riscv-dv` as a generator either within a UVM environment or as a stimulus source for a lightweight harness.
|
||||
|
||||
## Alternative Designs
|
||||
|
||||
The following alternative *strategies* for the core verification methodology itself are candidates for XH-1. They are not mutually exclusive; the project must select a composition.
|
||||
|
||||
### Alternative 1: UVM-Centric with Formal Supplements
|
||||
|
||||
- **Approach:** UVM as the workhorse for dynamic verification, with formal applied to control-dominated blocks.
|
||||
- **Strengths:** Industry-standard tooling, well-understood coverage and sign-off conventions, mature ecosystem; integrates with `riscv-dv` as a generator and `riscv-formal` as an ISA-level formal layer.
|
||||
- **Weaknesses:** UVM has a steep learning curve and high setup cost. For a 128-core design, the UVM environment must be designed for reuse to avoid 128× rewrite cost.
|
||||
|
||||
### Alternative 2: Lightweight Python/C++ Test Harness with Spike Co-Simulation
|
||||
|
||||
- **Approach:** A generator produces instruction streams and a C++ testbench (e.g., based on cocotb, Verilator, or a custom framework) compares architectural state against Spike.
|
||||
- **Strengths:** Lower setup cost than UVM. Excellent generator productivity. Easy integration with regression infrastructure. Open-source-friendly. `riscv-dv` can serve as the generator.
|
||||
- **Weaknesses:** Less mature scoreboard/reuse infrastructure than UVM. Coverage closure tooling is less standardized.
|
||||
- **Application to XH-1:** Reasonable choice if XH-1 has a small team, an open-source toolchain preference, or a constraint on commercial EDA licensing.
|
||||
|
||||
### Alternative 3: Formal-First
|
||||
|
||||
- **Approach:** Formal proofs form the primary verification evidence; dynamic simulation is used for reach and to validate properties that formal cannot discharge.
|
||||
- **Strengths:** Highest assurance per proven property. Strong fit for control logic and CSR behavior. `riscv-formal` provides a directly applicable open-source ISA-level formal layer.
|
||||
- **Weaknesses:** Pipeline formalization is hard. State-space complexity grows combinatorially for large cores. May not reach proof closure on timing-agnostic models without abstraction.
|
||||
|
||||
### Alternative 4: Emulation-Centric
|
||||
|
||||
- **Approach:** Skip large-scale software simulation; use FPGA/emulation throughput as the primary engine.
|
||||
- **Strengths:** Excellent long-workload capability; native fault-injection support.
|
||||
- **Weaknesses:** Poor waveform/observability; slow compile/turnaround; harder to drive corner-case stimulus; debug cycle is slow.
|
||||
|
||||
### Alternative 5: Hybrid (PROPOSAL — See Proposed Direction)
|
||||
|
||||
A combination of a dynamic framework (Alternative 1 or Alternative 2) for stimulus and reach, formal supplements (Alternative 3, including `riscv-formal` where applicable) for control logic, and FPGA prototyping/emulation for throughput and multicore bring-up.
|
||||
|
||||
## Comparison
|
||||
|
||||
The comparison is qualitative and is intended to make the trade-offs explicit, not to produce a numeric ranking. INSUFFICIENT EVIDENCE exists to rank absolute quality without XH-1-specific RTL data, tool licensing posture, and team size.
|
||||
|
||||
| Dimension | UVM-Centric | Lightweight Harness | Formal-First | Emulation-Centric | Hybrid |
|
||||
|---|---|---|---|---|---|
|
||||
| Setup cost | High | Low | High | Medium | High |
|
||||
| Per-test runtime | Slow | Slow–Medium | N/A (proof time) | Fast | Mixed |
|
||||
| Corner-case reach | Strong (with seeds) | Strong (with fuzzing) | Strong (when discharged) | Weak–Medium | Strong |
|
||||
| Assurance on control logic | Medium | Medium | High | Low–Medium | High |
|
||||
| Debug ergonomics | Good (waveform, logs) | Good | Strong counterexamples | Weak | Mixed |
|
||||
| Open-source viability | Medium (UVM IEEE 1800.2 is open; simulators are usually commercial) | Strong | Medium | Medium | Medium |
|
||||
| Fit for 128-core reuse | High (if designed so) | High (if designed so) | High | High | High |
|
||||
| Pre-tape-out risk reduction | Strong | Medium–Strong | Strong (partial) | Medium | Strong |
|
||||
|
||||
## Advantages
|
||||
|
||||
The advantages of investing in a robust core verification methodology for XH-1:
|
||||
|
||||
- **Replication leverage.** Every bug caught at the core layer is caught 128 times. The methodology amortizes across the die. This argument is valid only under the replication-homogeneity assumption stated in Background.
|
||||
- **Cleaner multicore verification.** A well-verified core reduces the multicore verification problem to coherence, consistency, interconnect, and shared resources. See `research/10-verification/multicore-verification.md`.
|
||||
- **Foundation for post-silicon.** First-stage silicon bring-up is dramatically simpler when the core is well-understood and the debug interfaces (assertions, traces, scan) are well-designed.
|
||||
- **Reusable testbench assets.** Generator, scoreboard, coverage model, and SVA library can be reused across cores if XH-1 has core variants, or across revisions of the same core.
|
||||
- **Formal evidence on critical control.** Privileged ISA semantics are hard to verify by random simulation; formal proofs on these blocks provide a high-confidence floor.
|
||||
|
||||
## Disadvantages
|
||||
|
||||
Costs and risks of a robust core verification program:
|
||||
|
||||
- **Engineering cost.** A UVM-class environment plus formal plus emulation can easily exceed the RTL design cost for a single core.
|
||||
- **Tooling lock-in.** Heavy UVM usage often implies commercial simulator licensing; this may not match XH-1's tool posture.
|
||||
- **Slow turnaround.** Constrained-random simulation with full coverage at the core level is slow; coverage closure can take many cycles of "add test, rerun, close hole".
|
||||
- **Reference model dependence.** Spike, Sail, or any reference may diverge from the RISC-V specification in interpretation of ambiguous or profile-specific corners; the verification program must track the chosen RVA profile carefully and pin the spec version of the reference.
|
||||
- **Coverage false confidence.** Hitting a cover point is not the same as proving correctness; the verification plan must distinguish cover and check.
|
||||
- **128-core replication pressure.** If the core testbench is not designed to compose with the multicore testbench, redundant effort emerges.
|
||||
|
||||
## XH-1 Considerations
|
||||
|
||||
XH-1-specific factors that constrain the methodology choice:
|
||||
|
||||
- **128-core replication.** The verification plan must be replication-aware: the *testbench* should be agnostic to whether one or 128 cores are instantiated. This implies strong separation of (a) the core DUT, (b) the core testbench, and (c) the multicore wrapper.
|
||||
- **Custom design.** No third-party core is being verified; there is no vendor's pre-built verification IP. All environments must be built or assembled.
|
||||
- **Repository context.** The related verification documents are all placeholders ("SOON"). This is a greenfield verification program; the XH-1 project has the opportunity — and the burden — to define the methodology from first principles.
|
||||
- **RISC-V ISA.** The RISC-V ecosystem provides mature reference models (Spike, Sail), compliance / architectural test frameworks (`riscv-tests`, `riscv-arch-test`), and open-source verification tooling (`riscv-formal`, `riscv-dv`). These are the natural anchors.
|
||||
- **Tool posture (OPEN QUESTION 3).** Without evidence in the repository, the project's tool posture (open-source only, commercial EDA, mixed) is OPEN. The methodology composition must remain valid across posture choices.
|
||||
- **Design language (OPEN).** The choice of design language — SystemVerilog, traditional Verilog, Chisel/Spinal HDL, Bluespec — has direct implications for verification methodology (e.g., formal property availability on generated RTL, simulation throughput, the place of unit-level testing at the HDL-source level vs. the generated-RTL level). This is OPEN pending XH-1 design-language disclosure.
|
||||
- **Microarchitectural class (OPEN QUESTION 2).** The pipeline class — in-order vs. out-of-order, superscalar width, depth of speculation, branch-prediction aggressiveness — materially changes the cost-benefit of formal vs. dynamic and the complexity of pipeline flush / replay verification. The methodology composition must accommodate both extremes.
|
||||
- **Power architecture (OPEN QUESTION 7).** Whether the core supports clock gating, power gating, retention, and/or DVFS is OPEN. The verification methodology must accommodate each combination. Assuming any specific power feature is invalid without evidence.
|
||||
|
||||
## 128-Core Scalability
|
||||
|
||||
Replication does not change which verification techniques apply to a single core, but it changes their cost-benefit and their composition.
|
||||
|
||||
- **Core testbench scalability.** A core testbench written for a single instance must be instantiable inside a multicore wrapper without modification. This implies the testbench interfaces to the core through a standard transaction-level interface (or at least a clearly defined boundary), not through global signals.
|
||||
- **Cost-benefit of formal at replication.** Per-block formal verification has high fixed cost but a per-instance cost of zero: once a control property is proven, it holds for every replicated instance. Under the replication-homogeneity assumption, formal's per-die marginal cost decreases as the replication count increases. The opposite is true for dynamic simulation at full coverage: a per-instance cost must be paid for each replicated instance if all instances are to be exercised at full coverage. This trade-off is a primary argument for the hybrid approach at 128-core scale.
|
||||
- **Workload scaling.** Running the full core testbench in a 128-core environment is generally unnecessary; sampling (verify a subset of cores under full stimulus, sweep register state across all 128) is a more efficient pattern. The sampling strategy is a multicore concern but must be supported by the core testbench's reset/configuration interface. The statistical argument for the chosen sampling fraction (defect-escape probability vs. simulation cost) is a multicore-layer deliverable; the core testbench must support the interface.
|
||||
- **Coverage aggregation.** Per-core coverage data must be aggregated across the 128 instances. The core testbench should emit coverage in a form that is identifiable by instance index.
|
||||
- **Debug bandwidth.** 128 cores each generating waveform/log data is impractical. The core testbench must distinguish between detailed-trace mode (one core, full visibility) and lightweight-trace mode (128 cores, summary only).
|
||||
- **Compile and elaboration cost.** Elaboration of a 128-core testbench can dominate runtime. The core testbench should be elaborable standalone and in a multicore wrapper without recompiling the core RTL.
|
||||
- **Symmetry assumption revisit.** The replication-homogeneity assumption is load-bearing throughout this section. Heterogeneous cores (e.g., big/little, mixed application/accelerator, per-core ISA-subset configuration) invalidate the per-die marginal-cost argument and require per-variant verification, including per-variant testbenches, per-variant coverage, and per-variant formal. The methodology should be designed so that the additional cost of a variant is linear in the number of variants, not linear in 128 × variants.
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
The following are framework-level considerations. INSUFFICIENT EVIDENCE exists to provide numerical throughput estimates without XH-1-specific RTL data, simulator choice, and toolchain selection; any specific number is unsourced and is therefore not asserted here.
|
||||
|
||||
- **Testbench throughput.** Dynamic simulation throughput determines how many random seeds can be run per unit wall-clock. Constrained-random at the core level is sensitive to simulator choice, RTL abstraction level, and coverage instrumentation; representative numbers are available in vendor documentation (commercial simulators) and in the open-source Verilator and cocotb benchmark literature, but are workload-dependent and cannot be quoted as typical without measurement on the XH-1 DUT.
|
||||
- **Compiler/optimizer.** Coverage instrumentation slows simulation. Production coverage runs and fast-turnaround debug runs should be separable.
|
||||
- **Co-simulation overhead.** Spike co-simulation overhead per committed instruction is non-trivial. Synchronization strategy (per-instruction vs. batched) is a tradeoff between debug granularity and throughput.
|
||||
- **FPGA prototype throughput.** FPGA prototype throughput is set by Fmax, which is determined by LUT count, routing, and the memory subsystem. Quoted MHz figures for an XH-1 core on a specific FPGA platform are not asserted here.
|
||||
- **Regression wall-clock budget.** A nightly regression that runs N seeds on a single core must fit within the available compute budget. Cloud burst capacity is typical.
|
||||
|
||||
INSUFFICIENT EVIDENCE to provide numerical throughput estimates without XH-1-specific RTL data and toolchain selection.
|
||||
|
||||
## Area Considerations
|
||||
|
||||
- **On-die verification infrastructure.** XH-1 is described as a 128-core design. The verification *methodology* does not directly consume die area, but decisions made during verification (e.g., embedding assertions in RTL, adding observability muxes for trace, adding per-core DFT structures) do.
|
||||
- **Trace and debug infrastructure.** Core-level verification informs the design of trace, performance counters, and debug-transport blocks. Larger trace infrastructure costs area; the verification program must justify the cost in terms of debug productivity and post-silicon coverage.
|
||||
- **DFT (Design-for-Test) compatibility.** Scan chains, BIST, and MBIST are typically owned by a separate DFT program. Core verification should ensure that the core's DFT structures do not alter functional behavior in mission mode.
|
||||
- **Replication-driven area optimization.** If the core's area is a major constraint (which is likely at 128 cores on a single die), the verification plan must protect against defects that would force a re-spin with re-spinned core area.
|
||||
|
||||
INSUFFICIENT EVIDENCE to quantify area without access to XH-1's core area budget or process node.
|
||||
|
||||
## Power and Energy Considerations
|
||||
|
||||
The following considerations apply **if and only if** XH-1 implements the corresponding power features. Each is OPEN QUESTION 7-conditional.
|
||||
|
||||
- **Power-aware verification (conditional).** If the core supports power gating or retention, the core testbench must be able to drive representative switching activity to enable power estimation, and the testbench must verify that power-gated domains behave correctly (controlled wake, retention restore, ordering with respect to coherence traffic).
|
||||
- **Energy proportionality (conditional).** If the core supports clock gating at the instruction or block level, the testbench should verify that idle and active transitions behave correctly at the architectural interface (no observable state loss in idle, correct wake on interrupt).
|
||||
- **Thermal and DVFS interactions (conditional).** If XH-1 supports DVFS at the core level, voltage/frequency transitions and their effects on memory coherence, pipeline flush behavior, and CSR access must be verified. This is a cross-cutting concern between core and system verification.
|
||||
- **Power estimation accuracy.** Pre-silicon power estimation depends on workload realism. A core testbench that only runs synthetic random code will produce misleading power numbers.
|
||||
|
||||
No power feature is assumed. Verification provisions for any power feature must be activated only when OPEN QUESTION 7 is resolved.
|
||||
|
||||
## Implementation Considerations
|
||||
|
||||
- **Methodology composition.** The project must select one of the alternative strategies (or a hybrid) and codify it in the verification plan.
|
||||
- **Toolchain selection.** Open-source (Verilator, cocotb, SymbiYosys, `riscv-formal`, `riscv-dv`, Spike, RISCV-ISAC, GTKWave) vs. commercial (Synopsys VCS, Cadence Xcelium, Siemens Questa, JasperGold, FormalPro) is a strategic decision. The choice cascades through licensing, hiring, training, and CI. These categories are not equivalent in role: Verilator and cocotb are simulation/cosim frameworks; SymbiYosys is a formal-verification flow; `riscv-formal` is an ISA-level formal property set; `riscv-dv` is a stimulus generator; RISCV-ISAC is a coverage tool. The choice within each role is independent.
|
||||
- **Reference model choice.** Spike (C++, instruction-accurate) vs. Sail (functional-language specification implementation) vs. a custom golden model. Spike is the most common reference; the project must document any corner-case behaviors of the chosen reference against the RISC-V specification.
|
||||
- **Coverage database.** Coverage data are stored in vendor-specific formats (e.g., Synopsys UCDB / DDC, Cadence IMC, Siemens UCDB) and may be aggregated into a custom backend (e.g., SQLite) for CI. The format is a tool-vendor concern; the aggregation backend is a CI concern. These are distinct choices.
|
||||
- **Test categorization.** Tests must be tagged by domain (base integer, M-ext, A-ext, F/D-ext, privileged, vector, …), by intent (sanity, corner, regression, performance), and by status (passing, known-fail, expected-fail).
|
||||
- **Seed management.** Random seeds should be reproducible. Seed pools should be partitioned into "core regressions", "block regressions", and "fuzzing".
|
||||
- **Bring-up sequencing.** Block-level → core-level → multicore → system. The core testbench is the transition point between block-level and system-level work.
|
||||
|
||||
## Sign-Off Criteria
|
||||
|
||||
A core verification sign-off is the point at which the methodology exits the coverage-closure funnel. The following are candidate criteria; numeric thresholds are PROPOSAL pending OPEN QUESTION resolution.
|
||||
|
||||
- **Functional coverage.** All defined cover points hit, with cross-coverage exercised. Threshold PROPOSAL: ≥ 95% cover-point hit, ≥ 90% cross-coverage hit; remaining items must be justified as unreachable or formally proven. Final threshold to be set in the verification plan.
|
||||
- **Code coverage.** Statement, branch, expression, toggle, and FSM coverage reported; excluded regions justified. Threshold PROPOSAL: ≥ 95% statement, ≥ 90% branch, with manual exclusion of reset/DFT/clock-gating logic. Final threshold to be set in the verification plan.
|
||||
- **Assertion coverage.** All SVA / immediate assertions vacuously satisfied or actively proven; assertion cover points hit. Open assertions must be reviewed and dispositioned.
|
||||
- **Formal closure.** Per-block formal sign-off, with proof closure recorded for each property. Inconclusive results must be either strengthened (via abstraction / decomposition) or accepted with risk justification.
|
||||
- **ISA compliance evidence.** All applicable tests in `riscv-arch-test` (the architectural test framework) and `riscv-tests` (the legacy compliance suite) pass. Signed certificate or equivalent record retained.
|
||||
- **Mutation testing.** Periodic injection of known-defect mutations into a frozen RTL revision; a target mutation-kill rate PROPOSAL: ≥ 80% of injected mutations detected by the testbench + assertions + formal. Final threshold to be set in the verification plan.
|
||||
- **Reference model validation.** Spike or Sail validated against the chosen RISC-V compliance tests before being used as a checker; validation evidence archived.
|
||||
- **Testbench self-check verification.** Every scoreboard check has at least one negative test that is expected to fail.
|
||||
- **Reproducibility.** Every sign-off run is reproducible from committed source, seed, and tool version.
|
||||
|
||||
## Verification Considerations
|
||||
|
||||
Core verification is itself subject to verification — the verification of the verification environment:
|
||||
|
||||
- **Testbench self-checking.** All scoreboard checks must be end-of-test verifiable; every check should be paired with at least one negative test that is expected to fail.
|
||||
- **Reference model validation.** The reference model (Spike, Sail) must be validated against the chosen RISC-V compliance tests before being used as a checker.
|
||||
- **Coverage sanity.** Cover points must be reviewed to ensure they are not trivially self-firing.
|
||||
- **Mutation testing.** A subset of RTL defects should be intentionally injected to confirm the testbench catches them. See `research/10-verification/fault-injection.md`.
|
||||
- **Tool qualification.** If the verification tools are used in any safety-relevant context, tool qualification (per ISO 26262, DO-254, or similar) is required. The applicability of these standards to XH-1 is OPEN QUESTION 8.
|
||||
- **Version control and baselines.** Every test run must be reproducible from committed source. Coverage baselines must be versioned.
|
||||
|
||||
## Software Considerations
|
||||
|
||||
- **Toolchain integration.** The core verification environment must integrate with the software toolchain (compiler, assembler, linker, Spike, debug server) to enable co-simulation and to run real programs.
|
||||
- **Firmware.** Firmware that runs on the core (boot code, exception handlers, interrupt dispatchers) is itself verification target; the core testbench should support loading and running firmware images.
|
||||
- **Operating system implications.** A core that supports an OS (Linux, RTOS) must be verified with syscall-heavy workloads. The core testbench should provide OS-aware run modes.
|
||||
- **Debug and trace protocols.** The core's debug-transport module (per the RISC-V Debug specification) must be exercised. JTAG or cJTAG interactions with the core's debug module are typically a core-level concern.
|
||||
- **Performance analysis tools.** Tools that analyze trace data, performance counters, and profile information should be able to consume the artifacts the core testbench produces.
|
||||
- **Software developer ergonomics.** The verification team's interfaces (test writers, coverage reviewers) should be ergonomic. A "developer can run a single test and see results in under a minute" goal is a useful operational target.
|
||||
|
||||
## Security Considerations (Out of Scope, but Flagged)
|
||||
|
||||
**ASSUMPTION (out-of-scope for this revision):** Transient-execution / side-channel verification (e.g., Spectre-class, Meltdown-class, and constant-time verification) is treated as out of scope for the XH-1 core verification layer for this revision. For a 128-core design with privilege boundaries, this is a real concern, and a follow-up document should cover it. The current methodology is sufficient for functional and microarchitectural correctness but does not, by itself, provide security assurance.
|
||||
|
||||
## Proposed Direction
|
||||
|
||||
PROPOSAL — A **hybrid methodology** for XH-1 core verification, with the following structure:
|
||||
|
||||
1. **Dynamic simulation** as the primary verification engine, with the framework (UVM vs. lightweight) gated on OPEN QUESTION 3 (tool posture), OPEN QUESTION 4 (team size and timeline), and the design-language choice.
|
||||
- If UVM: pair with `riscv-dv` as the instruction-stream generator and Spike or Sail as the reference model.
|
||||
- If lightweight: pair `riscv-dv` (or a custom generator) with cocotb / Verilator and Spike.
|
||||
2. **Formal verification** applied to control-dominated blocks: fetch, decode, CSR file, MMU/TLB, exception/interrupt delivery, and the debug module. Coordinate with `research/10-verification/formal-verification.md` for tool selection. Apply `riscv-formal` as an ISA-level formal layer where the open-source flow is in scope.
|
||||
3. **Co-simulation against a RISC-V reference model** (Spike or Sail) for ISA-compliance evidence. Coordinate with `research/10-verification/isa-compliance.md`. Pin the RISC-V ISA, Privileged ISA, and Debug spec versions.
|
||||
4. **Directed tests** for XH-1 microarchitectural corner cases that random cannot reach economically. The corner cases to target are conditional on the in-order / out-of-order answer in OPEN QUESTION 2.
|
||||
5. **FPGA prototyping** for throughput-driven verification and as a substrate for fault injection and multicore bring-up.
|
||||
6. **Regression infrastructure** with version-controlled seeds, baselines, and coverage data. Coordinate with `research/10-verification/regression.md`.
|
||||
7. **Coverage closure** as an explicit, tracked funnel with the sign-off criteria in the Sign-Off Criteria section. Numeric thresholds are deferred to the verification plan.
|
||||
8. **Reusable testbench architecture** so the core testbench composes inside the multicore testbench without modification, supporting both the per-die cost argument and the sampling strategy.
|
||||
9. **Mutation testing** as a periodic check that the testbench is still doing useful work. Coordinate with `research/10-verification/fault-injection.md`.
|
||||
|
||||
The hybrid is proposed; the specific framework within the hybrid (UVM vs. lightweight) is gated on OPEN QUESTION 3 and OPEN QUESTION 4, and the power-feature sub-portion is gated on OPEN QUESTION 7. The document does not pick between alternatives that the OPEN QUESTIONS do not yet support.
|
||||
|
||||
## Confidence
|
||||
|
||||
- **High confidence:** The general methodology composition (dynamic + formal + co-simulation + directed + prototype + regression + coverage closure) corresponds to the IEEE 1800.2 UVM standard, the IEEE 1800 SystemVerilog assertion subset, and the RISC-V International published ISA / Privileged / Debug specifications, and is broadly accepted in industrial practice. The references supporting each component are cited at the point of use.
|
||||
- **High confidence:** The 128-core scaling argument (formal has near-zero per-instance marginal cost under replication homogeneity; dynamic at full coverage has per-instance cost) is a direct consequence of the replication-homogeneity assumption and does not require empirical data.
|
||||
- **Medium confidence:** Specific tool selections, sign-off numeric thresholds, and area/power tradeoffs, given the absence of repository evidence on the OPEN QUESTIONS.
|
||||
- **Low confidence (OPEN):** XH-1's tool licensing posture, core microarchitecture details, ISA profile selection, team size, design language, and power architecture. These drive the specifics within the proposed hybrid.
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. What RISC-V profile does the XH-1 core implement? Candidate RISC-V profiles are defined against a base XLEN: e.g., RVA23U64 / RVA23S64 are 64-bit application / supervisor profiles, while a 32-bit core would follow a different profile track (e.g., RVA22U32). The reference model (Spike variant, Sail configuration) and the compliance test scope follow from this choice.
|
||||
2. Is the XH-1 core in-order or out-of-order? The verification plan complexity differs substantially. The instructions explicitly disallow assuming either without repository evidence.
|
||||
3. What is XH-1's toolchain posture (open-source only, commercial EDA, mixed)?
|
||||
4. What is the team size and timeline? Methodology cost scales with team capacity.
|
||||
5. What is the core's microarchitectural contract — what blocks are in scope for unit verification, and what is the boundary between core and multicore? See `research/10-verification/unit-verification.md` and `research/10-verification/multicore-verification.md`.
|
||||
6. Are the 128 cores architecturally identical, or are there variants (big/little, mixed application/accelerator, per-core ISA-subset configuration)? Heterogeneity changes the testbench architecture and invalidates the per-die cost-leverage argument.
|
||||
7. What is the core's power architecture — is clock gating, power gating, retention, or DVFS at the core level, and what are the state-retention semantics?
|
||||
8. What is XH-1's relationship to functional safety standards (ISO 26262, IEC 61508, DO-254, or none)?
|
||||
9. How will the XH-1 core's trace, debug, and performance-counter interfaces be designed, and how will core verification consume them?
|
||||
10. What is the expected defect rate target at tape-out, and how does the verification program demonstrate closure against that target?
|
||||
11. What is the XH-1 design language (SystemVerilog, Verilog-2001, Chisel/Spinal HDL, Bluespec), and does verification target the HDL source level, the generated-RTL level, or both?
|
||||
|
||||
## Sources
|
||||
|
||||
- IEEE 1800.2-2020, *Standard for Universal Verification Methodology Based on Accellera SystemVerilog 3.1*. Defines UVM.
|
||||
- IEEE 1800-2017, *Standard for SystemVerilog — Unified Hardware Design, Specification, and Verification Language*. Defines SVA and immediate assertions.
|
||||
- RISC-V International, *The RISC-V Instruction Set Manual, Volume I: Unprivileged Architecture* (versioned).
|
||||
- RISC-V International, *The RISC-V Instruction Set Manual, Volume II: Privileged Architecture* (versioned).
|
||||
- RISC-V International, *RISC-V External Debug Support* (versioned).
|
||||
- RISC-V International, *RISC-V Profiles* (versioned, e.g., RVA23).
|
||||
- Symbiotic EDA / YosysHQ, *SymbiYosys* documentation (open-source formal verification flow).
|
||||
- `riscv-formal` framework (Verilog, BSD-licensed), maintained at github.com/YosysHQ/riscv-formal.
|
||||
- `riscv-dv` instruction-stream generator, maintained at github.com/google/riscv-dv.
|
||||
- `riscv-tests` and `riscv-arch-test`, maintained by RISC-V International.
|
||||
- Spike (the official RISC-V ISA simulator), maintained at github.com/riscv-software-src/riscv-isa-sim.
|
||||
- Sail RISC-V formal specification, maintained at github.com/rems-project/sail.
|
||||
- Watson et al., *Kami: a Single-Document, Executable, Formal Specification of the RISC-V ISA* (Cambridge). Open-source formal ISA specification in Coq with a Bluespec front-end.
|
||||
|
||||
Quantitative throughput, area, power, and timing claims are not asserted in the body. The XH-1 repository documents referenced (under `research/10-verification/`) are placeholders ("SOON") and do not provide factual content to cite. No measurements, benchmarks, processor capabilities, URLs outside the repositories named above, or experimental results have been invented.
|
||||
+301
File diff suppressed because one or more lines are too long
+370
@@ -0,0 +1,370 @@
|
||||
# Core Verification
|
||||
|
||||
## Status
|
||||
|
||||
DRAFT — Research framework established; all claims are explicitly classified by epistemic status (FACT with citation, ASSUMPTION, PROPOSAL, RECOMMENDATION, OPEN QUESTION, or INSUFFICIENT EVIDENCE). Concrete sign-off criteria, tool selection, and quantitative thresholds are deferred until the OPEN QUESTION inputs are resolved. Example numeric thresholds in the Sign-Off section are starting points for the verification plan and are not validated targets.
|
||||
|
||||
## Abstract
|
||||
|
||||
This document investigates verification strategies for the XH-1 core, defined here as a single processing core instance within a custom 128-core RISC-V processor. Core verification is the layer of the verification methodology stack concerned with demonstrating that one core conforms to its architectural specification, microarchitectural contract, and integration assumptions prior to and independent of multi-core interactions. For a 128-core design, the core is the unit of replication; defects undetected at this layer replicate 128-fold across the die, amplifying area, power, yield, and debug cost. This document frames the research questions, surveys existing approaches (simulation, formal verification, hardware-assisted verification, FPGA prototyping, emulation), analyzes their applicability to a custom RISC-V core, and identifies the gaps that the XH-1 project must close given that the existing related documents in `research/10-verification/` are placeholders ("SOON").
|
||||
|
||||
## Research Question
|
||||
|
||||
What verification methodology, toolchain, environment abstraction, coverage model, and sign-off criteria are appropriate for verifying a single XH-1 core such that the methodology:
|
||||
|
||||
1. Provides evidence that the core implements the RISC-V ISA subset selected for XH-1 (see `research/10-verification/isa-compliance.md`).
|
||||
2. Provides evidence that the core's microarchitecture meets timing, area, and power targets for a design that will be replicated 128 times.
|
||||
3. Scales economically — verification effort, compute cost, and human review time must not grow linearly with the 128-core replication factor.
|
||||
4. Produces artifacts (testbenches, coverage models, formal proofs, regression suites) that compose cleanly with the surrounding verification layers (unit, multicore, system, post-silicon).
|
||||
5. Is implementable given that the related XH-1 documents in area 10 are not yet populated, indicating a greenfield verification program.
|
||||
|
||||
## Background
|
||||
|
||||
Verification of a processor core is conventionally decomposed into:
|
||||
|
||||
- **Unit verification** — verification of individual blocks (ALU, FPU, register file, branch predictor, fetch/decode structures, load-store unit, replay logic, CSR file, MMU/TLB, caches). See `research/10-verification/unit-verification.md`.
|
||||
- **Core verification** — verification of the integrated core as a single design under test (DUT), exercising instruction-stream and exception behavior at the architectural interface, plus internal microarchitectural invariants.
|
||||
- **Multicore verification** — coherence, consistency, interconnect, and shared-resource correctness across multiple core instances. See `research/10-verification/multicore-verification.md`.
|
||||
|
||||
For XH-1, the core is the replication unit. The cost model is asymmetric under the replication-homogeneity assumption below: the marginal cost of catching a defect at the core level is lower than the marginal cost of catching it after tape-out, in silicon debug, or in the field. The phrase "every bug caught at the core layer is caught 128 times" is shorthand for: every defect that escapes core verification is replicated 128-fold in silicon, multiplying area, power, yield, and debug impact. The verification effort itself is paid once; the fix is paid once; the unfixed-defect cost is multiplied by the replication count.
|
||||
|
||||
**ASSUMPTION (replication homogeneity):** The 128 XH-1 cores are architecturally identical instances of a single RTL core produced by elaboration of the same source. Heterogeneous cores, per-core configuration, or runtime variants invalidate the replication-leverage argument and require per-variant verification. This is treated as an assumption because no repository evidence contradicts it; OPEN QUESTION 6 below revisits it. Note that the formal "per-instance cost of zero" claim for proven properties holds for bit-identical replicated RTL; if the source is parameterized and elaborated with per-instance parameters, the proof must be re-established for any parameter set that differs, and the cost is not strictly zero.
|
||||
|
||||
**ASSUMPTION (target implementation class):** XH-1 targets ASIC tape-out. FPGA-only deployment would change the methodology (no DFT/ATPG, different sign-off criteria, no post-silicon bring-up in the same sense). This assumption is conditional and is revisited in OPEN QUESTION 12.
|
||||
|
||||
**ASSUMPTION (verification language availability):** The verification language of the testbench is the same as or interoperable with the design language. This is revisited in OPEN QUESTION 11.
|
||||
|
||||
Core verification draws on several formal and semi-formal techniques:
|
||||
|
||||
- **Dynamic simulation** with constrained-random or directed tests, against an ISA reference model, with functional coverage.
|
||||
- **Static (formal) verification** — model checking (bounded model checking, k-induction, IC3/PDR), equivalence checking (combinational and sequential), and property proving against the RTL.
|
||||
- **Hardware-assisted verification** — FPGA prototyping and emulation (commercial emulators, open-source flows such as FireSim on Amazon EC2 F1), trading compile time for runtime speed.
|
||||
- **Co-simulation** — two simulation engines running in lockstep (e.g., an RTL simulator and an architectural reference-model engine such as Spike) synchronized at defined events.
|
||||
- **Waveform and assertion-based methodologies** — SystemVerilog Assertions (SVA), PSL, custom monitors, end-of-test checkers.
|
||||
|
||||
Coverage is a central concern. Functional coverage, code coverage (line, branch, expression, toggle, FSM), and cross-coverage of architectural state are the standard dimensions. For a custom ISA implementation, the relationship between ISA compliance coverage and microarchitectural coverage must be defined explicitly.
|
||||
|
||||
Standard core-verification concerns that must be addressed in addition to instruction-stream behavior:
|
||||
|
||||
- **Lint** — static analysis of RTL style, common bug patterns, and synthesizability (e.g., Verilator lint, commercial lint tools). A prerequisite to dynamic and formal work.
|
||||
- **Clock-domain crossing (CDC) and reset-domain crossing (RDC)** — analysis of signals crossing asynchronous clock or reset domains. Typically owned by a separate methodology but the core must be CDC/RDC-clean for integration.
|
||||
- **X-propagation** — controlled handling of unknown ('X) propagation in simulation, using X-pessimism removal or X-init optimization, to avoid false coverage from masked X's.
|
||||
- **Reset verification** — power-on reset, software reset, debug reset, partial reset, and reset-domain interaction.
|
||||
- **Power intent** — if power features are present (see OPEN QUESTION 7), UPF/CPF power intent formats integrate with simulation to model power-gated, retention, and isolation behavior.
|
||||
- **DFT/ATPG interaction** — scan, BIST, and MBIST structures must not alter mission-mode behavior; their interaction with the core testbench is a coordination point.
|
||||
|
||||
These are not optional; their absence in a verification plan is a known gap in pre-silicon methodology.
|
||||
|
||||
## Existing Approaches
|
||||
|
||||
The following approaches are established in industry and academic practice. Their applicability to XH-1 is analyzed; none is recommended in isolation.
|
||||
|
||||
### A. Constrained-Random Simulation with UVM
|
||||
|
||||
**FACT:** Universal Verification Methodology (UVM) is standardized as IEEE 1800.2-2020, *Standard for Universal Verification Methodology Based on Accellera SystemVerilog 3.1* [IEEE 1800.2-2020], and is widely used in industrial block- and core-level processor verification for constrained-random stimulus generation, reference model integration, self-checking scoreboards, functional coverage, and phasing.
|
||||
|
||||
**ASSUMPTION (conditional on design language):** UVM is applicable only if the XH-1 design language supports SystemVerilog class constructs — either the source is SystemVerilog, or the verification environment targets SystemVerilog regardless of source language. If the design language is Verilog-2001 only, or if the project commits to a Chisel/Spinal HDL or Bluespec-only flow, UVM is not directly applicable at the HDL-source level and an alternative testbench harness is required. This assumption is revisited in OPEN QUESTION 11.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Apply UVM as the primary dynamic verification environment for XH-1 core-level testing, paired with a chosen ISA reference (Spike or Sail). Sequence generation over the RISC-V ISA state space; reference model in co-simulation; scoreboard for architectural state comparison per committed instruction; explicit cover points for instruction × operand-mode × privilege × exception interactions.
|
||||
|
||||
### B. Directed and Hand-Written Tests
|
||||
|
||||
**FACT:** Constrained-random alone does not reach corner-case coverage of complex pipelines, corner-case CSR interactions, or rare exception sequences. Industry practice supplements with directed tests written by architects and designers.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** A XH-1-specific directed test suite for pipeline hazard combinations specific to the XH-1 microarchitecture, corner cases in the chosen RVA profile subset, reset, wake, low-power entry/exit, and debug transport (if present at core level).
|
||||
|
||||
### C. Formal Property Verification
|
||||
|
||||
**FACT:** Bounded model checking, k-induction, and IC3/PDR are applied to control-dominated blocks: fetch, decode, CSR, MMU/TLB, branch predictor, retirement, and exception delivery. Commercial model checkers include Cadence JasperGold, Synopsys VC Formal, and Siemens FormalPro. Open-source flows include SymbiYosys (SBY), based on the Yosys synthesis and formal toolchain. Equivalence checking is a distinct formal category: tools include Synopsys Formality and Cadence Conformal, used to verify that two RTL representations (e.g., pre- and post-synthesis, or pre- and post-retiming) are functionally equivalent. Equivalence checkers are not model checkers and are not used for property proving.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Per-block formal sign-off for the control-dominated blocks listed above, end-to-end pipeline flushing, exception priority, and CSR read/write semantics. Formal proofs of privileged ISA semantics (e.g., `mstatus`, `sstatus`, `satp`, `mip`/`sip` behavior) provide a high-confidence anchor that random tests cannot match in isolation. Sequential equivalence checking between major RTL revisions is a separate, complementary activity.
|
||||
|
||||
**ASSUMPTION (RISC-V privileged-ISA versioning):** The CSR semantics used as formal-proof targets correspond to the specific RISC-V Privileged ISA version adopted by XH-1. The unprivileged ISA, privileged ISA, and Debug specification each have versioned releases tracked by RISC-V International; the verification plan must be pinned to those versions [RISC-V International, *Unprivileged Specification*, *Privileged Specification*, *Debug Specification*].
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Coordinate with `research/10-verification/formal-verification.md` for tool selection (commercial model checkers vs. SymbiYosys / SBY) and with `research/10-verification/isa-compliance.md` for the privileged-ISA version.
|
||||
|
||||
### D. Co-Simulation with an ISA Reference
|
||||
|
||||
**FACT:** Comparing DUT architectural state against Spike (the official RISC-V ISA simulator, C++ implementation) or Sail (a first-order imperative language designed for ISA specification, with a RISC-V model generated from the RISC-V ISA formal definition) per instruction is a widely deployed pattern. Spike + RTL constitutes co-simulation in the strict sense: two simulation engines synchronized at defined events. The DUT may run ahead of the reference using a deferred-check queue, with results compared at synchronization points. A single simulator that incorporates the reference model as a transactor is more accurately described as reference-model checking rather than co-simulation.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Required for ISA-compliance evidence. See `research/10-verification/isa-compliance.md`. Reference model parallelism is a 128-core scaling concern: Spike and Sail are single-threaded by default; running 128 reference models in lockstep with 128 RTL instances is itself a significant engineering problem. The pattern of choice is a single shared reference model servicing all DUT instances through a deferred-check queue, or sampling (multicore-layer decision) with the core testbench supporting the interface.
|
||||
|
||||
### E. FPGA Prototyping and Emulation
|
||||
|
||||
**FACT:** FPGA-based prototyping runs the RTL at higher throughput than software simulation, and emulation platforms (commercial: Palladium, Veloce, ZeBu; open-source: FireSim on Amazon EC2 F1) provide a middle ground. The throughput multiplier is workload- and instrumentation-dependent; published numbers for FPGA prototyping vs. software simulation range roughly from 10× to over 1000× depending on workload, Fmax, and observability overhead. Quoted multipliers are approximate and are not asserted here as specific numbers for XH-1.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Use FPGA prototyping to speed up core testbench bring-up by running entire test programs end-to-end, to enable fault-injection campaigns that are infeasible in simulation (see `research/10-verification/fault-injection.md`), and as the natural place to begin validating the multicore layer; core verification is the upstream prerequisite. Commercial emulation is a viable alternative for very large workloads or when observability requirements are relaxed; the decision depends on licensing posture (OPEN QUESTION 3) and workload characteristics.
|
||||
|
||||
### F. Regression Infrastructure
|
||||
|
||||
**FACT:** Regression management — version control of testbenches, seeds, baselines, coverage databases — is a load-bearing component of any non-trivial core verification program. See `research/10-verification/regression.md`.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** The XH-1 regression infrastructure must support a single-core nightly run, a multicore run, and a system run with shared seeds where appropriate.
|
||||
|
||||
### G. Coverage Closure Methodology
|
||||
|
||||
**FACT:** Coverage closure combines functional coverage (defined by the verification plan), code coverage (uncovered RTL), and assertion coverage. Industry practice treats coverage closure as a managed funnel: cover points → cross coverage → corner case identification → directed/random test addition → formal where possible. Coverage baselines drift as RTL changes; a baseline-management policy is required.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Codify the funnel explicitly in the verification plan with tracked stages, baseline-management policy, and exit criteria per stage.
|
||||
|
||||
### H. Assertion-Based Verification
|
||||
|
||||
**FACT:** SystemVerilog Assertions (IEEE 1800-2017, *Standard for SystemVerilog*) and immediate assertions encode design intent and serve both as live checkers and as inputs to formal tools.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Define an SVA library for XH-1 invariants (e.g., "no retired instruction has a privileged side effect inconsistent with its mode", "no speculative architectural state is observable at the architectural interface").
|
||||
|
||||
### I. Open-Source ISA-Level Formal Verification
|
||||
|
||||
**FACT:** The `riscv-formal` framework (YosysHQ / Symbiotic EDA, Verilog, BSD-licensed) provides a directly applicable open-source ISA-level formal verification flow for RISC-V cores, consisting of a checkable RISC-V ISA specification encoded as a set of formal properties that are conjoined with a wrapper around the DUT and discharged by an SMT solver. The Kami project (Cambridge) provides an alternative formal RISC-V ISA specification in Coq with a Bluespec front-end; the project documentation and the publication by the Kami team should be cited from the project's own canonical sources rather than from a secondary attribution.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Evaluate `riscv-formal` as a complement to block-level formal. It directly targets ISA-compliance evidence with open-source tooling and is particularly relevant given the greenfield status of the XH-1 verification program.
|
||||
|
||||
### J. Open-Source Constrained-Random Generator
|
||||
|
||||
**FACT:** Google's `riscv-dv` is an open-source UVM-based constrained-random instruction-stream generator for RISC-V, widely used in industrial and academic flows. The license should be confirmed against the current repository state at the time of toolchain selection; this document does not assert a specific license version.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Evaluate `riscv-dv` as a generator either within a UVM environment or as a stimulus source for a lightweight harness.
|
||||
|
||||
## Alternative Designs
|
||||
|
||||
The following alternative *strategies* for the core verification methodology itself are candidates for XH-1. They are not mutually exclusive; the project must select a composition.
|
||||
|
||||
### Alternative 1: UVM-Centric with Formal Supplements
|
||||
|
||||
- **Approach:** UVM as the workhorse for dynamic verification, with formal applied to control-dominated blocks.
|
||||
- **Strengths:** Industry-standard tooling, well-understood coverage and sign-off conventions, mature ecosystem; integrates with `riscv-dv` as a generator and `riscv-formal` as an ISA-level formal layer.
|
||||
- **Weaknesses:** UVM has a steep learning curve and high setup cost. For a 128-core design, the UVM environment must be designed for reuse to avoid 128× rewrite cost. Requires SystemVerilog (OPEN QUESTION 11).
|
||||
|
||||
### Alternative 2: Lightweight Python/C++ Test Harness with Spike Co-Simulation
|
||||
|
||||
- **Approach:** A generator produces instruction streams and a C++ testbench (e.g., based on cocotb, Verilator, or a custom framework) compares architectural state against Spike.
|
||||
- **Strengths:** Lower setup cost than UVM. Excellent generator productivity. Easy integration with regression infrastructure. Open-source-friendly. `riscv-dv` can serve as the generator. Verilator-based flows are not bound to SystemVerilog UVM at the source level.
|
||||
- **Weaknesses:** Less mature scoreboard/reuse infrastructure than UVM. Coverage closure tooling is less standardized.
|
||||
- **Application to XH-1:** Reasonable choice if XH-1 has a small team, an open-source toolchain preference, or a constraint on commercial EDA licensing.
|
||||
|
||||
### Alternative 3: Formal-First
|
||||
|
||||
- **Approach:** Formal proofs form the primary verification evidence; dynamic simulation is used for reach and to validate properties that formal cannot discharge.
|
||||
- **Strengths:** Highest assurance per proven property. Strong fit for control logic and CSR behavior. `riscv-formal` provides a directly applicable open-source ISA-level formal layer. Under the replication-homogeneity assumption, proven properties hold for all 128 instances at near-zero marginal cost (modulo parameter-set caveats in the Background assumption).
|
||||
- **Weaknesses:** Pipeline formalization is hard. State-space complexity grows combinatorially for large cores. May not reach proof closure on timing-agnostic models without abstraction. Inconclusive results are not equivalent to failure; risk-acceptance must be tracked.
|
||||
|
||||
### Alternative 4: Emulation-Centric
|
||||
|
||||
- **Approach:** Skip large-scale software simulation; use FPGA/emulation throughput as the primary engine.
|
||||
- **Strengths:** Excellent long-workload capability; native fault-injection support.
|
||||
- **Weaknesses:** Poor waveform/observability; slow compile/turnaround; harder to drive corner-case stimulus; debug cycle is slow.
|
||||
|
||||
### Alternative 5: Hybrid (PROPOSAL — See Proposed Direction)
|
||||
|
||||
A combination of a dynamic framework (Alternative 1 or Alternative 2) for stimulus and reach, formal supplements (Alternative 3, including `riscv-formal` where applicable) for control logic, and FPGA prototyping/emulation for throughput and multicore bring-up.
|
||||
|
||||
## Comparison
|
||||
|
||||
The comparison is qualitative and is intended to make the trade-offs explicit, not to produce a numeric ranking. INSUFFICIENT EVIDENCE exists to rank absolute quality without XH-1-specific RTL data, tool licensing posture, and team size. The Hybrid column is a composition of the others and is not an independent point of comparison; the table is structured to show how each pure strategy performs on the dimensions, with the Hybrid column indicating how the strengths compose.
|
||||
|
||||
| Dimension | UVM-Centric | Lightweight Harness | Formal-First | Emulation-Centric | Hybrid |
|
||||
|---|---|---|---|---|---|
|
||||
| Setup cost | High | Low | High | Medium | High |
|
||||
| Per-test runtime | Slow | Slow–Medium | N/A (proof time) | Fast | Mixed |
|
||||
| Corner-case reach | Strong (with seeds) | Strong (with fuzzing) | Strong (when discharged) | Weak–Medium | Strong |
|
||||
| Assurance on control logic | Medium | Medium | High | Low–Medium | High |
|
||||
| Debug ergonomics | Good (waveform, logs) | Good | Strong counterexamples | Weak | Mixed |
|
||||
| Open-source viability | Medium (UVM IEEE 1800.2 is open; simulators are usually commercial) | Strong | Medium | Medium | Medium |
|
||||
| Fit for 128-core reuse | High (if designed so) | High (if designed so) | High (under replication homogeneity) | High | High |
|
||||
| Pre-tape-out risk reduction | Strong | Medium–Strong | Strong (partial; inconclusive ≠ proven) | Medium | Strong |
|
||||
|
||||
## Advantages
|
||||
|
||||
The advantages of investing in a robust core verification methodology for XH-1:
|
||||
|
||||
- **Replication leverage.** Under the replication-homogeneity assumption, an unfixed defect in the core RTL is replicated 128-fold in silicon. The cost-leverage argument is about the cost of escaping defects, not about replicated verification effort: verification is performed once, the defect (if found) is fixed once, and the fixed RTL is then replicated. The benefit of core-level detection is the avoidance of multiplied post-silicon cost.
|
||||
- **Cleaner multicore verification.** A well-verified core reduces the multicore verification problem to coherence, consistency, interconnect, and shared resources. See `research/10-verification/multicore-verification.md`.
|
||||
- **Foundation for post-silicon.** First-stage silicon bring-up is dramatically simpler when the core is well-understood and the debug interfaces (assertions, traces, scan) are well-designed. Post-silicon validation feeds back into pre-silicon coverage targets; this feedback loop is not detailed in this document and is left to the post-silicon layer.
|
||||
- **Reusable testbench assets.** Generator, scoreboard, coverage model, and SVA library can be reused across cores if XH-1 has core variants, or across revisions of the same core.
|
||||
- **Formal evidence on critical control.** Privileged ISA semantics are hard to verify by random simulation; formal proofs on these blocks provide a high-confidence floor.
|
||||
|
||||
## Disadvantages
|
||||
|
||||
Costs and risks of a robust core verification program:
|
||||
|
||||
- **Engineering cost.** A UVM-class environment plus formal plus emulation is a substantial program. The claim that it "can easily exceed the RTL design cost for a single core" is folkloric and is presented here as a planning consideration, not as a measured fact.
|
||||
- **Tooling lock-in.** Heavy UVM usage often implies commercial simulator licensing; this may not match XH-1's tool posture. Formal tool lock-in is similar: commercial model checkers differ in property language and run scripts.
|
||||
- **Slow turnaround.** Constrained-random simulation with full coverage at the core level is slow; coverage closure can take many cycles of "add test, rerun, close hole". Coverage baselines drift with RTL changes, requiring re-establishment.
|
||||
- **Reference model dependence.** Spike, Sail, or any reference may diverge from the RISC-V specification in interpretation of ambiguous or profile-specific corners; the verification program must track the chosen RVA profile carefully and pin the spec version of the reference.
|
||||
- **Coverage false confidence.** Hitting a cover point is not the same as proving correctness; the verification plan must distinguish cover and check.
|
||||
- **128-core replication pressure.** If the core testbench is not designed to compose with the multicore testbench, redundant effort emerges.
|
||||
- **Inconclusive formal ≠ proven.** Formal results of "inconclusive" must be risk-accepted, not treated as success.
|
||||
- **Equivalent mutants.** Mutation-kill rates are affected by equivalent mutants (mutations that produce behaviorally identical RTL); kill-rate targets are therefore noisy and must be interpreted with caution.
|
||||
|
||||
## XH-1 Considerations
|
||||
|
||||
XH-1-specific factors that constrain the methodology choice:
|
||||
|
||||
- **128-core replication.** The verification plan must be replication-aware: the *testbench* should be agnostic to whether one or 128 cores are instantiated. This implies strong separation of (a) the core DUT, (b) the core testbench, and (c) the multicore wrapper.
|
||||
- **Custom design.** No third-party core is being verified; there is no vendor's pre-built verification IP. All environments must be built or assembled.
|
||||
- **Repository context.** The related verification documents are all placeholders ("SOON"). This is a greenfield verification program; the XH-1 project has the opportunity — and the burden — to define the methodology from first principles.
|
||||
- **RISC-V ISA.** The RISC-V ecosystem provides mature reference models (Spike, Sail), compliance / architectural test frameworks (`riscv-tests`, `riscv-arch-test`), and open-source verification tooling (`riscv-formal`, `riscv-dv`). These are the natural anchors.
|
||||
- **Tool posture (OPEN QUESTION 3).** Without evidence in the repository, the project's tool posture (open-source only, commercial EDA, mixed) is OPEN. The methodology composition must remain valid across posture choices.
|
||||
- **Design language (OPEN QUESTION 11).** The choice of design language — SystemVerilog, traditional Verilog, Chisel/Spinal HDL, Bluespec — has direct implications for verification methodology (e.g., formal property availability on generated RTL, simulation throughput, the place of unit-level testing at the HDL-source level vs. the generated-RTL level, and the applicability of UVM at the source level). This is OPEN pending XH-1 design-language disclosure. Section A's UVM assumption is conditional on this resolution.
|
||||
- **Microarchitectural class (OPEN QUESTION 2).** The pipeline class — in-order vs. out-of-order, superscalar width, depth of speculation, branch-prediction aggressiveness — materially changes the cost-benefit of formal vs. dynamic and the complexity of pipeline flush / replay verification. The methodology composition must accommodate both extremes.
|
||||
- **Power architecture (OPEN QUESTION 7).** Whether the core supports clock gating, power gating, retention, and/or DVFS is OPEN. The verification methodology must accommodate each combination. Assuming any specific power feature is invalid without evidence. Power intent (UPF/CPF) integration is conditional on this.
|
||||
- **Target implementation class (OPEN QUESTION 12).** Whether XH-1 targets ASIC tape-out or FPGA-only deployment fundamentally changes the verification methodology (DFT/ATPG scope, post-silicon layer, sign-off criteria). Treated here as an assumption (ASIC) pending resolution.
|
||||
|
||||
## 128-Core Scalability
|
||||
|
||||
Replication does not change which verification techniques apply to a single core, but it changes their cost-benefit and their composition.
|
||||
|
||||
- **Core testbench scalability.** A core testbench written for a single instance must be instantiable inside a multicore wrapper without modification. This implies the testbench interfaces to the core through a standard transaction-level interface (or at least a clearly defined boundary), not through global signals.
|
||||
- **Cost-benefit of formal at replication.** Per-block formal verification has high fixed cost but, under bit-identical replication, a per-instance marginal cost of zero for the proven properties: once a control property is proven, it holds for every replicated instance of the same RTL. If the source is parameterized and elaborated with per-instance parameters, the proof must be re-established for any parameter set that differs. Under the replication-homogeneity assumption, formal's per-die marginal cost decreases as the replication count increases. The opposite is true for dynamic simulation at full coverage: a per-instance simulation cost must be paid for each replicated instance if all instances are to be exercised at full coverage, though the wall-clock growth is not strictly linear (the simulator elaborates and runs them in the same run, and the per-instance cost is amortized across the simulation kernel). The trade-off is a primary argument for the hybrid approach at 128-core scale, and the practical mitigation is sampling, not full per-instance coverage.
|
||||
- **Workload scaling.** Running the full core testbench in a 128-core environment is generally unnecessary; sampling (verify a subset of cores under full stimulus, sweep register state across all 128) is a more efficient pattern. The sampling strategy is a multicore concern but must be supported by the core testbench's reset/configuration interface. The statistical argument for the chosen sampling fraction (defect-escape probability vs. simulation cost) is a multicore-layer deliverable; the core testbench must support the interface.
|
||||
- **Coverage aggregation.** Per-core coverage data must be aggregated across the 128 instances. The core testbench should emit coverage in a form that is identifiable by instance index.
|
||||
- **Debug bandwidth.** 128 cores each generating waveform/log data is impractical. The core testbench must distinguish between detailed-trace mode (one core, full visibility) and lightweight-trace mode (128 cores, summary only).
|
||||
- **Compile and elaboration cost.** Elaboration of a 128-core testbench can dominate runtime. The core testbench should be elaborable standalone and in a multicore wrapper without recompiling the core RTL. Elaboration-time behavior (reset, clock tree, memory subsystem) can differ between 1-core and 128-core contexts; the core testbench must be valid in both.
|
||||
- **Testbench memory footprint.** 128 instances of a complex scoreboard and 128 reference models can dominate host memory. Memory-aware testbench design (shared reference models, deferred-check queues, summary-only trace mode) is required. This is acknowledged here and detailed in the multicore layer.
|
||||
- **Reference-model parallelism.** Spike and Sail are single-threaded by default. Running 128 reference models in lockstep with 128 RTL instances is a significant engineering problem. The recommended pattern is a single shared reference model servicing all DUT instances through a deferred-check queue, or sampling. This is a multicore-layer design constraint that the core testbench interface must support.
|
||||
- **Symmetry assumption revisit.** The replication-homogeneity assumption is load-bearing throughout this section. Heterogeneous cores (e.g., big/little, mixed application/accelerator, per-core ISA-subset configuration) invalidate the per-die marginal-cost argument and require per-variant verification, including per-variant testbenches, per-variant coverage, and per-variant formal. The methodology should be designed so that the additional cost of a variant is linear in the number of variants, not linear in 128 × variants.
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
The following are framework-level considerations. INSUFFICIENT EVIDENCE exists to provide numerical throughput estimates without XH-1-specific RTL data, simulator choice, and toolchain selection; any specific number is unsourced and is therefore not asserted here.
|
||||
|
||||
- **Testbench throughput.** Dynamic simulation throughput determines how many random seeds can be run per unit wall-clock. Constrained-random at the core level is sensitive to simulator choice, RTL abstraction level, and coverage instrumentation; representative numbers are available in vendor documentation (commercial simulators) and in the open-source Verilator and cocotb benchmark literature, but are workload-dependent and cannot be quoted as typical without measurement on the XH-1 DUT.
|
||||
- **Compiler/optimizer.** Coverage instrumentation slows simulation. Production coverage runs and fast-turnaround debug runs should be separable.
|
||||
- **Co-simulation overhead.** Spike co-simulation overhead per committed instruction is non-trivial. Synchronization strategy (per-instruction vs. batched) is a tradeoff between debug granularity and throughput. At 128 instances, the cost of a per-instruction synchronization across all instances is prohibitive; batched or sampled synchronization is the practical pattern.
|
||||
- **FPGA prototype throughput.** FPGA prototype throughput is set by Fmax, which is determined by LUT count, routing, and the memory subsystem. Quoted MHz figures for an XH-1 core on a specific FPGA platform are not asserted here.
|
||||
- **Regression wall-clock budget.** A nightly regression that runs N seeds on a single core must fit within the available compute budget. Cloud burst capacity is typical.
|
||||
|
||||
INSUFFICIENT EVIDENCE to provide numerical throughput estimates without XH-1-specific RTL data and toolchain selection.
|
||||
|
||||
## Area Considerations
|
||||
|
||||
- **On-die verification infrastructure.** XH-1 is described as a 128-core design. The verification *methodology* does not directly consume die area, but decisions made during verification (e.g., embedding assertions in RTL, adding observability muxes for trace, adding per-core DFT structures) do.
|
||||
- **Trace and debug infrastructure.** Core-level verification informs the design of trace, performance counters, and debug-transport blocks. Larger trace infrastructure costs area; the verification program must justify the cost in terms of debug productivity and post-silicon coverage.
|
||||
- **DFT (Design-for-Test) compatibility.** Scan chains, BIST, and MBIST are typically owned by a separate DFT program. Core verification should ensure that the core's DFT structures do not alter functional behavior in mission mode. If XH-1 is FPGA-only (OPEN QUESTION 12), DFT is out of scope.
|
||||
- **Replication-driven area optimization.** If the core's area is a major constraint (which is likely at 128 cores on a single die), the verification plan must protect against defects that would force a re-spin with re-spinned core area.
|
||||
|
||||
INSUFFICIENT EVIDENCE to quantify area without access to XH-1's core area budget or process node.
|
||||
|
||||
## Power and Energy Considerations
|
||||
|
||||
The following considerations apply **if and only if** XH-1 implements the corresponding power features. Each is OPEN QUESTION 7-conditional.
|
||||
|
||||
- **Power-aware verification (conditional).** If the core supports power gating or retention, the core testbench must be able to drive representative switching activity to enable power estimation, and the testbench must verify that power-gated domains behave correctly (controlled wake, retention restore, ordering with respect to coherence traffic). UPF/CPF power intent integration with the simulator is the standard mechanism.
|
||||
- **Energy proportionality (conditional).** If the core supports clock gating at the instruction or block level, the testbench should verify that idle and active transitions behave correctly at the architectural interface (no observable state loss in idle, correct wake on interrupt).
|
||||
- **Thermal and DVFS interactions (conditional).** If XH-1 supports DVFS at the core level, voltage/frequency transitions and their effects on memory coherence, pipeline flush behavior, and CSR access must be verified. This is a cross-cutting concern between core and system verification.
|
||||
- **Power estimation accuracy.** Pre-silicon power estimation depends on workload realism. A core testbench that only runs synthetic random code will produce misleading power numbers.
|
||||
|
||||
No power feature is assumed. Verification provisions for any power feature must be activated only when OPEN QUESTION 7 is resolved.
|
||||
|
||||
## Implementation Considerations
|
||||
|
||||
- **Methodology composition.** The project must select one of the alternative strategies (or a hybrid) and codify it in the verification plan.
|
||||
- **Toolchain selection.** Open-source (Verilator, cocotb, SymbiYosys, `riscv-formal`, `riscv-dv`, Spike, RISCV-ISAC, GTKWave) vs. commercial (Synopsys VCS, Cadence Xcelium, Siemens Questa, JasperGold, VC Formal, FormalPro, Formality, Conformal, Palladium, Veloce, ZeBu) is a strategic decision. The choice cascades through licensing, hiring, training, and CI. These categories are not equivalent in role: Verilator and cocotb are simulation/cosim frameworks; SymbiYosys is a formal-verification flow; `riscv-formal` is an ISA-level formal property set; `riscv-dv` is a stimulus generator; RISCV-ISAC is a coverage tool; commercial emulators are hardware-assisted verification platforms distinct from FPGA prototyping. The choice within each role is independent.
|
||||
- **Reference model choice.** Spike (C++, instruction-accurate) vs. Sail (first-order imperative ISA-specification language with a RISC-V model) vs. a custom golden model. Spike is the most common reference; the project must document any corner-case behaviors of the chosen reference against the RISC-V specification.
|
||||
- **Coverage database.** Coverage data are stored in vendor-specific formats (e.g., Synopsys UCDB / DDC, Cadence IMC, Siemens UCDB) and may be aggregated into a custom backend (e.g., SQLite) for CI. The format is a tool-vendor concern; the aggregation backend is a CI concern. These are distinct choices.
|
||||
- **Test categorization.** Tests must be tagged by domain (base integer, M-ext, A-ext, F/D-ext, privileged, vector, …), by intent (sanity, corner, regression, performance), and by status (passing, known-fail, expected-fail).
|
||||
- **Seed management.** Random seeds should be reproducible. Seed pools should be partitioned into "core regressions", "block regressions", and "fuzzing".
|
||||
- **Bring-up sequencing.** Block-level → core-level → multicore → system. The core testbench is the transition point between block-level and system-level work.
|
||||
|
||||
## Sign-Off Criteria
|
||||
|
||||
A core verification sign-off is the point at which the methodology exits the coverage-closure funnel. The following are candidate criteria; numeric thresholds are starting examples to be refined in the verification plan and are not validated targets. The relationship to DRAFT status is explicit: the numbers below are illustrative starting points, not sign-off-ready thresholds.
|
||||
|
||||
- **Functional coverage.** All defined cover points hit, with cross-coverage exercised. Threshold starting example: ≥ 95% cover-point hit, ≥ 90% cross-coverage hit; remaining items must be justified as unreachable or formally proven. Final threshold to be set in the verification plan.
|
||||
- **Code coverage.** Statement, branch, expression, toggle, and FSM coverage reported; excluded regions justified. Threshold starting example: ≥ 95% statement, ≥ 90% branch, with manual exclusion of reset/DFT/clock-gating logic. Final threshold to be set in the verification plan.
|
||||
- **Assertion coverage.** All SVA / immediate assertions vacuously satisfied or actively proven; assertion cover points hit. Open assertions must be reviewed and dispositioned.
|
||||
- **Formal closure.** Per-block formal sign-off, with proof closure recorded for each property. Inconclusive results must be either strengthened (via abstraction / decomposition) or accepted with risk justification; "inconclusive" is not equivalent to "proven".
|
||||
- **ISA compliance evidence.** All applicable tests in `riscv-arch-test` (the architectural test framework) and `riscv-tests` (the legacy compliance suite) pass. Signed certificate or equivalent record retained.
|
||||
- **Mutation testing.** Periodic injection of known-defect mutations into a frozen RTL revision; mutation-kill rate as a starting example: ≥ 80% of injected mutations detected by the testbench + assertions + formal. This number is a starting point and requires empirical validation on the XH-1 DUT. Equivalent-mutant handling (mutations that produce behaviorally identical RTL) must be defined; kill-rate targets are noisy and must be interpreted with caution. Final threshold and operator set to be set in the verification plan.
|
||||
- **Reference model validation.** Spike or Sail validated against the chosen RISC-V compliance tests before being used as a checker; validation evidence archived.
|
||||
- **Testbench self-check verification.** Every scoreboard check has at least one negative test that is expected to fail.
|
||||
- **Reproducibility.** Every sign-off run is reproducible from committed source, seed, and tool version.
|
||||
- **CDC/RDC closure.** CDC and RDC analysis clean for all core-level crossing structures.
|
||||
- **Lint clean.** Lint clean at the policy-defined severity level.
|
||||
- **Reset verification.** All reset domains and reset sequences verified; reset recovery verified.
|
||||
|
||||
## Verification Considerations
|
||||
|
||||
Core verification is itself subject to verification — the verification of the verification environment:
|
||||
|
||||
- **Testbench self-checking.** All scoreboard checks must be end-of-test verifiable; every check should be paired with at least one negative test that is expected to fail.
|
||||
- **Reference model validation.** The reference model (Spike, Sail) must be validated against the chosen RISC-V compliance tests before being used as a checker.
|
||||
- **Coverage sanity.** Cover points must be reviewed to ensure they are not trivially self-firing.
|
||||
- **Mutation testing.** A subset of RTL defects should be intentionally injected to confirm the testbench catches them. See `research/10-verification/fault-injection.md`. Equivalent-mutant handling must be defined.
|
||||
- **Coverage baseline stability.** Coverage baselines drift as RTL changes; a baseline-management policy (when a baseline is re-established, by whom, with what approval) is required.
|
||||
- **Tool qualification.** If the verification tools are used in any safety-relevant context, tool qualification (per ISO 26262, IEC 61508, DO-254, or similar) is required. The applicability of these standards to XH-1 is OPEN QUESTION 8. Tool qualification is a verification-of-verification concern, not only a sign-off checkbox.
|
||||
- **Version control and baselines.** Every test run must be reproducible from committed source. Coverage baselines must be versioned.
|
||||
- **Testbench verification at 128-instance scale.** The testbench itself has bugs. Verification of the testbench (negative tests, mutation of the testbench, equivalence of testbench revisions) is part of the methodology. At 128 instances, the testbench's own correctness becomes a non-trivial verification problem and is acknowledged here as a multicore-layer concern that the core testbench interface must support.
|
||||
|
||||
## Software Considerations
|
||||
|
||||
- **Toolchain integration.** The core verification environment must integrate with the software toolchain (compiler, assembler, linker, Spike, debug server) to enable co-simulation and to run real programs.
|
||||
- **Firmware.** Firmware that runs on the core (boot code, exception handlers, interrupt dispatchers) is itself verification target; the core testbench should support loading and running firmware images.
|
||||
- **Operating system implications.** A core that supports an OS (Linux, RTOS) must be verified with syscall-heavy workloads. The core testbench should provide OS-aware run modes.
|
||||
- **Debug and trace protocols.** The core's debug-transport module (per the RISC-V Debug specification) must be exercised. JTAG or cJTAG interactions with the core's debug module are typically a core-level concern.
|
||||
- **Performance analysis tools.** Tools that analyze trace data, performance counters, and profile information should be able to consume the artifacts the core testbench produces.
|
||||
- **Software developer ergonomics.** The verification team's interfaces (test writers, coverage reviewers) should be ergonomic. A "developer can run a single test and see results in under a minute" goal is a useful operational target.
|
||||
|
||||
## Security Considerations (Out of Scope, but Flagged)
|
||||
|
||||
**ASSUMPTION (out-of-scope for this revision):** Transient-execution / side-channel verification (e.g., Spectre-class, Meltdown-class, and constant-time verification) is treated as out of scope for the XH-1 core verification layer for this revision. For a 128-core design with privilege boundaries, this is a real and high-risk concern. This is flagged as a high-risk gap and as a candidate for a follow-up document. The current methodology is sufficient for functional and microarchitectural correctness but does not, by itself, provide security assurance.
|
||||
|
||||
## Proposed Direction
|
||||
|
||||
PROPOSAL — A **hybrid methodology** for XH-1 core verification, with the following structure:
|
||||
|
||||
1. **Dynamic simulation** as the primary verification engine, with the framework (UVM vs. lightweight) gated on OPEN QUESTION 3 (tool posture), OPEN QUESTION 4 (team size and timeline), and the design-language choice (OPEN QUESTION 11).
|
||||
- If UVM: pair with `riscv-dv` as the instruction-stream generator and Spike or Sail as the reference model.
|
||||
- If lightweight: pair `riscv-dv` (or a custom generator) with cocotb / Verilator and Spike.
|
||||
2. **Formal verification** applied to control-dominated blocks: fetch, decode, CSR file, MMU/TLB, exception/interrupt delivery, and the debug module. Coordinate with `research/10-verification/formal-verification.md` for tool selection. Apply `riscv-formal` as an ISA-level formal layer where the open-source flow is in scope. Sequential equivalence checking between major RTL revisions is a complementary activity.
|
||||
3. **Co-simulation against a RISC-V reference model** (Spike or Sail) for ISA-compliance evidence. Coordinate with `research/10-verification/isa-compliance.md`. Pin the RISC-V ISA, Privileged ISA, and Debug spec versions. At 128 instances, use a shared reference model with a deferred-check queue or sampled synchronization.
|
||||
4. **Directed tests** for XH-1 microarchitectural corner cases that random cannot reach economically. The corner cases to target are conditional on the in-order / out-of-order answer in OPEN QUESTION 2.
|
||||
5. **FPGA prototyping** for throughput-driven verification and as a substrate for fault injection and multicore bring-up. Commercial emulation is a conditional alternative based on licensing posture (OPEN QUESTION 3) and workload.
|
||||
6. **Regression infrastructure** with version-controlled seeds, baselines, and coverage data. Coordinate with `research/10-verification/regression.md`.
|
||||
7. **Coverage closure** as an explicit, tracked funnel with the sign-off criteria in the Sign-Off Criteria section, including baseline-management policy. Numeric thresholds are deferred to the verification plan and the numbers in the Sign-Off section are starting examples, not validated targets.
|
||||
8. **Reusable testbench architecture** so the core testbench composes inside the multicore testbench without modification, supporting both the per-die cost argument and the sampling strategy. The interface must support shared reference models, summary-only trace mode, and per-instance coverage identification.
|
||||
9. **Mutation testing** as a periodic check that the testbench is still doing useful work, with explicit equivalent-mutant handling. Coordinate with `research/10-verification/fault-injection.md`.
|
||||
10. **Standard core-verification adjuncts:** lint, CDC/RDC analysis, X-propagation handling, reset verification, and (conditional on OPEN QUESTION 7) UPF/CPF power-intent integration.
|
||||
|
||||
The hybrid is proposed; the specific framework within the hybrid (UVM vs. lightweight) is gated on OPEN QUESTION 3, OPEN QUESTION 4, and OPEN QUESTION 11, and the power-feature sub-portion is gated on OPEN QUESTION 7. The implementation-class sub-portion (DFT/ATPG scope, post-silicon layer) is gated on OPEN QUESTION 12. The document does not pick between alternatives that the OPEN QUESTIONS do not yet support.
|
||||
|
||||
## Confidence
|
||||
|
||||
- **High confidence:** The general methodology composition (dynamic + formal + co-simulation + directed + prototype + regression + coverage closure, with the standard adjuncts: lint, CDC/RDC, X-propagation, reset) corresponds to the IEEE 1800.2 UVM standard, the IEEE 1800 SystemVerilog assertion subset, and the RISC-V International published ISA / Privileged / Debug specifications, and is broadly accepted in industrial practice. The references supporting each component are cited at the point of use.
|
||||
- **High confidence (conditional):** The 128-core scaling argument (formal has near-zero per-instance marginal cost under bit-identical replication homogeneity; dynamic at full coverage has per-instance cost) is a direct consequence of the replication-homogeneity assumption and does not require empirical data. The parameter-set caveat (re-elaboration for differing parameters) is acknowledged.
|
||||
- **Medium confidence:** Specific tool selections, sign-off numeric thresholds, and area/power tradeoffs, given the absence of repository evidence on the OPEN QUESTIONS.
|
||||
- **Low confidence (OPEN):** XH-1's tool licensing posture, core microarchitecture details, ISA profile selection, team size, design language, power architecture, and target implementation class. These drive the specifics within the proposed hybrid.
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. What RISC-V profile does the XH-1 core implement? Candidate RISC-V profiles are defined against a base XLEN: e.g., RVA23U64 / RVA23S64 are 64-bit application / supervisor profiles, while a 32-bit core would follow a different profile track (e.g., RVA22U32). The reference model (Spike variant, Sail configuration) and the compliance test scope follow from this choice.
|
||||
2. Is the XH-1 core in-order or out-of-order? The verification plan complexity differs substantially. The instructions explicitly disallow assuming either without repository evidence.
|
||||
3. What is XH-1's toolchain posture (open-source only, commercial EDA, mixed)?
|
||||
4. What is the team size and timeline? Methodology cost scales with team capacity.
|
||||
5. What is the core's microarchitectural contract — what blocks are in scope for unit verification, and what is the boundary between core and multicore? See `research/10-verification/unit-verification.md` and `research/10-verification/multicore-verification.md`.
|
||||
6. Are the 128 cores architecturally identical, or are there variants (big/little, mixed application/accelerator, per-core ISA-subset configuration)? Heterogeneity changes the testbench architecture and invalidates the per-die cost-leverage argument.
|
||||
7. What is the core's power architecture — is clock gating, power gating, retention, or DVFS at the core level, and what are the state-retention semantics?
|
||||
8. What is XH-1's relationship to functional safety standards (ISO 26262, IEC 61508, DO-254, or none)?
|
||||
9. How will the XH-1 core's trace, debug, and performance-counter interfaces be designed, and how will core verification consume them?
|
||||
10. What is the expected defect rate target at tape-out, and how does the verification program demonstrate closure against that target?
|
||||
11. What is the XH-1 design language (SystemVerilog, Verilog-2001, Chisel/Spinal HDL, Bluespec), and does verification target the HDL source level, the generated-RTL level, or both?
|
||||
12. What is XH-1's target implementation class — ASIC tape-out, FPGA-only deployment, or both? This determines DFT/ATPG scope, post-silicon validation depth, and sign-off criteria.
|
||||
13. What physical implementation tool flow (synthesis, PnR, timing) is envisioned, and what are the timing-closure constraints (target clock frequency, slack budget)?
|
||||
|
||||
## Sources
|
||||
|
||||
- IEEE 1800.2-2020, *Standard for Universal Verification Methodology Based on Accellera SystemVerilog 3.1*. Defines UVM.
|
||||
- IEEE 1800-2017, *Standard for SystemVerilog — Unified Hardware Design, Specification, and Verification Language*. Defines SVA and immediate assertions.
|
||||
- RISC-V International, *The RISC-V Instruction Set Manual, Volume I: Unprivileged Architecture* (versioned).
|
||||
- RISC-V International, *The RISC-V Instruction Set Manual, Volume II: Privileged Architecture* (versioned).
|
||||
- RISC-V International, *RISC-V External Debug Support* (versioned).
|
||||
- RISC-V International, *RISC-V Profiles* (versioned, e.g., RVA23).
|
||||
- Symbiotic EDA / YosysHQ, *SymbiYosys* documentation (open-source formal verification flow).
|
||||
- `riscv-formal` framework (Verilog, BSD-licensed), maintained at github.com/YosysHQ/riscv-formal.
|
||||
- `riscv-dv` instruction-stream generator, maintained at github.com/google/riscv-dv.
|
||||
- `riscv-tests` and `riscv-arch-test`, maintained by RISC-V International.
|
||||
- Spike (the official RISC-V ISA simulator), maintained at github.com/riscv-software-src/riscv-isa-sim.
|
||||
- Sail RISC-V formal specification, maintained at github.com/rems-project/sail. Sail is a first-order imperative ISA-specification language; the RISC-V model is generated from the RISC-V ISA formal definition.
|
||||
- Kami project (Cambridge), formal RISC-V ISA specification in Coq with a Bluespec front-end. Project publications and documentation should be cited from the project's own canonical sources.
|
||||
|
||||
Quantitative throughput, area, power, and timing claims are not asserted in the body. The XH-1 repository documents referenced (under `research/10-verification/`) are placeholders ("SOON") and do not provide factual content to cite. No measurements, benchmarks, processor capabilities, URLs outside the repositories named above, or experimental results have been invented.
|
||||
+285
File diff suppressed because one or more lines are too long
+385
@@ -0,0 +1,385 @@
|
||||
# Core Verification
|
||||
|
||||
## Status
|
||||
|
||||
DRAFT — Research framework established; all claims are explicitly classified by epistemic status (FACT with citation, ASSUMPTION, PROPOSAL, RECOMMENDATION, OPEN QUESTION, or INSUFFICIENT EVIDENCE). Concrete sign-off criteria, tool selection, and quantitative thresholds are deferred until the OPEN QUESTION inputs are resolved. The numeric thresholds appearing in the Sign-Off Criteria section are illustrative starting points for the verification plan and are not validated targets. Every threshold below is restated with this caveat at the point of use so that the sign-off list is internally consistent with this status.
|
||||
|
||||
## Abstract
|
||||
|
||||
This document investigates verification strategies for the XH-1 core, defined here as a single processing core instance within a custom 128-core RISC-V processor. Core verification is the layer of the verification methodology stack concerned with demonstrating that one core conforms to its architectural specification, microarchitectural contract, and integration assumptions prior to and independent of multi-core interactions. For a 128-core design, the core is the unit of replication; defects undetected at this layer replicate 128-fold across the die, amplifying area, power, yield, and debug cost. This document frames the research questions, surveys existing approaches (simulation, formal verification, hardware-assisted verification, FPGA prototyping, emulation), analyzes their applicability to a custom RISC-V core, and identifies the gaps that the XH-1 project must close given that the existing related documents in `research/10-verification/` are placeholders ("SOON").
|
||||
|
||||
## Research Question
|
||||
|
||||
What verification methodology, toolchain, environment abstraction, coverage model, and sign-off criteria are appropriate for verifying a single XH-1 core such that the methodology:
|
||||
|
||||
1. Provides evidence that the core implements the RISC-V ISA subset selected for XH-1 (see `research/10-verification/isa-compliance.md`).
|
||||
2. Provides evidence that the core's microarchitecture meets timing, area, and power targets for a design that will be replicated 128 times.
|
||||
3. Scales economically — verification effort, compute cost, and human review time must not grow linearly with the 128-core replication factor.
|
||||
4. Produces artifacts (testbenches, coverage models, formal proofs, regression suites) that compose cleanly with the surrounding verification layers (unit, multicore, system, post-silicon).
|
||||
5. Is implementable given that the related XH-1 documents in area 10 are not yet populated, indicating a greenfield verification program.
|
||||
|
||||
## Background
|
||||
|
||||
Verification of a processor core is conventionally decomposed into:
|
||||
|
||||
- **Unit verification** — verification of individual blocks (ALU, FPU, register file, branch predictor, fetch/decode structures, load-store unit, replay logic, CSR file, MMU/TLB, caches). See `research/10-verification/unit-verification.md`.
|
||||
- **Core verification** — verification of the integrated core as a single design under test (DUT), exercising instruction-stream and exception behavior at the architectural interface, plus internal microarchitectural invariants.
|
||||
- **Multicore verification** — coherence, consistency, interconnect, and shared-resource correctness across multiple core instances. See `research/10-verification/multicore-verification.md`.
|
||||
|
||||
For XH-1, the core is the replication unit. The cost model must distinguish two quantities that this document often refers to together as "cost":
|
||||
|
||||
- **Defect-escape cost (multiplied by replication).** The downstream cost of an undetected defect (silicon debug, re-spin, field failure, yield loss). Under the replication-homogeneity assumption, an undetected core defect is replicated 128-fold in silicon, multiplying its downstream impact.
|
||||
- **Verification effort cost (paid once).** The engineering cost of running the testbench, discharging formal properties, maintaining the regression, and writing tests. This is paid against one RTL; the fix, once made, is replicated.
|
||||
|
||||
The phrase "every bug caught at the core layer is caught 128 times" is shorthand for the first quantity: every defect that escapes core verification multiplies its downstream cost by the replication count. It is not a claim that the verification effort itself runs 128 times — the testbench runs against the RTL once.
|
||||
|
||||
**ASSUMPTION (replication homogeneity):** The 128 XH-1 cores are architecturally identical instances of a single RTL core produced by elaboration of the same source. Heterogeneous cores, per-core configuration, or runtime variants invalidate the replication-leverage argument on defect-escape cost and require per-variant verification. This is treated as an assumption because no repository evidence contradicts it; OPEN QUESTION 6 below revisits it. The "per-instance marginal cost of zero" claim for proven formal properties holds for bit-identical replicated RTL and refers to the marginal cost of *re-running* a proof against another elaborated instance of the same source — not to the engineering cost of constructing the proof. If the source is parameterized and elaborated with per-instance parameters that differ, the proof must be re-established for any parameter set that differs, and that re-establishment cost is not zero.
|
||||
|
||||
**ASSUMPTION (target implementation class):** XH-1 targets ASIC tape-out. FPGA-only deployment would change the methodology (no DFT/ATPG, different sign-off criteria, no post-silicon bring-up in the same sense). This assumption is conditional and is revisited in OPEN QUESTION 12.
|
||||
|
||||
**ASSUMPTION (verification language availability):** The verification language of the testbench is the same as or interoperable with the design language. This is revisited in OPEN QUESTION 11.
|
||||
|
||||
Core verification draws on several formal and semi-formal techniques:
|
||||
|
||||
- **Dynamic simulation** with constrained-random or directed tests, against an ISA reference model, with functional coverage.
|
||||
- **Static (formal) verification** — model checking (bounded model checking, k-induction, IC3/PDR), equivalence checking (combinational and sequential), and property proving against the RTL.
|
||||
- **Hardware-assisted verification** — FPGA prototyping and emulation (commercial emulators, open-source flows such as FireSim on Amazon EC2 F1), trading compile time for runtime speed.
|
||||
- **Co-simulation** — two simulation engines running in lockstep (e.g., an RTL simulator and an architectural reference-model engine such as Spike) synchronized at defined events. This is distinct from **reference-model checking**, in which the reference model is integrated as a transactor within a single simulator; the latter is a common industrial pattern and is more accurately described as reference-model checking, not co-simulation in the strict sense.
|
||||
- **Waveform and assertion-based methodologies** — SystemVerilog Assertions (SVA), PSL, custom monitors, end-of-test checkers.
|
||||
|
||||
Coverage is a central concern. Functional coverage, code coverage (line, branch, expression, toggle, FSM), and cross-coverage of architectural state are the standard dimensions. For a custom ISA implementation, the relationship between ISA compliance coverage and microarchitectural coverage must be defined explicitly.
|
||||
|
||||
Standard core-verification concerns that must be addressed in addition to instruction-stream behavior:
|
||||
|
||||
- **Lint** — static analysis of RTL style, common bug patterns, and synthesizability (e.g., Verilator lint, commercial lint tools). A prerequisite to dynamic and formal work.
|
||||
- **Clock-domain crossing (CDC) and reset-domain crossing (RDC)** — analysis of signals crossing asynchronous clock or reset domains. Typically owned by a separate methodology but the core must be CDC/RDC-clean for integration.
|
||||
- **X-propagation** — controlled handling of unknown ('X) propagation in simulation, using X-pessimism removal or X-init optimization, to avoid false coverage from masked X's.
|
||||
- **Reset verification** — power-on reset, software reset, debug reset, partial reset, and reset-domain interaction.
|
||||
- **Power intent** — if power features are present (see OPEN QUESTION 7), UPF/CPF power intent formats integrate with simulation to model power-gated, retention, and isolation behavior.
|
||||
- **DFT/ATPG interaction** — scan, BIST, and MBIST structures must not alter mission-mode behavior; their interaction with the core testbench is a coordination point.
|
||||
|
||||
These are not optional; their absence in a verification plan is a known gap in pre-silicon methodology.
|
||||
|
||||
## Existing Approaches
|
||||
|
||||
The following approaches are established in industry and academic practice. Their applicability to XH-1 is analyzed; none is recommended in isolation.
|
||||
|
||||
### A. Constrained-Random Simulation with UVM
|
||||
|
||||
**FACT:** Universal Verification Methodology (UVM) is standardized as IEEE 1800.2-2017, *Standard for Universal Verification Methodology* [IEEE 1800.2-2017], and is widely used in industrial block- and core-level processor verification for constrained-random stimulus generation, reference model integration, self-checking scoreboards, functional coverage, and phasing. The Accellera UVM reference implementation, which is the source code that vendors and users actually consume, is distributed under the Apache License 2.0; the IEEE 1800.2-2017 standard document itself is an IEEE publication and is not freely redistributable. The two are distinct artifacts and this distinction is load-bearing when characterizing "open-source viability" below.
|
||||
|
||||
**ASSUMPTION (conditional on design language):** UVM is applicable only if the XH-1 design language supports SystemVerilog class constructs — either the source is SystemVerilog, or the verification environment targets SystemVerilog regardless of source language. If the design language is Verilog-2001 only, or if the project commits to a Chisel/Spinal HDL or Bluespec-only flow, UVM is not directly applicable at the HDL-source level and an alternative testbench harness is required. This assumption is revisited in OPEN QUESTION 11.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Apply UVM as the primary dynamic verification environment for XH-1 core-level testing, paired with a chosen ISA reference (Spike or Sail). Sequence generation over the RISC-V ISA state space; reference model in co-simulation; scoreboard for architectural state comparison per committed instruction; explicit cover points for instruction × operand-mode × privilege × exception interactions.
|
||||
|
||||
### B. Directed and Hand-Written Tests
|
||||
|
||||
**FACT:** Constrained-random alone does not reach corner-case coverage of complex pipelines, corner-case CSR interactions, or rare exception sequences. Industry practice supplements with directed tests written by architects and designers.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** A XH-1-specific directed test suite for pipeline hazard combinations specific to the XH-1 microarchitecture, corner cases in the chosen RVA profile subset, reset, wake, low-power entry/exit, and debug transport (if present at core level).
|
||||
|
||||
### C. Formal Property Verification
|
||||
|
||||
**FACT:** Bounded model checking, k-induction, and IC3/PDR are applied to control-dominated blocks: fetch, decode, CSR, MMU/TLB, branch predictor, retirement, and exception delivery. Commercial model checkers include Cadence JasperGold, Synopsys VC Formal, and Siemens FormalPro. Open-source flows include SymbiYosys (SBY), based on the Yosys synthesis and formal toolchain. Equivalence checking is a distinct formal category: tools include Synopsys Formality and Cadence Conformal, used to verify that two RTL representations (e.g., pre- and post-synthesis, or pre- and post-retiming) are functionally equivalent. Equivalence checkers are not model checkers and are not used for property proving.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Per-block formal sign-off for the control-dominated blocks listed above, end-to-end pipeline flushing, exception priority, and CSR read/write semantics. Formal proofs of privileged ISA semantics (e.g., `mstatus`, `sstatus`, `satp`, `mip`/`sip` behavior) provide a high-confidence anchor that random tests cannot match in isolation. Sequential equivalence checking between major RTL revisions is a separate, complementary activity.
|
||||
|
||||
**ASSUMPTION (RISC-V privileged-ISA versioning):** The CSR semantics used as formal-proof targets correspond to the specific RISC-V Privileged ISA version adopted by XH-1. The unprivileged ISA, privileged ISA, and Debug specification each have versioned releases tracked by RISC-V International; the verification plan must be pinned to those versions. The specific version is OPEN until resolved (see OPEN QUESTION 1 and the Sources section).
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Coordinate with `research/10-verification/formal-verification.md` for tool selection (commercial model checkers vs. SymbiYosys / SBY) and with `research/10-verification/isa-compliance.md` for the privileged-ISA version.
|
||||
|
||||
### D. Reference-Model Checking and Co-Simulation
|
||||
|
||||
**FACT:** Comparing DUT architectural state against Spike (the official RISC-V ISA simulator, C++ implementation) or Sail (a first-order imperative language designed for ISA specification, with a RISC-V model generated from the RISC-V ISA formal definition) per instruction is a widely deployed pattern. The terminology matters:
|
||||
|
||||
- **Co-simulation (strict sense):** two independent simulation engines (e.g., an RTL simulator and Spike) running in lockstep, synchronized at defined events. The DUT may run ahead of the reference using a deferred-check queue, with results compared at synchronization points.
|
||||
- **Reference-model checking:** a single simulator that incorporates the reference model as a transactor or foreign-language call. This is a common industrial pattern but is not co-simulation in the strict sense.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Required for ISA-compliance evidence. See `research/10-verification/isa-compliance.md`. The specific architecture proposed is **reference-model checking within a single simulator** (Spike invoked as a transactor from the UVM or lightweight harness) as the default for day-to-day runs, with **co-simulation in the strict sense** retained as a cross-check on a sampled subset of tests to detect synchronization artifacts that a single-simulator harness could mask. Reference model parallelism is a 128-core scaling concern: Spike and Sail are single-threaded by default; running 128 reference models in lockstep with 128 RTL instances is itself a significant engineering problem. The pattern of choice is a single shared reference model servicing all DUT instances through a deferred-check queue, or sampling (multicore-layer decision) with the core testbench supporting the interface.
|
||||
|
||||
### E. FPGA Prototyping and Emulation
|
||||
|
||||
**FACT:** FPGA-based prototyping runs the RTL at higher throughput than software simulation, and emulation platforms (commercial: Palladium, Veloce, ZeBu; open-source: FireSim on Amazon EC2 F1) provide a middle ground. The throughput multiplier relative to software simulation is workload- and instrumentation-dependent.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Use FPGA prototyping to speed up core testbench bring-up by running entire test programs end-to-end, to enable fault-injection campaigns that are infeasible in simulation (see `research/10-verification/fault-injection.md`), and as the natural place to begin validating the multicore layer; core verification is the upstream prerequisite. Commercial emulation is a viable alternative for very large workloads or when observability requirements are relaxed; the decision depends on licensing posture (OPEN QUESTION 3) and workload characteristics. No specific throughput multiplier for the XH-1 DUT is asserted here; INSUFFICIENT EVIDENCE without XH-1-specific RTL and platform data.
|
||||
|
||||
### F. Regression Infrastructure
|
||||
|
||||
**FACT:** Regression management — version control of testbenches, seeds, baselines, coverage databases — is a load-bearing component of any non-trivial core verification program. See `research/10-verification/regression.md`.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** The XH-1 regression infrastructure must support a single-core nightly run, a multicore run, and a system run with shared seeds where appropriate.
|
||||
|
||||
### G. Coverage Closure Methodology
|
||||
|
||||
**FACT:** Coverage closure combines functional coverage (defined by the verification plan), code coverage (uncovered RTL), and assertion coverage. Industry practice treats coverage closure as a managed funnel: cover points → cross coverage → corner case identification → directed/random test addition → formal where possible. Coverage baselines drift as RTL changes; a baseline-management policy is required.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Codify the funnel explicitly in the verification plan with tracked stages, baseline-management policy, and exit criteria per stage.
|
||||
|
||||
### H. Assertion-Based Verification
|
||||
|
||||
**FACT:** SystemVerilog Assertions (IEEE 1800-2017, *Standard for SystemVerilog*) and immediate assertions encode design intent and serve both as live checkers and as inputs to formal tools.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Define an SVA library for XH-1 invariants (e.g., "no retired instruction has a privileged side effect inconsistent with its mode", "no speculative architectural state is observable at the architectural interface").
|
||||
|
||||
### I. Open-Source ISA-Level Formal Verification
|
||||
|
||||
**FACT:** The `riscv-formal` framework (YosysHQ / Symbiotic EDA, Verilog, BSD-licensed) provides a checkable RISC-V ISA specification encoded as a set of formal properties that are conjoined with a wrapper around the DUT and discharged by an SMT solver. The Kami project (Cambridge) provides a formal RISC-V ISA specification in Coq with a Bluespec front-end. INSUFFICIENT EVIDENCE to assert at the FACT level that `riscv-formal` is "directly applicable" to a specific XH-1 microarchitecture and ISA profile without further evaluation: applicability depends on the DUT's memory and CSR interface matching the framework's wrapper assumptions, on the chosen RISC-V profile, and on the unprivileged/privileged ISA versions adopted.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Evaluate `riscv-formal` as a complement to block-level formal. It targets ISA-compliance evidence with open-source tooling and is particularly relevant given the greenfield status of the XH-1 verification program, subject to the applicability evaluation above.
|
||||
|
||||
### J. Open-Source Constrained-Random Generator
|
||||
|
||||
**FACT:** Google's `riscv-dv` is an open-source UVM-based constrained-random instruction-stream generator for RISC-V, widely used in industrial and academic flows. The license should be confirmed against the current repository state at the time of toolchain selection; this document does not assert a specific license version.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Evaluate `riscv-dv` as a generator either within a UVM environment or as a stimulus source for a lightweight harness.
|
||||
|
||||
### K. RISC-V Architectural Test Framework
|
||||
|
||||
**FACT:** RISC-V International maintains `riscv-arch-test` as the architectural test framework, which is the conformance test suite against which cores are signed off for profile compliance. `riscv-tests` is a related umbrella repository containing `riscv-arch-test` as well as additional tests; the relationship is not "legacy vs. replacement" — `riscv-tests` is the broader collection and `riscv-arch-test` is the architectural conformance subset. The two should be treated as distinct but related inputs, both of which produce evidence in the sign-off record. The framework's coverage analyzer, `RISCV-ISAC`, is an instruction-stream coverage analyzer used to confirm that the architectural test signatures cover the required instruction/operand/privilege combinations; it is not a generic functional-coverage database and is not a substitute for the core testbench's own coverage model.
|
||||
|
||||
**PROPOSAL (applicability to XH-1):** Use `riscv-arch-test` for the ISA-compliance certificate, with `RISCV-ISAC` analysis of the architectural test signatures as part of the coverage evidence, and `riscv-tests` more broadly as a regression input. The license and current revision of each must be confirmed at toolchain selection.
|
||||
|
||||
## Alternative Designs
|
||||
|
||||
The following alternative *strategies* for the core verification methodology itself are candidates for XH-1. They are not mutually exclusive; the project must select a composition.
|
||||
|
||||
### Alternative 1: UVM-Centric with Formal Supplements
|
||||
|
||||
- **Approach:** UVM as the workhorse for dynamic verification, with formal applied to control-dominated blocks.
|
||||
- **Strengths:** Industry-standard methodology, well-understood coverage and sign-off conventions, mature ecosystem; integrates with `riscv-dv` as a generator and `riscv-formal` as an ISA-level formal layer.
|
||||
- **Weaknesses:** UVM has a steep learning curve and high setup cost. For a 128-core design, the UVM environment must be designed for reuse to avoid 128× rewrite cost. Requires SystemVerilog (OPEN QUESTION 11). Implies commercial simulator licensing in most deployments, because open-source SystemVerilog simulation support (Verilator) has limited UVM-class support relative to commercial offerings; the open-source viability of this alternative is therefore medium, not high.
|
||||
|
||||
### Alternative 2: Lightweight Python/C++ Test Harness with Reference-Model Checking
|
||||
|
||||
- **Approach:** A generator produces instruction streams and a C++ testbench (e.g., based on cocotb, Verilator, or a custom framework) compares architectural state against Spike.
|
||||
- **Strengths:** Lower setup cost than UVM. Excellent generator productivity. Easy integration with regression infrastructure. Open-source-friendly. `riscv-dv` can serve as the generator. Verilator-based flows are not bound to SystemVerilog UVM at the source level.
|
||||
- **Weaknesses:** Verilator's debug, SVA, and functional-coverage support are not equivalent to commercial simulators; cocotb+Verilator is therefore not a drop-in replacement for UVM, and the trade-off should be evaluated against the XH-1 debug and coverage requirements. Coverage closure tooling is less standardized than the UVM vendor ecosystem. The reference-model-checking pattern (Section D) is the natural fit for this harness.
|
||||
- **Application to XH-1:** Reasonable choice if XH-1 has a small team, an open-source toolchain preference, or a constraint on commercial EDA licensing.
|
||||
|
||||
### Alternative 3: Formal-First
|
||||
|
||||
- **Approach:** Formal proofs form the primary verification evidence; dynamic simulation is used for reach and to validate properties that formal cannot discharge.
|
||||
- **Strengths:** Highest assurance per proven property. Strong fit for control logic and CSR behavior. `riscv-formal` provides an open-source ISA-level formal layer subject to the applicability evaluation in Section I. Under the replication-homogeneity assumption, proven properties transfer to all 128 instances at near-zero marginal *re-proof* cost (the proof is re-run, not re-constructed, against another elaborated instance of the same RTL); the engineering cost of constructing the proof is paid once.
|
||||
- **Weaknesses:** Pipeline formalization is hard. State-space complexity grows combinatorially for large cores. May not reach proof closure on timing-agnostic models without abstraction. Inconclusive results are not equivalent to failure; risk-acceptance must be tracked.
|
||||
|
||||
### Alternative 4: Emulation-Centric
|
||||
|
||||
- **Approach:** Skip large-scale software simulation; use FPGA/emulation throughput as the primary engine.
|
||||
- **Strengths:** Excellent long-workload capability; native fault-injection support.
|
||||
- **Weaknesses:** Poor waveform/observability; slow compile/turnaround; harder to drive corner-case stimulus; debug cycle is slow.
|
||||
|
||||
### Alternative 5: Hybrid (PROPOSAL — See Proposed Direction)
|
||||
|
||||
A combination of a dynamic framework (Alternative 1 or Alternative 2) for stimulus and reach, formal supplements (Alternative 3, including `riscv-formal` where applicable) for control logic, and FPGA prototyping/emulation for throughput and multicore bring-up.
|
||||
|
||||
## Comparison
|
||||
|
||||
The comparison is qualitative and is intended to make the trade-offs explicit, not to produce a numeric ranking. INSUFFICIENT EVIDENCE exists to rank absolute quality without XH-1-specific RTL data, tool licensing posture, and team size. The Hybrid column is a composition of the others and is not an independent point of comparison; the table is structured to show how each pure strategy performs on the dimensions, with the Hybrid column indicating how the strengths compose.
|
||||
|
||||
| Dimension | UVM-Centric | Lightweight Harness | Formal-First | Emulation-Centric | Hybrid |
|
||||
|---|---|---|---|---|---|
|
||||
| Setup cost | High | Low | High | Medium | High |
|
||||
| Per-test runtime | Slow | Slow–Medium | N/A (proof time) | Fast | Mixed |
|
||||
| Corner-case reach | Strong (with seeds) | Strong (with fuzzing) | Strong (when discharged) | Weak–Medium | Strong |
|
||||
| Assurance on control logic | Medium | Medium | High | Low–Medium | High |
|
||||
| Debug ergonomics | Good (waveform, logs) | Good (caveat: Verilator/cocotb debug support not equivalent to commercial) | Strong counterexamples | Weak | Mixed |
|
||||
| Open-source viability | Medium (UVM standard is IEEE-published; UVM reference implementation is Apache 2.0; simulators are usually commercial) | Strong | Medium | Medium | Medium |
|
||||
| Fit for 128-core reuse | High (if designed so) | High (if designed so) | High (under replication homogeneity) | High | High |
|
||||
| Pre-tape-out risk reduction | Strong | Medium–Strong | Strong (partial; inconclusive ≠ proven) | Medium | Strong |
|
||||
|
||||
## Advantages
|
||||
|
||||
The advantages of investing in a robust core verification methodology for XH-1:
|
||||
|
||||
- **Replication leverage on defect-escape cost.** Under the replication-homogeneity assumption, an unfixed defect in the core RTL is replicated 128-fold in silicon, multiplying its downstream (silicon-debug, re-spin, field-failure, yield-loss) cost. Verification effort itself is paid once against one RTL; the defect, if found, is fixed once; the fixed RTL is then replicated. The benefit of core-level detection is the avoidance of multiplied post-silicon cost.
|
||||
- **Cleaner multicore verification.** A well-verified core reduces the multicore verification problem to coherence, consistency, interconnect, and shared resources. See `research/10-verification/multicore-verification.md`.
|
||||
- **Foundation for post-silicon.** First-stage silicon bring-up is dramatically simpler when the core is well-understood and the debug interfaces (assertions, traces, scan) are well-designed. Post-silicon validation feeds back into pre-silicon coverage targets; this feedback loop is not detailed in this document and is left to the post-silicon layer.
|
||||
- **Reusable testbench assets.** Generator, scoreboard, coverage model, and SVA library can be reused across cores if XH-1 has core variants, or across revisions of the same core.
|
||||
- **Formal evidence on critical control.** Privileged ISA semantics are hard to verify by random simulation; formal proofs on these blocks provide a high-confidence floor.
|
||||
|
||||
## Disadvantages
|
||||
|
||||
Costs and risks of a robust core verification program:
|
||||
|
||||
- **Engineering cost.** A UVM-class environment plus formal plus emulation is a substantial program. The claim that it "can easily exceed the RTL design cost for a single core" is folkloric and is presented here as a planning consideration, not as a measured fact.
|
||||
- **Tooling lock-in.** Heavy UVM usage often implies commercial simulator licensing; this may not match XH-1's tool posture. Formal tool lock-in is similar: commercial model checkers differ in property language and run scripts.
|
||||
- **Slow turnaround.** Constrained-random simulation with full coverage at the core level is slow; coverage closure can take many cycles of "add test, rerun, close hole". Coverage baselines drift with RTL changes, requiring re-establishment.
|
||||
- **Reference model dependence.** Spike, Sail, or any reference may diverge from the RISC-V specification in interpretation of ambiguous or profile-specific corners; the verification program must track the chosen RVA profile carefully and pin the spec version of the reference.
|
||||
- **Coverage false confidence.** Hitting a cover point is not the same as proving correctness; the verification plan must distinguish cover and check.
|
||||
- **128-core replication pressure.** If the core testbench is not designed to compose with the multicore testbench, redundant effort emerges.
|
||||
- **Inconclusive formal ≠ proven.** Formal results of "inconclusive" must be risk-accepted, not treated as success.
|
||||
- **Equivalent mutants.** Mutation-kill rates are affected by equivalent mutants (mutations that produce behaviorally identical RTL); kill-rate targets are therefore noisy and must be interpreted with caution.
|
||||
|
||||
## XH-1 Considerations
|
||||
|
||||
XH-1-specific factors that constrain the methodology choice:
|
||||
|
||||
- **128-core replication.** The verification plan must be replication-aware: the *testbench* should be agnostic to whether one or 128 cores are instantiated. This implies strong separation of (a) the core DUT, (b) the core testbench, and (c) the multicore wrapper.
|
||||
- **Custom design.** No third-party core is being verified; there is no vendor's pre-built verification IP. All environments must be built or assembled.
|
||||
- **Repository context.** The related verification documents are all placeholders ("SOON"). This is a greenfield verification program; the XH-1 project has the opportunity — and the burden — to define the methodology from first principles.
|
||||
- **RISC-V ISA.** The RISC-V ecosystem provides mature reference models (Spike, Sail), compliance / architectural test frameworks (`riscv-arch-test`, `riscv-tests`), and open-source verification tooling (`riscv-formal`, `riscv-dv`, `RISCV-ISAC`). These are the natural anchors.
|
||||
- **Tool posture (OPEN QUESTION 3).** Without evidence in the repository, the project's tool posture (open-source only, commercial EDA, mixed) is OPEN. The methodology composition must remain valid across posture choices.
|
||||
- **Design language (OPEN QUESTION 11).** The choice of design language — SystemVerilog, traditional Verilog, Chisel/Spinal HDL, Bluespec — has direct implications for verification methodology (e.g., formal property availability on generated RTL, simulation throughput, the place of unit-level testing at the HDL-source level vs. the generated-RTL level, and the applicability of UVM at the source level). This is OPEN pending XH-1 design-language disclosure. Section A's UVM assumption is conditional on this resolution.
|
||||
- **Microarchitectural class (OPEN QUESTION 2).** The pipeline class — in-order vs. out-of-order, superscalar width, depth of speculation, branch-prediction aggressiveness — materially changes the cost-benefit of formal vs. dynamic and the complexity of pipeline flush / replay verification. The methodology composition must accommodate both extremes.
|
||||
- **Power architecture (OPEN QUESTION 7).** Whether the core supports clock gating, power gating, retention, and/or DVFS is OPEN. The verification methodology must accommodate each combination. Assuming any specific power feature is invalid without evidence. Power intent (UPF/CPF) integration is conditional on this.
|
||||
- **Target implementation class (OPEN QUESTION 12).** Whether XH-1 targets ASIC tape-out or FPGA-only deployment fundamentally changes the verification methodology (DFT/ATPG scope, post-silicon layer, sign-off criteria). Treated here as an assumption (ASIC) pending resolution.
|
||||
|
||||
## 128-Core Scalability
|
||||
|
||||
Replication does not change which verification techniques apply to a single core, but it changes their cost-benefit and their composition. Throughout this section, the two cost quantities from the Background are kept distinct: **defect-escape cost** (multiplied by replication count under the homogeneity assumption) versus **verification effort cost** (paid against one RTL).
|
||||
|
||||
- **Core testbench scalability.** A core testbench written for a single instance must be instantiable inside a multicore wrapper without modification. This implies the testbench interfaces to the core through a standard transaction-level interface (or at least a clearly defined boundary), not through global signals.
|
||||
- **Cost-benefit of formal at replication.** Per-block formal verification has high fixed engineering cost (constructing the proof), but under bit-identical replication, the *re-proof* cost against another elaborated instance of the same RTL is near zero: once a control property is proven, re-running the proof against a re-elaborated instance of the same source is mechanical. This is a property of the proof, not of the underlying effort. If the source is parameterized and elaborated with per-instance parameters that differ, the proof must be re-established for any parameter set that differs. The defect-escape cost argument is the dominant benefit: a proven property that prevents a class of bug multiplies that benefit by the replication count.
|
||||
- **Cost-benefit of dynamic at replication.** Dynamic simulation at full coverage also has its verification effort paid against one RTL, but full per-instance coverage at 128 instances requires running 128 instances through the same stimulus, which is a real additional compute cost (not strictly linear in wall-clock because the simulator elaborates them in the same run, but real in host compute and memory). The trade-off is a primary argument for the hybrid approach at 128-core scale, and the practical mitigation is sampling, not full per-instance coverage.
|
||||
- **Workload scaling.** Running the full core testbench in a 128-core environment is generally unnecessary; sampling (verify a subset of cores under full stimulus, sweep register state across all 128) is a more efficient pattern. The sampling strategy is a multicore concern but must be supported by the core testbench's reset/configuration interface. The statistical argument for the chosen sampling fraction (defect-escape probability vs. simulation cost) is a multicore-layer deliverable; the core testbench must support the interface.
|
||||
- **Coverage aggregation.** Per-core coverage data must be aggregated across the 128 instances. The core testbench should emit coverage in a form that is identifiable by instance index.
|
||||
- **Debug bandwidth.** 128 cores each generating waveform/log data is impractical. The core testbench must distinguish between detailed-trace mode (one core, full visibility) and lightweight-trace mode (128 cores, summary only).
|
||||
- **Compile and elaboration cost.** Elaboration of a 128-core testbench can dominate runtime. The core testbench should be elaborable standalone and in a multicore wrapper without recompiling the core RTL. Elaboration-time behavior (reset, clock tree, memory subsystem) can differ between 1-core and 128-core contexts; the core testbench must be valid in both.
|
||||
- **Testbench memory footprint.** 128 instances of a complex scoreboard and 128 reference models can dominate host memory. Memory-aware testbench design (shared reference models, deferred-check queues, summary-only trace mode) is required. This is acknowledged here and detailed in the multicore layer.
|
||||
- **Reference-model parallelism.** Spike and Sail are single-threaded by default. Running 128 reference models in lockstep with 128 RTL instances is a significant engineering problem. The recommended pattern is a single shared reference model servicing all DUT instances through a deferred-check queue, or sampling. This is a multicore-layer design constraint that the core testbench interface must support.
|
||||
- **Symmetry assumption revisit.** The replication-homogeneity assumption is load-bearing throughout this section. Heterogeneous cores (e.g., big/little, mixed application/accelerator, per-core ISA-subset configuration) invalidate the per-die marginal-cost argument and require per-variant verification, including per-variant testbenches, per-variant coverage, and per-variant formal. The methodology should be designed so that the additional cost of a variant is linear in the number of variants, not linear in 128 × variants.
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
The following are framework-level considerations. INSUFFICIENT EVIDENCE exists to provide numerical throughput estimates without XH-1-specific RTL data, simulator choice, and toolchain selection; any specific number is unsourced and is therefore not asserted here.
|
||||
|
||||
- **Testbench throughput.** Dynamic simulation throughput determines how many random seeds can be run per unit wall-clock. Constrained-random at the core level is sensitive to simulator choice, RTL abstraction level, and coverage instrumentation. ASSUMPTION: representative numbers are available in vendor documentation (commercial simulators) and in the open-source Verilator and cocotb benchmark literature, but are workload-dependent and cannot be quoted as typical without measurement on the XH-1 DUT. No specific throughput range is asserted.
|
||||
- **Compiler/optimizer.** Coverage instrumentation slows simulation. Production coverage runs and fast-turnaround debug runs should be separable.
|
||||
- **Co-simulation overhead.** PROPOSAL: Spike co-simulation overhead per committed instruction is non-trivial in co-simulation in the strict sense (two-engine lockstep); under the reference-model-checking pattern adopted in Section D, the overhead is a foreign-language call per synchronization point. Synchronization strategy (per-instruction vs. batched) is a tradeoff between debug granularity and throughput. At 128 instances, the cost of a per-instruction synchronization across all instances is prohibitive; batched or sampled synchronization is the practical pattern.
|
||||
- **FPGA prototype throughput.** FPGA prototype throughput is set by Fmax, which is determined by LUT count, routing, and the memory subsystem. Quoted MHz figures for an XH-1 core on a specific FPGA platform are not asserted here; INSUFFICIENT EVIDENCE without platform and synthesis data.
|
||||
- **Regression wall-clock budget.** A nightly regression that runs N seeds on a single core must fit within the available compute budget. Cloud burst capacity is typical.
|
||||
|
||||
INSUFFICIENT EVIDENCE to provide numerical throughput estimates without XH-1-specific RTL data and toolchain selection.
|
||||
|
||||
## Area Considerations
|
||||
|
||||
- **On-die verification infrastructure.** XH-1 is described as a 128-core design. The verification *methodology* does not directly consume die area, but decisions made during verification (e.g., embedding assertions in RTL, adding observability muxes for trace, adding per-core DFT structures) do.
|
||||
- **Trace and debug infrastructure.** Core-level verification informs the design of trace, performance counters, and debug-transport blocks. Larger trace infrastructure costs area; the verification program must justify the cost in terms of debug productivity and post-silicon coverage.
|
||||
- **DFT (Design-for-Test) compatibility.** Scan chains, BIST, and MBIST are typically owned by a separate DFT program. Core verification should ensure that the core's DFT structures do not alter functional behavior in mission mode. If XH-1 is FPGA-only (OPEN QUESTION 12), DFT is out of scope.
|
||||
- **Replication-driven area optimization.** If the core's area is a major constraint (which is likely at 128 cores on a single die), the verification plan must protect against defects that would force a re-spin with re-spinned core area.
|
||||
|
||||
INSUFFICIENT EVIDENCE to quantify area without access to XH-1's core area budget or process node.
|
||||
|
||||
## Power and Energy Considerations
|
||||
|
||||
The following considerations apply **if and only if** XH-1 implements the corresponding power features. Each is OPEN QUESTION 7-conditional.
|
||||
|
||||
- **Power-aware verification (conditional).** If the core supports power gating or retention, the core testbench must be able to drive representative switching activity to enable power estimation, and the testbench must verify that power-gated domains behave correctly (controlled wake, retention restore, ordering with respect to coherence traffic). UPF/CPF power intent integration with the simulator is the standard mechanism.
|
||||
- **Energy proportionality (conditional).** If the core supports clock gating at the instruction or block level, the testbench should verify that idle and active transitions behave correctly at the architectural interface (no observable state loss in idle, correct wake on interrupt).
|
||||
- **Thermal and DVFS interactions (conditional).** If XH-1 supports DVFS at the core level, voltage/frequency transitions and their effects on memory coherence, pipeline flush behavior, and CSR access must be verified. This is a cross-cutting concern between core and system verification.
|
||||
- **Power estimation accuracy.** Pre-silicon power estimation depends on workload realism. A core testbench that only runs synthetic random code will produce misleading power numbers.
|
||||
|
||||
No power feature is assumed. Verification provisions for any power feature must be activated only when OPEN QUESTION 7 is resolved.
|
||||
|
||||
## Implementation Considerations
|
||||
|
||||
- **Methodology composition.** The project must select one of the alternative strategies (or a hybrid) and codify it in the verification plan.
|
||||
- **Toolchain selection.** Open-source (Verilator, cocotb, SymbiYosys, `riscv-formal`, `riscv-dv`, Spike, RISCV-ISAC, GTKWave) vs. commercial (Synopsys VCS, Cadence Xcelium, Siemens Questa, JasperGold, VC Formal, FormalPro, Formality, Conformal, Palladium, Veloce, ZeBu) is a strategic decision. The choice cascades through licensing, hiring, training, and CI. These categories are not equivalent in role: Verilator and cocotb are simulation/cosim frameworks; SymbiYosys is a formal-verification flow; `riscv-formal` is an ISA-level formal property set; `riscv-dv` is a stimulus generator; RISCV-ISAC is an instruction-stream coverage analyzer for the architectural test framework; commercial emulators are hardware-assisted verification platforms distinct from FPGA prototyping. The choice within each role is independent.
|
||||
- **Reference model choice.** Spike (C++, instruction-accurate) vs. Sail (first-order imperative ISA-specification language with a RISC-V model) vs. a custom golden model. Spike is the most common reference; the project must document any corner-case behaviors of the chosen reference against the RISC-V specification.
|
||||
- **Coverage database.** Coverage data are stored in vendor-specific formats (e.g., Synopsys UCDB / DDC, Cadence IMC, Siemens UCDB) and may be aggregated into a custom backend (e.g., SQLite) for CI. The format is a tool-vendor concern; the aggregation backend is a CI concern. These are distinct choices.
|
||||
- **Test categorization.** Tests must be tagged by domain (base integer, M-ext, A-ext, F/D-ext, privileged, vector, …), by intent (sanity, corner, regression, performance), and by status (passing, known-fail, expected-fail).
|
||||
- **Seed management.** Random seeds should be reproducible. Seed pools should be partitioned into "core regressions", "block regressions", and "fuzzing".
|
||||
- **Bring-up sequencing.** Block-level → core-level → multicore → system. The core testbench is the transition point between block-level and system-level work.
|
||||
|
||||
## Sign-Off Criteria
|
||||
|
||||
A core verification sign-off is the point at which the methodology exits the coverage-closure funnel. The following are candidate criteria; **every numeric threshold below is an illustrative starting point and is not a validated target**. The relationship to DRAFT status is restated at each threshold so that the list is internally consistent with the document status header.
|
||||
|
||||
- **Functional coverage.** All defined cover points hit, with cross-coverage exercised. **Illustrative starting threshold (not a validated target):** ≥ 95% cover-point hit, ≥ 90% cross-coverage hit; remaining items must be justified as unreachable or formally proven. Final threshold to be set in the verification plan.
|
||||
- **Code coverage.** Statement, branch, expression, toggle, and FSM coverage reported; excluded regions justified. **Illustrative starting threshold (not a validated target):** ≥ 95% statement, ≥ 90% branch, with manual exclusion of reset/DFT/clock-gating logic. Final threshold to be set in the verification plan.
|
||||
- **Assertion coverage.** All SVA / immediate assertions vacuously satisfied or actively proven; assertion cover points hit. Open assertions must be reviewed and dispositioned.
|
||||
- **Formal closure.** Per-block formal sign-off, with proof closure recorded for each property. **Operational requirement:** every inconclusive formal result must have a documented risk-acceptance disposition (strengthen via abstraction/decomposition, or accept with justification) recorded in the sign-off package. "Inconclusive" is not equivalent to "proven".
|
||||
- **ISA compliance evidence.** All applicable tests in `riscv-arch-test` pass and produce the architectural test framework's signature. `riscv-tests` (broader collection including `riscv-arch-test`) is run as a regression input. Signed certificate or equivalent record retained. **Prerequisite:** the specific RISC-V ISA, Privileged ISA, Debug, and Profile versions adopted by XH-1 are pinned in the verification plan.
|
||||
- **Mutation testing.** Periodic injection of known-defect mutations into a frozen RTL revision. **Illustrative starting threshold (not a validated target):** ≥ 80% of injected mutations detected by the testbench + assertions + formal. This number is a starting point and requires empirical validation on the XH-1 DUT. Equivalent-mutant handling (mutations that produce behaviorally identical RTL) must be defined; kill-rate targets are noisy and must be interpreted with caution. Final threshold and operator set to be set in the verification plan.
|
||||
- **Reference model validation.** Spike or Sail validated against the chosen RISC-V compliance tests before being used as a checker; validation evidence archived.
|
||||
- **Testbench self-check verification.** Every scoreboard check has at least one negative test that is expected to fail.
|
||||
- **Reproducibility.** Every sign-off run is reproducible from committed source, seed, and tool version.
|
||||
- **CDC/RDC closure.** CDC and RDC analysis clean for all core-level crossing structures.
|
||||
- **Lint clean.** Lint clean at the policy-defined severity level.
|
||||
- **Reset verification.** All reset domains and reset sequences verified; reset recovery verified.
|
||||
|
||||
## Verification Considerations
|
||||
|
||||
Core verification is itself subject to verification — the verification of the verification environment:
|
||||
|
||||
- **Testbench self-checking.** All scoreboard checks must be end-of-test verifiable; every check should be paired with at least one negative test that is expected to fail.
|
||||
- **Reference model validation.** The reference model (Spike, Sail) must be validated against the chosen RISC-V compliance tests before being used as a checker.
|
||||
- **Coverage sanity.** Cover points must be reviewed to ensure they are not trivially self-firing.
|
||||
- **Mutation testing.** A subset of RTL defects should be intentionally injected to confirm the testbench catches them. See `research/10-verification/fault-injection.md`. Equivalent-mutant handling must be defined.
|
||||
- **Coverage baseline stability.** Coverage baselines drift as RTL changes; a baseline-management policy (when a baseline is re-established, by whom, with what approval) is required.
|
||||
- **Tool qualification.** If the verification tools are used in any safety-relevant context, tool qualification (per ISO 26262, IEC 61508, DO-254, or similar) is required. The applicability of these standards to XH-1 is OPEN QUESTION 8. Tool qualification is a verification-of-verification concern, not only a sign-off checkbox.
|
||||
- **Version control and baselines.** Every test run must be reproducible from committed source. Coverage baselines must be versioned.
|
||||
- **Testbench verification at 128-instance scale.** The testbench itself has bugs. Verification of the testbench (negative tests, mutation of the testbench, equivalence of testbench revisions) is part of the methodology. At 128 instances, the testbench's own correctness becomes a non-trivial verification problem and is acknowledged here as a multicore-layer concern that the core testbench interface must support.
|
||||
|
||||
## Software Considerations
|
||||
|
||||
- **Toolchain integration.** The core verification environment must integrate with the software toolchain (compiler, assembler, linker, Spike, debug server) to enable reference-model checking / co-simulation and to run real programs.
|
||||
- **Firmware.** Firmware that runs on the core (boot code, exception handlers, interrupt dispatchers) is itself verification target; the core testbench should support loading and running firmware images.
|
||||
- **Operating system implications.** A core that supports an OS (Linux, RTOS) must be verified with syscall-heavy workloads. The core testbench should provide OS-aware run modes.
|
||||
- **Debug and trace protocols.** The core's debug-transport module (per the RISC-V Debug specification) must be exercised. JTAG or cJTAG interactions with the core's debug module are typically a core-level concern.
|
||||
- **Performance analysis tools.** Tools that analyze trace data, performance counters, and profile information should be able to consume the artifacts the core testbench produces.
|
||||
- **Software developer ergonomics.** The verification team's interfaces (test writers, coverage reviewers) should be ergonomic. A "developer can run a single test and see results in under a minute" goal is a useful operational target.
|
||||
|
||||
## Security Considerations (Out of Scope, but Flagged)
|
||||
|
||||
**ASSUMPTION (out-of-scope for this revision):** Transient-execution / side-channel verification (e.g., Spectre-class, Meltdown-class, and constant-time verification) is treated as out of scope for the XH-1 core verification layer for this revision. For a 128-core design with privilege boundaries, this is a real and high-risk concern. This is flagged as a high-risk gap and as a candidate for a follow-up document. The current methodology is sufficient for functional and microarchitectural correctness but does not, by itself, provide security assurance.
|
||||
|
||||
## Proposed Direction
|
||||
|
||||
PROPOSAL — A **hybrid methodology** for XH-1 core verification, with the following structure:
|
||||
|
||||
1. **Dynamic simulation** as the primary verification engine, with the framework (UVM vs. lightweight) gated on OPEN QUESTION 3 (tool posture), OPEN QUESTION 4 (team size and timeline), and the design-language choice (OPEN QUESTION 11).
|
||||
- If UVM: pair with `riscv-dv` as the instruction-stream generator and Spike or Sail as the reference model.
|
||||
- If lightweight: pair `riscv-dv` (or a custom generator) with cocotb / Verilator and Spike, with explicit acknowledgment of Verilator's reduced debug, SVA, and coverage support relative to commercial simulators.
|
||||
2. **Formal verification** applied to control-dominated blocks: fetch, decode, CSR file, MMU/TLB, exception/interrupt delivery, and the debug module. Coordinate with `research/10-verification/formal-verification.md` for tool selection. Apply `riscv-formal` as an ISA-level formal layer subject to the applicability evaluation in Section I. Sequential equivalence checking between major RTL revisions is a complementary activity.
|
||||
3. **Reference-model checking against a RISC-V reference model** (Spike or Sail) as the default, with **co-simulation in the strict sense** retained as a sampled cross-check (per Section D). Coordinate with `research/10-verification/isa-compliance.md`. Pin the RISC-V ISA, Privileged ISA, Debug, and Profile versions. At 128 instances, use a shared reference model with a deferred-check queue or sampled synchronization.
|
||||
4. **Directed tests** for XH-1 microarchitectural corner cases that random cannot reach economically. The corner cases to target are conditional on the in-order / out-of-order answer in OPEN QUESTION 2.
|
||||
5. **FPGA prototyping** for throughput-driven verification and as a substrate for fault injection and multicore bring-up. Commercial emulation is a conditional alternative based on licensing posture (OPEN QUESTION 3) and workload.
|
||||
6. **Regression infrastructure** with version-controlled seeds, baselines, and coverage data. Coordinate with `research/10-verification/regression.md`.
|
||||
7. **Coverage closure** as an explicit, tracked funnel with the sign-off criteria in the Sign-Off Criteria section, including baseline-management policy. Numeric thresholds are illustrative starting points and are not validated targets; final thresholds are deferred to the verification plan.
|
||||
8. **Reusable testbench architecture** so the core testbench composes inside the multicore testbench without modification, supporting both the per-die cost argument and the sampling strategy. The interface must support shared reference models, summary-only trace mode, and per-instance coverage identification.
|
||||
9. **Mutation testing** as a periodic check that the testbench is still doing useful work, with explicit equivalent-mutant handling. Coordinate with `research/10-verification/fault-injection.md`.
|
||||
10. **Standard core-verification adjuncts:** lint, CDC/RDC analysis, X-propagation handling, reset verification, and (conditional on OPEN QUESTION 7) UPF/CPF power-intent integration.
|
||||
|
||||
The hybrid is proposed; the specific framework within the hybrid (UVM vs. lightweight) is gated on OPEN QUESTION 3, OPEN QUESTION 4, and OPEN QUESTION 11, and the power-feature sub-portion is gated on OPEN QUESTION 7. The implementation-class sub-portion (DFT/ATPG scope, post-silicon layer) is gated on OPEN QUESTION 12. The document does not pick between alternatives that the OPEN QUESTIONS do not yet support.
|
||||
|
||||
## Confidence
|
||||
|
||||
- **High confidence:** The general methodology composition (dynamic + formal + reference-model checking / co-simulation + directed + prototype + regression + coverage closure, with the standard adjuncts: lint, CDC/RDC, X-propagation, reset) corresponds to the IEEE 1800.2-2017 UVM standard, the IEEE 1800-2017 SystemVerilog assertion subset, and the RISC-V International published ISA / Privileged / Debug / Profile specifications, and is broadly accepted in industrial practice. The references supporting each component are cited at the point of use.
|
||||
- **High confidence (conditional):** The 128-core scaling argument on defect-escape cost (an undetected defect is replicated 128-fold in silicon) is a direct consequence of the replication-homogeneity assumption and does not require empirical data. The argument about formal re-proof cost under bit-identical replication (near-zero marginal cost to re-run a proof against another elaborated instance of the same source; engineering cost of constructing the proof is paid once) is also a direct consequence of the assumption, with the parameter-set caveat acknowledged. The argument about dynamic simulation at full per-instance coverage requiring real additional compute is similarly structural.
|
||||
- **Medium confidence:** Specific tool selections, sign-off numeric thresholds, and area/power tradeoffs, given the absence of repository evidence on the OPEN QUESTIONS.
|
||||
- **Low confidence (OPEN):** XH-1's tool licensing posture, core microarchitecture details, ISA profile selection, team size, design language, power architecture, and target implementation class. These drive the specifics within the proposed hybrid.
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. What RISC-V profile does the XH-1 core implement, and what is the specific version pin for the Unprivileged ISA, Privileged ISA, and Debug specification? The reference model (Spike variant, Sail configuration) and the compliance test scope follow from this choice. The RISC-V International profile document should be consulted at toolchain selection to determine the exact profile designation adopted (e.g., a 64-bit XLEN core targeting an RVA application/supervisor profile, or a 32-bit XLEN core targeting a different profile track); this document does not assert a specific profile name as established fact.
|
||||
2. Is the XH-1 core in-order or out-of-order? The verification plan complexity differs substantially. The instructions explicitly disallow assuming either without repository evidence.
|
||||
3. What is XH-1's toolchain posture (open-source only, commercial EDA, mixed)?
|
||||
4. What is the team size and timeline? Methodology cost scales with team capacity.
|
||||
5. What is the core's microarchitectural contract — what blocks are in scope for unit verification, and what is the boundary between core and multicore? See `research/10-verification/unit-verification.md` and `research/10-verification/multicore-verification.md`.
|
||||
6. Are the 128 cores architecturally identical, or are there variants (big/little, mixed application/accelerator, per-core ISA-subset configuration)? Heterogeneity changes the testbench architecture and invalidates the per-die cost-leverage argument.
|
||||
7. What is the core's power architecture — is clock gating, power gating, retention, or DVFS at the core level, and what are the state-retention semantics?
|
||||
8. What is XH-1's relationship to functional safety standards (ISO 26262, IEC 61508, DO-254, or none)?
|
||||
9. How will the XH-1 core's trace, debug, and performance-counter interfaces be designed, and how will core verification consume them?
|
||||
10. What is the expected defect rate target at tape-out, and how does the verification program demonstrate closure against that target?
|
||||
11. What is the XH-1 design language (SystemVerilog, Verilog-2001, Chisel/Spinal HDL, Bluespec), and does verification target the HDL source level, the generated-RTL level, or both?
|
||||
12. What is XH-1's target implementation class — ASIC tape-out, FPGA-only deployment, or both? This determines DFT/ATPG scope, post-silicon validation depth, and sign-off criteria.
|
||||
13. What physical implementation tool flow (synthesis, PnR, timing) is envisioned, and what are the timing-closure constraints (target clock frequency, slack budget)?
|
||||
|
||||
## Sources
|
||||
|
||||
- IEEE 1800.2-2017, *Standard for Universal Verification Methodology*. Defines UVM. The Accellera UVM reference implementation is distributed under the Apache License 2.0; the IEEE 1800.2-2017 standard document is an IEEE publication.
|
||||
- IEEE 1800-2017, *Standard for SystemVerilog — Unified Hardware Design, Specification, and Verification Language*. Defines SVA and immediate assertions.
|
||||
- RISC-V International, *The RISC-V Instruction Set Manual, Volume I: Unprivileged Architecture*. **Version pin: OPEN** — to be selected at toolchain selection and recorded in the verification plan.
|
||||
- RISC-V International, *The RISC-V Instruction Set Manual, Volume II: Privileged Architecture*. **Version pin: OPEN** — to be selected at toolchain selection and recorded in the verification plan.
|
||||
- RISC-V International, *RISC-V External Debug Support*. **Version pin: OPEN** — to be selected at toolchain selection and recorded in the verification plan.
|
||||
- RISC-V International, *RISC-V Profiles*. **Version pin: OPEN** — the specific profile designation adopted by XH-1 is OPEN until OPEN QUESTION 1 is resolved. The published RISC-V profile document should be consulted directly at toolchain selection; this document does not assert a specific profile name.
|
||||
- Symbiotic EDA / YosysHQ, *SymbiYosys* documentation (open-source formal verification flow).
|
||||
- `riscv-formal` framework (Verilog, BSD-licensed), maintained at github.com/YosysHQ/riscv-formal.
|
||||
- `riscv-dv` instruction-stream generator, maintained at github.com/google/riscv-dv.
|
||||
- `riscv-tests` and `riscv-arch-test`, maintained by RISC-V International. `riscv-arch-test` is the architectural conformance test framework contained within the broader `riscv-tests` repository.
|
||||
- Spike (the official RISC-V ISA simulator), maintained at github.com/riscv-software-src/riscv-isa-sim.
|
||||
- Sail RISC-V formal specification, maintained at github.com/rems-project/sail. Sail is a first-order imperative ISA-specification language; the RISC-V model is generated from the RISC-V ISA formal definition.
|
||||
- RISCV-ISAC, instruction-stream coverage analyzer for the RISC-V architectural test framework, maintained by RISC-V International.
|
||||
|
||||
Quantitative throughput, area, power, and timing claims are not asserted in the body. The XH-1 repository documents referenced (under `research/10-verification/`) are placeholders ("SOON") and do not provide factual content to cite. No measurements, benchmarks, processor capabilities, URLs outside the repositories named above, or experimental results have been invented.
|
||||
+353
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user