Brief Introduction to Verification Methodology

Design verification is the process used to demonstrate the correctness of a design w.r.t. the requirements and specifications.

In digital design flow, verification ensures that the chip functions correctly per the design intent before sending the design off for manufacturing.

Specifically, verification methodology is a standardized way to verify integrated circuit designs.

A verification methodology defines the models that are created, how they are used and the ways in which tools are used to manipulate them. Models can define the design at several levels of abstraction, they can define the requirements of the design or they can define closure criteria.

Verification methodology is also a systematic way of doing things with a rich set of standard rules and guidelines. Verification methodology provides means to build a robust, reliable and complete verification environment. Verification methodology reduces verification efforts with its predefined libraries.

Verification methodology further provides a framework for easing implementation of coverage driven constrained random verification environments.

Verification methodology also improves the portability. Consistency and portability makes the adoption of the third party vendor solutions easy. Third party BFM (Bus Functional Model) usage is one such example.

The main purpose of a verification methodology is to optimize some aspect of the design while minimizing the time spent on it. It also ensures consistency within a company such that reuse becomes possible. Verification methodology’s primary goal is to make the adoption of best-known practices of verification easier. The verification environments built according to the methodology provides the consistency in building verification environments.  Verification methodologies restrict the verification environments architecture to certain standard patterns. This restriction allows consistency and standardization when used right.

Verification methodologies came into existence soon after the first dedicated HVLs (Hardware Verification Languages) appeared. The main advantages of adopting a methodology (such as UVM) are
• Reusability through test bench re‐use and verification IP allowing plug and play
• A proven methodology with industry wide support and availability of engineers with existing knowledge and/or experience
• Simulator and vendor independence

History of Verification Methodologies

Verification methodology is basically set of base class library which we can use to build our testbenches. Verification methodology by itself will not do any functional verification. It’s just an enabler.

Design verification takes anywhere from 40 to 70 percent of the total development effort for the design.

A wide variety of verification technology options are available and these can be broadly categorized into four classifications: simulation based technologies, static technologies, formal technologies, and physical verification and analysis. To achieve the required verification goals, a combination of these methods must be used.

Over the years, companies have released numerous different methodologies using different languages. In this essay we enlist some of the key methodologies according to their popularity as of today.

Universal Verification Methodology (UVM) (SystemVerilog) by Accellera (open source)

The Universal Verification Methodology (UVM) is a standardized methodology for verifying integrated circuit designs. UVM is derived mainly from the OVM (Open Verification Methodology) which was, to a large part, based on the eRM (e Reuse Methodology) for the e Verification Language developed by Verisity Design in 2001. The UVM class library brings much automation to the SystemVerilog language such as sequences and data automation features (packing, copy, compare) etc., and unlike the previous methodologies developed independently by the simulator vendors, is an Accellera standard with support from multiple vendors: Aldec, Cadence, Mentor Graphics, Synopsys, Xilinx Simulator(XSIM). Source: https://en.wikipedia.org/wiki/Universal_Verification_Methodology

Open Verification Methodology (OVM) (SystemVerilog/ SystemC) by Cadence Design Systems and Mentor Graphics
Replaced by UVM

The Open Verification Methodology (OVM) is the first truly open, interoperable, and proven verification methodology. The OVM is an open-source SystemVerilog class library and methodology that defines a framework for reusable verification IP (VIP) and tests. It is 100% IEEE 1800 SystemVerilog and provides building blocks (objects) and a common set of verification-related utilities. The OVM release will be under the Apache 2.0 license, enabling anyone to use OVM libraries for any purpose, including creation of derivative work.

The OVM is jointly developed by Cadence and Mentor Graphics to facilitate true SystemVerilog interoperability with a standard library and a proven methodology. Completely open, it combines the best of the Cadence® Incisive® Plan-to-Closure Universal Reuse Methodology (URM) and the Mentor Advanced Verification Methodology (AVM), and is usable on two-thirds of the world's SystemVerilog systems. The OVM will also facilitate the development and usage of plug-and-play verification IP (VIP) written in SystemVerilog, SystemC®, and e languages.

In December of 2009, Accellera selected the OVM to be the base of the emerging Universal Verification Methodology (UVM). The first version of the UVM, released in May of 2010, is based on OVM 2.1.1 with the “ovm_” names converted by a script to “uvm_” plus a few minor changes to the callbacks and end-of-test, but not to the core methodology or base classes.

Source: https://www.cadence.com/en_US/home/alliances/standards-and-languages/open-verification-methodology.html

Open Source VHDL Verification Methodology (OSVVM) by Aldec/ Synthworks

OSVVM is an intelligent testbench methodology that allows mixing of "Intelligent Coverage" with directed, algorithmic, file based, and constrained random approaches. OSVVM is an integrated environment designated for verification of VHDL. OSVVM stands for "Open Source VHDL Verification Methodology". OSVVM is a set of VHDL packages, initially developed by Aldec and Synthworks. OSVVM helps you adopt modern constrained random verification techniques using VHDL. With OSVVM, one can add advanced verification methodologies to their current testbenches without having to learn a new language or throw out the existing testbench model. OSVVM supports the same features as those based on other verification methodologies. That includes Transaction Level Modeling, Constraint random test generation, Functional Coverage, Message filtering ,Scoreboards and FIFOs, Error reporting, etc.
Source: https://www.aldec.com/en/support/resources/documentation/articles/1902

Universal VHDL Verification Methodology - UVM For VHDL (UVVM) by Bitvis

UVVM (Universal VHDL Verification Methodology) is a free and Open Source Methodology and Library for very efficient VHDL verification of FPGA and ASIC – resulting also in significant quality improvement.

The UVVM (Universal VHDL Verification Methodology) is the fastest growing FPGA verification methodology – independent of language. This is due to the improvement UVVM yields in both FPGA quality and development time. This open source Library and Methodology has the most extensive VHDL verification support available and lets you verify really complex DUTs in a very efficient manner providing modularity, reusability, constrained-random stimulus and functional coverage similar to UVM. UVVM also has the largest library of open source VHDL verification models and components. With more than 50% of all FPGA designers using VHDL, UVVM provides a great verification solution for these users.

Verification Methodology Manual (VMM) (SystemVerilog) by Synopsys

Verification Methodology Manual(VMM) was the first successful and widely implemented set of practices for creation of reusable verification environments in SystemVerilog. Created by Synopsys, one of the strong proponents of SystemVerilog, VMM harnesses language features such as object-oriented programming, randomization, constraints, functional coverage to enable both novices and experts to create powerful verification environments. VMM contribution was an important factor in creation of UVM.

Reference Verification Methodology (RVM) (Open Vera) by Synopsys

The Reference Verification Methodology (RVM) is a complete set of metrics and methods for performing Functional verification of complex designs such as for Application-specific integrated circuits or other semiconductor devices. It was published by Synopsys in 2003.

RVM is implemented under OpenVera.

The SystemVerilog implementation of the RVM is known as the VMM (Verification Methodology Manual). It contains a small library of base classes.

Advanced Verification Methodology (AVM) (System C & SystemVerilog) by Mentor Graphics
Replaced by OVM and UVM

A verification methodology and base class library written in SystemVerilog and created by Mentor Graphics in 2006, the AVM was the precursor to OVM and UVM. It provided a framework for component hierarchy and TLM communication to provide a standardized use model for SystemVerilog verification environments. AVM is not recommended for new projects, refer instead to Universal Verification Methodology (UVM).

e Reuse Methodology (eRM) (e Hardware Verification Language) by Verisity Design      
Replaced by OVM and UVM

eRM à URM (Universal Reuse Methodology) by Cadence Design Systems for SystemVerilog + Mentor Graphics' AVM  à OVM (Open Verification Methodology) à  UVM (Universal Verification Methodology)

The e Reuse Methodology (eRM) was the first reuse methodology to emerge in the Hardware Verification Language space and was used in conjunction with the e Hardware Verification Language. It was invented in 2001 by Verisity Design and was released in 2002. The methodology was composed of methodology guidelines for such topics as:

    File naming conventions
Functional partitioning of the testbench
Code packaging Guidelines
Sequence and message class libraries

eRM formed the basis of the URM (Universal Reuse Methodology) developed by Cadence Design Systems for the SystemVerilog Language. URM, together with contribution from Mentor Graphics' AVM, later went on to become the OVM (Open Verification Methodology) and, finally the UVM (Universal Verification Methodology) today.

SVUNIT  (Verilog/SystemVerilog) by AgileSoC

SVUnit is an open-source test framework for ASIC and FPGA developers writing Verilog/SystemVerilog code. SVUnit is automated, fast, lightweight and easy to use making it the only SystemVerilog test framework in existence suited to both design and verification engineers that aspire to high quality code and low bug rates.

SVUnit For Verification Engineers

Verification engineers can use SVUnit to verify testbench components in isolation prior to being used in subsystem or chip/product level testbenches. Because SVUnit imposes very few restrictions on developers, it can be used to develop components for simple Verilog-based testbenches used for directed testing, complex SystemVerilog-based constrained-random testbenches and everything in between. SVUnit also supports development of UVM-based verification testbenches and IP.

SVUnit For Design Engineers

Both verification engineers and the EDA industry as a whole have failed to provide a designer-friendly option for testing RTL. SVUnit changes everything by giving design engineers a practical framework that is easy of use; it does what they need it to do without the unnecessary overhead and baggage of other industry test frameworks. SVUnit gives design engineers a platform for exhaustive stand-alone testing and/or unit testing critical logic prior to release to the verification team, all while avoiding a lot of the overhead required to maintain an ad-hoc testbench. The result is higher quality RTL and an overall decrease in time lost to debug.

VUNIT (VHDL/SystemVerilog)

VUnit is an open source unit testing framework for VHDL/SystemVerilog. It features the functionality needed to realize continuous and automated testing of your HDL code. VUnit doesn't replace but rather complements traditional testing methodologies by supporting a test early and often approach through automation.

VUnit reduces the overhead of testing by supporting automatic discovery of test benches and compilation order as well as including libraries for common verification tasks. It improves the speed of development by supporting incremental compilation and by enabling large test benches to be split up into smaller independent tests. It increases the quality of projects by enabling large regression suites to be run on a continuous integration server.

VUnit does not impose any specific verification methodology on its users. The benefits of VUnit can be enjoyed when writing tests first or last, when writing long running top level tests or short running unit tests, when using directed or constrained random testing. Often projects adopt mix of approaches for different testing needs. VUnit has been used in production environments where thousands of tests take several hours to run on powerful multi-core machines as well as in small open source projects where only a small package is tested in a few seconds.

COCOTB (COroutine based COsimulation TestBench)

COCOTB is a COroutine based COsimulation TestBench environment for verifying VHDL and SystemVerilog RTL using Python.

COCOTB is a CO-routine based CO-simulation Testbench environment for verifying VHDL/Verilog RTL using Python. It is an open-source environment and hosted on Github. COCOTB can use Riviera-PRO simulator to simulate the RTL. It basically helps in the testing process and makes it comfortable, especially for people who are not very familiar with the HDL concept of verification. It uses the same design-reuse and functional verification concepts like UVM, however is implemented in Python. Hardware Description Languages such as VHDL, Verilog and System Verilog are used for the synthesizable designs only. COCOTB also has a built in support for integrating with the Jenkins continuous integration system.

COCOTB requires a simulator to simulate the HDL design and has been used with a variety of simulators on Linux, Windows and macOS.