What is a Verification Test plan? What are details to be included in a Test plan? Why is it important in functional Verification? What are some sample test plan templates? How is a Verification Test plan used and helpful?
Design Specification
A design specification (also known Architecture/ Micro architecture Specification) is a document that is developed by an architect or micro-architect capturing all the details of a design and its implementation. This would include all the supported features, interfaces and protocols , configuration and initialization information including registers, and other details. A verification engineer who is responsible for verifying a given design considers the design specification as a golden reference. His job is to make sure that the design implementation is functionally correct with respect to this design specification.
Verification Test Plan
A Verification Test plan is a specification document that captures all the details needed for verifying a given design. A Verification engineer is responsible for developing this plan initially as he understands the details of the DUT (Design under Test). A proper planning is always important to complete verification with highest quality and in a predictable time period
By failing to prepare you are preparing to fail
– Benjamin Franklin
Many a times if you don’t give enough importance to this, you are setting yourself to failures later in the project – which can include bug escapes, re-working several infrastructure work, time crunches, and loosing on quality.
What all details should be included in a Verification plan?
The Verification plan should cover all aspects to ensure a quality verification in a predictable time. I would break this into following categories – What to Verify ? How to Verify ? When to Verify ? How to ensure completeness with quality ?
What to Verify ?
The plan should list down details on what all features of design to be verified. A brief explanation of all features should be listed as individual test plan items. In addition to the features, all supported design configurations (register settings) under which these features should be tested should be listed down as test plan items.
Not all of these features/configurations will need individual tests. Most of the times several of these features and configurations needs to be tested in combination. A good constrained random verification infrastructure will be designed with this in mind. Details on stimulus infrastructure should be included accordingly.
In addition to the features and combinations, it is good to capture specific micro-architectural cases that needs to be ensured for correctness. This could include explicitly calling out cases for individual test scenarios or coverage observation. Some examples of this include various interface properties and internal micro-architectural events ( like state machines, fifos, arbitration and other logical block interactions).
Another category of what to verify also includes specific stimulus patterns, interactions, high level usage scenarios, potential deadlock/livelock conditions etc – some of which depends on the type of design.
This section should finally translate to a good quality stimulus and coverage properties that can ensure the quality of stimulus.
How to Verify ?
Once details on what to be verified is understood and capture in the verification test plan, next step is to figure out how best each of the items can be verified. Based on the type of design and what needs to be verified , different set of methodologies, different types of stimulus , different type of checking mechanisms etc would need to be designed and implemented.
Majority of functional verification uses simulation and constrained random / coverage driven approach for faster verification. Selective areas of design and special features could also be tested using formal verification or other techniques. Details about how the stimulus infrastructure will be developed, the various knobs to control randomization for better coverage, development of sequences and tests etc should be in this section.
Checking mechanisms to ensure functional correctness should also be designed and captured in the verification test plan. Checks could be implemented as scoreboards, interface or embedded assertions inside RTL or verification components.
Both details on What to verify and How to verify are necessary to architect a good verification test bench and this should be documented in the test plan itself. A block diagram of various test bench components, hierarchy and stimulus patterns should be captured and explained well so that this can be translated into implementation with lesser issues later in the execution.
When to Verify ?
Every project has a time line for completion and as a matter of fact, there will always be more things to be verified in lesser time. The verification test plan should capture a rough effort estimate for a complete execution – in terms of time needed for development of verification test bench components, stimulus patterns, testing and regressing, coverage analysis, debug and quality completion.
Based on the effort estimate, it is a common practice to also classify the various features/configurations to be verified into at least three priorities (say High, Medium, Low). This helps in making better calls during project execution timeline to prioritize and de-prioritize various task and make informed decisions.
Reviews
Not all need to be perfect while developing a verification test plan, but it is still an important document to be made upfront when a design verification needs to start. Most of current design life cycles see several changes through development and the verification test plan will need updates as well. Another good practice seen is to review the verification test plan at least 3 times in a project life cycle – once in beginning, middle and towards end. This is collectively done with verification and design engineers along with architects and other experts.
