111122, 613 PM Coverage Cookbook/Executable Testplan Format Marepuan us Wiki Coverage Cookbook/ExecutableTestplan Format — Whi Keteropint: OS-VVM (lbnaos) | Coverage [Gexaany cmaa (23) =) Moxerparanc 1) ‘A testplan isa document which captures the important features ofa design and how they will be verified Conepacanne | Motivation for Creating a Testpan = 2.Creating a Tesplan = 3 Capturing the Tesiplan 4 Contents of the Testplan 44.1 Required Plan Structure + LI Section and Tile + 4.12 Description + 4.03 Link and Type 4.14 Weight +415 Goal 42 Other Columns Understood By Questa + 43 User Defined Columns 1 5 Re-tlsing Existing Testplans Motivation for Creating a Testplan Funetional Verification has been described as a major challenge in SoC Designs with many citing Inadequate visibility into the verifieation process as a major factor eontebuting to this challenge ‘This lack of visibility impacts design quality, (resourcetoolsintastrctur). Typical questions that veri schedule prediciability and cost ion managers need answers 0 arc: When are we done with verification? Are all critical requirements of the system verified? Are we using all verification resources adequately to mest project schedules? By putting an executable plan in place that captures a prioritized list of our verification intent, we ae able to do the following: + Organize ourselves better: Its not possible fo completely verity al tat eal, important and nice to have features, then put 2 plan in place where swe ean identity alc of a design however ‘we ensure that all ertical and some important features are fully verified within schedule. If schedule permits, other features canbe verified as well + Daring Day to Day execution ofthe Verification project, an executable plan can help give you insight into current status, Progress can now be tracked instantly because you ean sce exaclly where you are relative to what you intended to verify. ‘You can visualize this progress against Milestones, + Helps manage function coverage closure a the pro} only of feature, or even resource ‘is being executed, Verification Engincers can focus their coverage closure efforts on the eitical Layered Organization of Testbenches (en) poe flunuiow Crengnxanvs HM ‘Tecroeuit aman Oreste TeeroRoH nporpawan Sureparypa Conus Ken Cransapr1so 14443 Coverage Cookbook (en) Merpnt npouecos nop es) 1 Merpinca oxpsraa maa (8) + Merpnct byemanonasnoro sospatni (en) pesfication to tetplan (en) * Executable Tesiplan Format (en) 1 Testlan to fasctinal coverage (e) 1 Coding for analysis (en) Coverage Examples Practice) (2) Requirements Writing Guidlines (en) OVM nerosoor Cavsaps repncion encase Aes Heropres npoexra + os-vwM + features that have been identified as hole, follawed by important features and then by the nice to have features, Creating a Testplan In many cases, the features will be veriie (Oips:verifcatonacademy.com/cookbook!Coverage/Cade_Coverage_ Metrics) din simulation and recorded verified using either code coverage fanetional coverage (Gttps/erifcatonacademy.com/cookbook’Coverage/Functional_Coverage_ Metrics) . The testplan can also include information about lab validation and firmware/hardware integration testing. For testplans which include code coverage and functional coverage, the connection between the testplan and simulations can be automated. To make the testplan executable a certain document format must be followed. The format which Quest's Verification Management solution uses is described below. Capturing the Testplan Flexibility i key when it comes 10 connecting a estplan toa simulator. In Questa Verification Management's ease, several different source document formats ae valid: The only requirement is thatthe source document must be able to be exported to an XML document, For example, Microsoft Word, Microsoft Exeel, OpenOffice Writer, OpenOffice Cale and Adobe FrameMaker can all easily outpt an XML document which ean be processed and made executable ‘simard.comiwikiindex prpiCoverage_Cookboow/Exeeutable_Testplan_Format 4snitt22, 513 PM Coverage Cookbook/ExecutableTestplan Format — Wiki In fact it is perfectly accepabe to capture one section of your testplan in Word and another in Excel, These different sections cas then be combined together fag described below in the Re-Using Existing Testplans (tp! verifcationacademy.com/cookbook/Coverage’Executable_Testplan_FormaldRe- Using_Existing_Testplans) section OF the different input options, spreadsheets are the most commonly used and lend themselves very nicely to the type of dts which is captured ina testplan ‘The spreadsheet format provides a clean and concise means of visualizing verification intent. The rest ofthis article will describe both required information needed in spreadsheet and how to Mexibly add additional information for usage throughout the tesipan’s life eye Contents of the ‘Testplan Capturing the requirements for testing into the plan is @ process. That process is defined in the Specification to Testplan (Gutps:verifcationacademy.com/cookbook/Coverage/Specification_toesiplan) article. The process involves rigorous analysis of specifiatons, collaboration between architecture, design and verification teams snd reviews to ensure nothing is missed. AIL ofthe requirements which come out of this process need tobe captured inthe testplan with some required structure, The required structure allows the testplan fo become executable and to become part, ofthe verification eye, Required Plan Structure (Questas Verification Management solution requires four dstnet piece of information foreach requirement captured. They are Section, Tile, Link and Type. Adgitionall, other information is understood out of the box. The additional information for each requirement includes a Description, a Weight and a Goal ‘While these additional fields are not required, they are sed the majority of the time, Quest's Verification Management solution also has the flexbity to allow user defined fields (hrpsl'verifietionacademy.corseoekbookiCoverage'Execstable_Tesilan_Format#User_Defined Columns) Lf we look atthe typically used fields ina spreadsheet format, each fed is represented by a colums inthe spreadsheet. —t Pan Stactre Each row in the spreadsheet corresponds to a requirement captured during the tesplan creation process. Each column hes specific meaning in Quest's Verification Management solution. Section and Title The Section and le columns workin conjunction to create the naming end hierarchy within the testplan, Section “The Section column, usually a number, is used to create hierarchy within the testplan and group related testplan items together ina parent/child relationship. In spreadsheets, the user is responsible for entering this information. Typically you will start numbering sections with the number ‘I’ and continuing sequentially A sub-section beneath that section would then be numbered "I.1” and so on, where each additional lvel of hierarchy would be represented by the addition ofa.” between section numbers, ‘Tite ‘The ttle column captures the mame ofthe requirement or design feature tobe verified. This isthe name of the testplan section that will appear within Quest ‘The name chosen here should have meaning as it will be visible through the tool ow When the esiplan is extracted by Quesia, i ses the information inthe Section and Title columns to create a hierarchical name and unique tag foreach row ofthe tesiplan. For example, given the simple testplan example above, Section 1 would have a hierarchical name ‘estplan/Parent_1/Child_1 Description ‘The description column allows for more deal to be added to the spreadsheet. This could include references to otber documentation to allow engineers to father more information or it could be a simple explanation as to why the requirement exists. Any text ean be captured inthis column. ft is technically ‘optional, but in pratie a requirement captured ina tesiplan should have an entry inthe description column, Link and Type ‘The Link and Type columas are sed to specify the code or functional coverage items that will be linked to the requiressent A requirement canbe linked to multiple different coverage metrics, including mic of different types. Questa support linking to Covergroups, Covespoins, Crosses, Assrtions, Cover Directives, Directed tests and cade coverage metric types. These columns also allow other testplans to be imported as described in the Re-Using Existing “Testplans section ‘simard.comiwikiindex prpiCoverage_Cookbooi/Exeeutable_Testplan_Format 26snitt22, 513 PM Coverage Cookbook/ExecutableTestplan Format — Wiki Link ‘This column is where you specify the same ofthe actual coverage objec(s) in the coverage database that is linked to this respetive testplan item, This could include a specific covereroup instance, an assertion, et. ‘Type ‘This column is where you specify the type of the coverage object you specified in the Link column, ‘Together, he Link and Type column information is used to find the erresponding coverage abjects in the coverage database effcientty and ereate the Tinks ‘erween the tesiplan and the specified coverage objects, These columns enable the estan io become executable Weight “The weight column captures an integer number that reflects the relative importance of the curenttetplan item amongst its siblings, to its parent testplan section. The default is | if not specified. When coverage for the tstplan is being calculated by Questa, which uses a "weighted-average” calculation algorithm, these weights are taken into account. For more information about how Questa calculates tesplan coverage please see Questa documentation on Verification Management Adgitionally, the weight column can be used to exclude portions ofa testplan by specifying a value of 0 forthe testlan section / item row that need to be excluded, Goal ‘This colun specifies the verification objective fora particular tesiplan section, Legal values range from to 100, withthe default being 100 ifn0t specified. ‘Questa uses this information to determine the point at which atestpan section / tem is deemed tobe covered, It Joes not alter how coverage is calculated Other Columns Understood By Questa “There are other special purpose columns which can be wsed in a coverage texiplan. These columns are all optional however when they are present Questa leverages their content Path ‘When linking toa specifi instance of @ covergroup, two options exist, Optional path to the eovergroup could be added into the Link colums. This works very well if only one coverge item is being accessed at that level of design hierarchy. If however, multiple coverage items are being linked to inthe same level of design hierarchy; the Path column allows for the specification ofthe design path which willbe prepended tothe entry in the link column to create a fully qualified reference. Unimplemented ‘As tesslans are being define, it s common for requirements to be captured where corresponding coverage items dont yet exist i atesthench or desig, To handle this situation, a requirement can be maked as unimplemented by either adding a value of 'yes! or a number greater than zero to the Unimplmented ‘columa, This will cause testplan coverage calculations to accurately reflect that a requirement exists which i not yet covered by showing zero coverage for that requirement. By default i is assumed that coverage fora requirement is implemented ness ths column is specified, User Defined Columns ‘To ensure users ave the flexibility to recond other vital information related to a requirement, Quesa's Verification Management solution allows user defined columns to be created as wel, Tis allows fr the specification of anything that needs tobe tracked, It ean also give you beter visibility ino the verification process, Some columns which are routinely added inelude which engineers rexponsible for esting @ requirement and what the priity ofthe requirement oF design feature is ‘These added columns help to answer questions such as + Tow am I doing on my high priority testpan items? * Where am I with regards to meeting my current project milestones? + How are my doing in terms of my resources? The spreadsheet used in the System Level Functional Coverage Example (ips verifcationacademy.com/ cookbook Coverage/System_Level_Functonal_Coverage_ExampletFunctonal_Coverage Model_Creation:_ Spreadsheets) adds Owner and Priority user defined columns Re-Using Existing Testplans ‘simard.comiwikiindex prpiCoverage_Cookboow/Exeeutable_Testplan_Format 38snitt22, 513 PM Coverage Cookbook/ExecutableTestplan Format — Wiki Oen it is useful to be able to make use oF existing tesiplan’sin other contexts A few common situations where this becomes useful are: + Testplans were developed at the block level and need tobe incorporated int the chip or system level texiplan. + Atestplan is dalivered with apiece of IP ox verification IP, For example, protocol specific Verification IP should contain a complete protocol testplan te ensure a protocol is completely verified Incorporate auto generated testplans into our sub-system or SoC level plan, For example, for power aware simulations simulators should beable to «create a tesiplan to ensure the power seenarios ae completely + Combine testplans captured by diferent editing tls into one master tesplan In these situations, existing testplns can be hierarchically imported into the top-level testplan being created, allowing fo visualization ofthe fll depth of the verification effort. When creating such a testplan, the Type column will be given a value of "XML" to inform Questa that this setion does not reference a coverage object, but rather i refers to another testplan document, The Link column will be used to specify the actual tsfplan file that contains the tesiplan being imported (via either a relative or full-path the fl) erounn — etip:/simbardcomvwiklndexphpile-Coverage Cookbook Execuab Kereropn: OS-VVM Uliion) | Coverage -_Testplan_Formatdeoldid-2436» 1 Tlocasywes uaweneute 70H enpassas: 12:37, 12 anpena 2013. + Kesroit erpauane odpamanacs $173 pasa, [0 aGozatounne ysacrixos] ‘simard.comiwikiindex prpiCoverage_Cookboow/Exeeutable_Testplan_Format 48