Don't feel up to reading? Watch the recording!
The common definitions for V&V are: verification is “the process of establishing the truth, accuracy, or validity of something,” and validation is “the action of checking or proving the validity or accuracy of something.”
However, the DoD and other organizations view verification more as a Developmental Test & Evaluation (DT&E), which meets specification-level requirements by proving that the solution-dependent requirements are satisfied. Verification in this context shows that the solution was or is being built according to agreed requirements.
DoD sees validation as Operational Test & Evaluation (OT&E). OT&S demonstrates to users, customers, and other stakeholders that the system meets the solution-independent requirements, and is usually performed at the enterprise and/or system level.
As part of OT&E, operational suitability is also determined to ensure it meets the warfighting mission needs. In-process validation helps ensure the system will ultimately be part of the accepted solution, in the target environment.
Recently, Mr. Christopher Collins and Mr. Kenneth Senechal published an article in the ITEA Journal that proposed, “A critical change in how test and evaluation (T&E) support capability delivery.” The new approach uses T&E continually throughout the lifecycle to inform development and fielding decisions.
How can we implement this concept? We need to understand how the data collected during T&E influences decisions early in the lifecycle, which means we need an ontology to know what data to collect that will be of use to decision-makers. An ontology (taxonomy + relationships) provides a language for communications. Developing an ontology is a lot of work, but important when you need people to work together.
Each discipline has its language, and having different languages can cause confusion. This is why it is important to find an ontology that makes sense for all parties involved.
The Lifecycle Modeling Language (LML), first published in 2013 by the Lifecycle Modeling Organization, provides a simple ontology that covers systems engineering and program management. The current version, v1.4, was released October 20, 2022.
Each version includes extensions, in the form of Appendices including extensions for DoDAF and SysML. LML v1.5 will provide a mapping to SysML v2 and BPMN.
In March 2022, version 1.2 was released and included an extension for V&V. This extension came from users of the language proving it out on several programs over many years. Test Suites act as containers for Test Cases, which are V&V tasks.
The Verification Requirement (VR) class was added to enable the classic way systems engineers have developed requirements. In the old standards, a requirements document would contain several sections. Section 3 was the system (or subsystem/component) level requirements and Section 4 was the related verification requirements used to verify the Section 3 requirements.
The V&V extension for LML has relationships to requirements and other aspects of the lifecycle model. You can add your relationships as well to aid in searching or reporting.
Innoslate, SPEC Innovations’ flagship software, uses LML as its primary language. Innoslate acts as a test bed for the language and research tool for digital and systems engineering. It also provides a powerful capability to connect detailed engineering activities to the decision-maker level.
Innoslate’s Test Center provides a dashboard to create Test Suites that represent key V&V activities throughout the Lifecycle. Each Test Suite contains Test Cases that describe the specific V&V activities.
LML can also be implemented by creating verifiable requirements. To do this, we need to answer the following questions:
Make an effort to avoid words such as “excessive,” “sufficient,” “resistant,” etc., and make the requirement quantifiable. Even “suitability” can be quantified.
Other ways Innoslate implements LML is through the creation or modeling of the test process, using the Action Diagram with Test Cases, adding timing and cost and then simulating, and including as part of the test plan.
Innoslate provides the integrated environment needed to connect all parts of the lifecycle. This includes all other aspects of V&V planning and execution such as:
Customers have interfaced Innoslate with LabView for dynamic hardware-in-the-loop testing. SPEC Innovations has recently interfaced Innoslate with Selenium for software test tracking. Selenium uses Innoslate’s Java SDK to make changes to entities (editing, creating, deleting, etc.). With this integration, we have created a seamless interface between systems engineering and software engineering.
The primary purpose of the V-Model was to remind systems engineers that they have a responsibility to create the verification requirements for the V&V activities.
Now with Test Center, we can create the test cases directly. This reduces the time of developing separate VRs and provides detailed test plans early in the lifecycle. Innoslate will always follow the current software engineering best practices.
Another thing we can do earlier in the process is to conduct a simulation to predict test results. The use of simulation for this purpose is not new, but modern tools provide the capability to conduct modeling and simulation together.
LML provides an ontology that connects the dots between systems engineering and test and evaluation today. It has been tested by numerous organizations inside and outside the DoD. The language can be implemented by any tool with an open database schema. It’s time to move beyond diagrams and focus on the data. That’s where true interoperability can be attained!
Learn even more by watching our webinar, "Verify & Validate the System Lifecycle."