AI Tools to Support Requirements Engineering & Test Case Developments

OVERVIEW

At SPEC Innovations, Artificial Intelligence technology has evolved the process of requirements engineering and test case generation. Utilizing the latest GPT-4o (omni) models, two custom-trained GPTs were created to provide systems engineers and project managers with powerful tools to create proficient requirements and further create test cases to increase time efficiency and performance. 

The Requirements Engineering GPT is designed to generate comprehensive, clear, and testable requirements based on user inputs, following the guidelines and best practices outlined in the INCOSE Requirements Writing Guide.

The Test Case GPT then facilitates the creation of structured and thorough test cases, ensuring that all aspects of system functionality are rigorously validated through different verification methods, avoiding bias and false assumptions.

Finally, through OpenAI’s advanced referencing features, engineers can chain the two GPTs and use them in combination to enhance their workflow efficiency and performance.

This blog explores the development, purpose, and application of the two AI assistants, demonstrating their capabilities and utility in Systems Engineering’s workflow.

INTRODUCTION

SPEC Innovations' Requirements Engineering GPT and Test Case Suite GPT increase organizational efficiency by leveraging artificial intelligence to assist Systems Engineers with daily tasks. These GPT models were meticulously trained using a combination of detailed instructions and extensive file inputs, ensuring adherence to industry standards and best practices, thus tailored to meet the needs of the systems engineering and verification communities. 

For the Requirements Engineering GPT, the system was provided the latest INCOSE Requirements Writing Guide, followed by a mapping guide that gave additional instructions on automating the INCOSE rules. The Test Case Suite Generator model was trained using various instructions and a structured framework skeleton, ensuring that GPT produced outputs in a consistent format. 

The instruction process for the GPT configuration is displayed in Figure 1.
 

Figure 1. Requirements GPT Configuration

AI-driven assistants can reduce costs and time-to-market by increasing the productivity of engineers supporting system design and verification. SPEC Innovations’ custom-trained GPTs provide more eligible output than the standard GPT-4o models. 

They provide invaluable support by generating clear, comprehensive, and testable requirements, as well as structured and detailed test cases. This not only enhances the quality and consistency of the engineering documentation, but also significantly reduces the time and effort required to produce them. 

By integrating these GPTs into their workflows, engineers can focus more on critical thinking and innovation, while AI handles the strenuous, repetitive, and time-consuming aspects of requirements and test case generation. This synergy between human expertise and artificial intelligence could ultimately lead to more robust and efficient engineering processes, an element in our industrial development that needs to be tested and analyzed further.

REQUIREMENTS ENGINEERING GPT

The Requirements Engineering GPT by SPEC Innovations is equipped with a robust set of features designed to assist Systems Engineers in crafting high-quality requirements, including:

• Adherence to Standards: The GPT generates requirements that comply with the INCOSE Requirements Writing Guide, ensuring industry-standard clarity, precision, and proficiency.
• Comprehensive Requirement Statements: Each requirement includes clear and concise "shall" statements.
• Rationalization: Each requirement is followed by a rationale section that explains how the requirement satisfies user or client needs.
• Categorization and Numbering: Requirements are organized hierarchically and categorized appropriately, making it easy to manage and navigate complex requirement sets.
• Table of Contents: The GPT is trained to outline the table of contents within its response to best communicate the structure and titles that exist in the requirements generation.
• Assumption Awareness: The model is instructed to communicate its assumptions as requirements are generated, allowing the user to verify the assumptions based on their needs and preferences. If the assumptions don’t match the user objectives, users can prompt the AI to generate more relatable requirements based on the new assumptions.
  
  

The utility of the Requirements Engineering GPT lies in its ability to streamline the requirement generation process, saving engineers considerable time and effort. 

To achieve this, there is a need for appropriate prompt engineering to lead the GPT in the correct direction throughout its generation process. Some effective prompting methods include expansive word descriptions of the system, bulleted lists of expectations/behaviors, file/URL sources to provide more context, chronological operational step order, etc.

Figure 2 illustrates the capabilities of the Requirements Engineering GPT in generating comprehensive and structured requirements for complex systems like autonomous vehicles. It conveys the systematic approach the GPT takes, starting from establishing key assumptions that underpin the system design to detailing specific requirements that ensure the system's functionality and safety.

Figure 2. Requirements GPT Example Use Case

By outlining its assumptions, the GPT ensures that the user has a clear understanding of the context and foundational conditions that lead to information generated by the model. This clarity helps prevent misunderstandings and sets the stage for creating requirements that are relevant and correct.

As seen in Figure 2, the GPT generates in-depth and specific requirements. If applicable, they are trained to provide quantitative measures that will allow the requirement to be testable. These requirements are detailed and accompanied by a rationale that explains their importance, thereby providing a strong justification for their inclusion. 

This comprehensive approach ensures that the generated requirements are not just a checklist of features, but a well-reasoned set of specifications that enhance the system's overall design and functionality.

TEST CASE SUITE GPT

The Test Case Suite GPT by SPEC Innovations provides features to generate detailed and comprehensive test cases based on user or assumed requirements, including:

• Structured Test Cases: The GPT generates test cases following a uniform structure that includes a unique identifier, a summative test case name, a detailed description of steps, expected results, verification methods, and the requirement being verified by the test case.
• Clarity and Precision: Test cases are clear and precise, ensuring that they can be easily followed and executed by engineers.
• Versatile Verification Methods: The GPT supports various verification methods, including Inspection, Demonstration, Testing, Analysis, Modeling & Simulation, tailored to the specific method by which the case verifies the requirement within the project.
• Edge Case & Data Suggestions: The GPT can identify and generate test scenarios for edge cases, and suggests relevant test data required to execute the test cases.
  
  

The utility of the Test Case Suite GPT lies in its ability to automate the creation of well-structured and thorough test cases, significantly reducing the time and effort required by engineers to structure and process test cases. By providing clear and detailed test cases, the GPT ensures that all aspects of system functionality are rigorously validated, leading to higher quality and more reliable systems.

Similar to all chatbots, the Test Case Suite GPT requires appropriate input to generate the most optimal outputs. Engineers need to provide system descriptions for the test cases, followed by a set of requirements, if applicable. The more context provided to the Test Case Suite GPT, the better it can understand the context, environment, and constraints to the system the test cases are being generated for. 

Additionally, the GPT can be initialized as an independent session or onto an existing chat using OpenAI GPT referencing. The prompt engineering element of using these GPTs is crucial to the success of the engineers.

Figure 3. Test Case Suite GPT Example Use Case

Figure 3 shows the capabilities of the Test Case Suite GPT in generating structured and detailed test cases as prompted by the user. 

In this example, the GPT was given the existing Requirements Engineering GPT requirements and prompted to generate test cases using the existing context. Similar to the Requirements Engineering GPT, the Test Case Suite GPT is trained to consistently provide a table of contents to users as an outline of the test case hierarchy.

As shown in Figure 3 for “TC.2.1 Object Detection and Classification,” there is clarity and precision in the GPT's output. It includes the unique identifier, clear test case name, step-by-step description, expected results, verification method, and specific requirement being verified for the Object Detection and Classification requirement. This detailed and structured format ensures that test cases are easily understood and executable, enabling engineers to validate system functionality effectively.

GPT REFERENCING

ChatGPT 4o includes a revolutionary feature that allows users to reference and integrate other GPTs directly into an existing conversation by using the @ symbol. The @ (beta) feature, which allows users to switch between different AI models/GPTs within ChatGPT, was introduced in May 2024 as part of a significant update to GPT-4o.

Figure 4. @ Feature Demonstration on GPT-4o

This capability facilitates GPT re-call, enabling users to combine their GPTs seamlessly. For instance, engineers at SPEC Innovations can reference both the Requirements Engineering GPT and the Test Case Suite GPT in a single instance, effectively replicating an "agent" process where different GPTs work together to provide a comprehensive solution. One GPT produces impactful outputs that can then be used as inputs to another GPT. 

This integration allows for enhanced collaboration and more efficient workflows, as the combined capabilities of multiple GPTs can address the knowledge and performance gap AI critiques argue against. By leveraging this feature, and creating trained GPT models with access to the latest processes, procedures, and technical standards, users can achieve greater synergy between different AI assistants, resulting in more robust and effective engineering processes.

As AI technology continues to evolve, the use of specialized GPTs in systems engineering is poised to become even more integral, driving efficiency, reliability, and innovation in the field.

The Requirements Engineering GPT and Test Case Suite GPT developed by SPEC Innovations represent significant advancements in leveraging artificial intelligence to enhance the systems engineering and verification processes. These AI-driven tools are designed to adhere to industry standards and best practices, providing systems engineers with the means to generate clear, comprehensive, and testable requirements, as well as structured and meaningful test cases.

For those interested in exploring these powerful tools, look them up on the GPT store or visit the links below:

• Requirements Engineering GPT by SPEC Innovations
• Test Case Suite GPT by SPEC Innovations