Evaluation Phase Failures in RPAL Interpreter for Specific Test Cases

[UchihaIthachi]

Description:
The RPAL Interpreter, developed as part of the CS3513 course, is encountering issues during the evaluation phase of the interpretation process. The interpreter successfully passes the AST and ST phases but fails to match the expected results during the evaluation phase when compared to the reference implementation (rpal.exe by Steven V. Walstra). This discrepancy is highlighted in specific test cases, such as tiny.1 and tiny.

Processes in RPAL Interpreter

1. Source Code to Abstract Syntax Tree (AST)

   • Description: Converts the raw RPAL source code into an abstract syntax tree (AST), representing the structure of the code hierarchically.
   • Testing Type: AST tests
     • Purpose: Ensure that the AST generation accurately reflects the structure and semantics of the RPAL source code.
     • Status: Passed

2. Abstract Syntax Tree (AST) to Standard Tree (ST)

   • Description: Transforms the AST into a standard tree (ST), a more refined representation suitable for further evaluation.
   • Testing Type: ST tests
     • Purpose: Verify that the ST accurately represents the intended computation from the AST.
     • Status: Passed

3. Standard Tree (ST) to Final Evaluation

   • Description: Evaluates the ST using a Control Stack Environment (CSE) machine or similar process to produce the final result or output.
   • Testing Type: Evaluation tests
     • Purpose: Ensure that the final output of the RPAL interpreter matches the expected results for various RPAL programs.
     • Current Issue: This process is where the interpreter is currently encountering issues, specifically failing tests like tiny.1 and tiny compared to the reference implementation (rpal.exe).

Testing Types

• AST Tests:

  • Description: Validate the correctness of AST generation from RPAL source code.
  • Status: Passed

• ST Tests:

  • Description: Confirm the accuracy of the transformation from AST to ST.
  • Status: Passed

• Evaluation Tests:

  • Description: Verify the correctness of the final evaluation process, ensuring that the interpreter produces correct outputs for RPAL programs.
  • Current Status: Failing for certain test cases (tiny.1 and tiny), indicating discrepancies between the RPAL interpreter's output and rpal.exe.

Current Issue

The RPAL interpreter successfully passes tests for AST and ST phases but encounters failures during the evaluation phase (ST to Final). Specifically, the issue arises in aligning the final output produced by the interpreter with the expected results from the reference implementation (rpal.exe). This discrepancy is highlighted in tests such as tiny.1 and tiny.

Issue Summary

The RPAL interpreter is failing to pass the tiny.1 and tiny test cases under evaluation testing compared to rpal.exe by Steven V. Walstra. The interpreter's evaluation process includes steps from source code to abstract syntax tree (AST), followed by standard tree (ST) evaluation, leading to the final solution.

Steps to Reproduce

Using pytest:

If your environment supports make:

make test           # Executes all test cases.
make test F=tiny.1  # Executes a specific test case (in rpal_tests/rpal_sources).
make test F=tiny    # Executes a specific test case (in rpal_tests/rpal_sources).

If make is not available:

• For all tests:

  PYTHON -m pytest -v --no-summary rpal_tests/test_generate_tests_with_rpal_exe.py

• For tiny.1 specifically:

  PYTHON -m pytest -v -k "tiny.1" rpal_tests/test_generate_tests_with_rpal_exe.py -vvv --tb=short

• For tiny specifically:

  PYTHON -m pytest -v -k "tiny" rpal_tests/test_generate_tests_with_rpal_exe.py -vvv --tb=short

For debugging purposes, print the CSE (Control Stack Environment) machine table to trace the evaluation steps of the standard tree:

python .\myrpal.py -ct rpal_tests/rpal_sources/tiny.1
python .\myrpal.py -ct rpal_tests/rpal_sources/tiny

Additional Details

• The RPAL interpreter successfully passes AST and ST test cases.
• Investigation is focused on the machine evaluation process implemented in src/cse_machine.