CIAT (Compiler and Interpreter Acceptance Tester)

This is a copy of the README.rdoc from the CIAT gem, despite that same document's promise that this site is a tutorial and reference.

The tutorial and reference are coming!

DESCRIPTION

CIAT (pronounced "dog") provides a system for writing high-level acceptance tests for compilers and interpreters. Each acceptance test is entered into a single file, and that file identifies the elements of a test.

SYNOPSIS

Suppose you have a compiler written in Java that compiles a language named Hobbes. Your compiler targets the Parrot Virtual Machine. So you want to provide source code which is compiled with a Java program and that result is interpreted by Parrot.

Input File

Input files should be named with a .ciat extension and saved in a ciat folder.

A sample input file (simpleinteger5.ciat) for the scenario described above might look like this:

Compiles a simple integer.
==== source
5
==== compilation
.sub main
  print 5
  print "\n"
.end
==== execution
5

This file specifies four elements: description, source, compilation, and execution. The description is always the first element, always unlabeled, and used prominently in the HTML report. All of the other elements are dependent on the processors that you use.

In this example, we're using a "Java compiler" (a compiler written in Java) and a "Parrot executor". CIAT's "Java compiler" runs your compiler over the source, and that output is compared to the compilation element. Then the "Parrot executor" is executed with the generated compilation, and that output is compared to the execution element.

If any processor fails, either due to an error while running or a failure during checking the output, the remaining processors are not executed.

Some processors will use optional elements in a test file. For example, the "Parrot executor" knows about command-line arguments:

Compiles a simple integer and ignores the command-line arguments.
==== source
5
==== compilation
.sub main
  print 5
  print "\n"
.end
==== command line
89 pqp
==== execution
5

When the "Parrot executor" is run on the compilation, it'll also pass in 89 pqp as command-line arguments.

The Rakefile

This sample Rakefile pulls everything together:

require 'ciat'
require 'ciat/processors/java'
require 'ciat/processors/parrot'

def compiler
  classpath = Dir.glob('../lib/*.jar').join(':') + ":../bin"
  CIAT::Compilers::Java.new(classpath, 'org.norecess.hobbes.drivers.PIRCompiler')
end

def executor
  CIAT::Processors::Parrot.new
end

CIAT::RakeTask.new do |t|
  t.processors << compiler
  t.processors << executor
end

This rakefile will find all of the .ciat files inside a ciat directory, each one representing a test. Each test will be executed, and the results are put into a folder named +temp+, including the HTML report report.html. All of these settings can be tweaked; see the documentation for CIAT::RakeTask for more information.

REQUIREMENTS

INSTALL