CS 4980 CC: Assignment 2

The goal of this assignment is to give you experience using SableCC and regular expressions to specify lexemes for a simple programming language.

The textbook defines a language it calls "MiniJava". See the appendix for basic information. You are being given a starting point of some code to recognize the complete set of tokens in this language; the goal is to extend this to cover additional cases. The two files you are being given are

• mjava.grammar: SableCC source file that (once you have completed it) will recognize all MiniJava lexemes.
• Main.java: A wrapper for the lexical analyzer created by SableCC. You should not have to change this code. In particular, there is no need to introduce a package declaration.

When complete, your solution should work on the following test files:

• factorial.mj
• more_tokens.mj
• whitespace.mj

(These cases are also used by esubmit.) To get it to work on these examples, you will extend mjava.grammar to include a number of missing tokens such as while. You will also extend it to allow multi-character identifiers, multi-digit numbers, and more types of comments. esubmit will have additional test cases; remember you can download all test cases directly from esubmit. For full credit, your solution must have no differences for any tests; this may mean adjusting the names of tokens in your mjava.grammar file so that you match the printed names in the output.

Submit your solution to esubmit.msoe.edu. If you are on campus, you can access the server directly. If you are off campus, you must run GlobalProtect to get behind the campus firewall. Log in using your campus username and standard password. Then browse to cs4980cc-h, select as2lexer as the assignment, submit Main.java as the "main", and add mjava.grammar as an additional file. Click on the Submit button and it will build and test your code.

Notes and Additional Requirements

• You should be able to run the example without any changes to the files. Save the two source files in a folder, save sablecc.jar in the same folder (this is version 3.7 of SableCC), save (at least) factorial.mj, and type

          java -jar sablecc.jar mjava.grammar
          javac Main.java
          java Main < factorial.mj
  

  Note: be sure to recompile Main.java after re-generating the SableCC parser - running the sablecc.jar file creates .java files that need to be recompiled.

  You can problably get IntelliJ to build your solution, but it's valuable to learn to run Java from a command prompt. SableCC should work with most recent versions of Java, but esubmit is currently using Java8. Message me if you cannot determine how to run Java from a command prompt and I will update this writeup with more directions as needed. Note the main you are given reads from standard input; the < symbol above redirects the input from a file rather than a keyboard.

• The textbook suggests that the lexer should support nested comments (/* ... /* nested */ ... */), but this is incorrect. The first */ after a /* should terminate the comment.
• SableCC uses [...] to represent simple ranges of characters, not complete sets of alternatives; for example, to allow any upper or lower case character write the RE "['a'..'z']|['A'..'Z']". SableCC does not recognize ^ in [...], so the RE for (say) all ASCII characters but the newline would be "[0..9]|[11..127]".
• The most challenging part of this problem is multi-line comments. The first part is easy: '/*'. The hard part is to allow asterisks within comments but not followed by slash. The solution is to first add support for a simple case like '/* xyz */'. Then, add support for a '/*' followed by *s appearing before '*/'. Finally, allow the comment to appear on multiple lines.
• Note that identifiers allow digits. That is, implement the Java rules: an identifier is not a keyword, starts with a letter or underscore, and then is followed by letters, digits, and underscores.
• This assignment is more focused on learning to run SableCC than programming skills, so the code will not be graded against the coding standard.
• Feel free to solicit help from other students for this assignment. You can work in teams if you prefer, but each student must submit their own version.