logo Use CA10RAM to get 10%* Discount.
Order Nowlogo

In this assignment, you will implement a tokenization function for a hypothetical language with the

INSTRUCTIONS TO CANDIDATES
ANSWER ALL QUESTIONS

In this assignment, you will implement a tokenization function for a hypothetical language with the following syntactical elements: Floating point numbers in the following formats: 10, 0.1, -10, 10.12. A number consists of an integer part, followed (optionally) by a decimal point and a decimal part. Note that the integer part is required – .1 is not a valid floating-point number – and likewise, if a decimal point is present that at least one fractional digit must follow it –

0. is also not a valid floating-point number. Variables, which must start with an alphabetic character but may be followed by alphanumeric characters.

The operators +-*/ and parentheses (), each of which should be a single token. The comparison operators < <= > and >=.

You will need nine states to tokenize this language: INTEGER: state for the integer part of a FP number. DECIMAL: state for the decimal point of a FP number. FRAC: state for the fractional part of a FP number. ALPHA: state for the first character of a variable.

ALPHANUM: state for the following characters of a variable. OP_PAREN: state for the operator/paren tokens LT1, LT2, GT1, GT2: states for the first/second characters in less-than (or equal to), greater-than (or equal to). SPACE: sequences of whitespace. Here is the state-transition table for this language, showing how the state changes in response to each character type: State\Char Digit Alpha . +-*/() < > = Spaces INTEGER INTEGER ALPHA DECIMAL OP_PAREN LT1 GT1 error SPACE DECIMAL FRAC error error error error error error error FRAC FRAC ALPHA error OP_PAREN LT1 GT1 error SPACE ALPHA ALPHANUM ALPHANUM error OP_PAREN LT1 GT1 error SPACE ALPHANUM ALPHANUM ALPHANUM error OP_PAREN LT1 GT1 error SPACE OP_PAREN INTEGER ALPHA error OP_PAREN LT1 GT1 error SPACE LT1 error ALPHA error OP_PAREN LT1 GT1 LT2 SPACE LT2 error ALPHA error OP_PAREN LT1 GT1 error SPACE GT1 INTEGER ALPHA error OP_PAREN LT1 GT1 GT2 SPACE GT2 INTEGER ALPHA error OP_PAREN LT1 GT1 error SPACE SPACE INTEGER ALPHA error OP_PAREN LT1 GT1 error SPACE Use the function isdigit (from header) to check for numeric, and isalpha to check for alpha characters, and isspace to check for space characters. If you encounter an “error” during tokenization, your function should return false (see below). All state changes trigger output of an additional token, except for the following: INTEGER to DECIMAL DECIMAL to FRAC ALPHA to ALPHANUM LT1 to LT2 GT1 to GT2 SPACE to any other state In addition, every character of OP_PAREN outputs a token, even if no state change occurs, and LT1-to-LT1, or GT1-to-GT1 outputs a token. This means that "(*)" should tokenize into three tokens: "(", "*", ")", and "<<=" should tokenize into two tokens: "<", "<=".

If you want to see some additional valid/invalid tokenizations, take a look at the test file. You should write your tokenization function as a function bool tokenize(std::string s, std::vector& tokens); You can assume that the vector tokens will be empty when your function is called; your function should either a) Fill the vector with the tokens of the string and return true, if no errors occurred or b) Return false if an error occurred (one of the “error” entries in the above table was encountered).

Testing and submission

A test file is available for download as assign6_test.cpp, or on the server as /usr/local/class/src/assign6_test.cpp. Compile and link this with your .cpp file containing your implementation of the tokenize function. Run ./assign6_test to run the automated tests, or run ./assign6_test interactive to enter an interactive test loop. Here you can enter a string and its tokens will be printed out. As usual, save your work in cs133/assign6/.

Expert's Answer

457 Times Downloaded

Related Questions

. Introgramming & Unix Fall 2018, CRN 44882, Oakland University Homework Assignment 6 - Using Arrays and Functions in C

DescriptionIn this final assignment, the students will demonstrate their ability to apply two ma

. The standard path finding involves finding the (shortest) path from an origin to a destination, typically on a map. This is an

Path finding involves finding a path from A to B. Typically we want the path to have certain properties,such as being the shortest or to avoid going t

. Develop a program to emulate a purchase transaction at a retail store. This program will have two classes, a LineItem class and a Transaction class. The LineItem class will represent an individual

Develop a program to emulate a purchase transaction at a retail store. Thisprogram will have two classes, a LineItem class and a Transaction class. Th

. SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 1 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of

. Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of Sea Ports. Here are the classes and their instance variables we wish to define:

1 Project 2 Introduction - the SeaPort Project series For this set of projects for the course, we wish to simulate some of the aspects of a number of

457 Times Downloaded

Ask This Question To Be Solved By Our ExpertsGet A+ Grade Solution Guaranteed

expert
Atharva PatilComputer science

921 Answers

Hire Me
expert
Chrisantus MakokhaComputer science

565 Answers

Hire Me
expert
AyooluwaEducation

604 Answers

Hire Me
expert
RIZWANAMathematics

971 Answers

Hire Me

Get Free Quote!

262 Experts Online