reveal.js

# Parsers

---

CS 135 // 2021-04-08

## Administrivia
-

# Questions
## ...about anything?

# Grammars

## What is a grammar?
> A system of rules that defines the structure of a language
> 
> Merriam-Webster Dictionary

## English Grammar
- Consider the following sentence:
    ```text
    The dog chased the cat.
    ```
- 
 Sentences in a language conform to certain rules:
    ```bnf[1|2|3|4|5|6|1-7]
    <sentence>    ::= <noun-phrase> <verb> <noun-phrase>
    <noun-phrase> ::= <article> <noun>
    <verb-phrase> ::= <verb> <noun-phrase>
    <article>     ::= a | the
    <noun>        ::= cat | dog
    <verb>        ::= chased | saw
    ```
- 
 This grammar is in **Backus–Naur form** and is the standard for defining programming language syntax

## English Grammar Tree
```text
    The dog chased the cat.
```

---

<div class='jekyll-diagrams diagrams graphviz'>
  

<svg width="406pt" height="260pt"
 viewBox="0.00 0.00 406.00 260.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 256)">
<title>tree</title>
<polygon fill="white" stroke="transparent" points="-4,4 -4,-256 402,-256 402,4 -4,4"/>

<g id="node1" class="node">
<title>s</title>
<polygon fill="none" stroke="black" points="240,-252 160,-252 160,-216 240,-216 240,-252"/>
<text text-anchor="middle" x="200" y="-230.3" font-family="Times,serif" font-size="14.00">sentence</text>
</g>

<g id="node2" class="node">
<title>nf1</title>
<polygon fill="none" stroke="black" points="154.5,-180 49.5,-180 49.5,-144 154.5,-144 154.5,-180"/>
<text text-anchor="middle" x="102" y="-158.3" font-family="Times,serif" font-size="14.00">noun-phrase</text>
</g>

<g id="edge1" class="edge">
<title>s->nf1</title>
<path fill="none" stroke="black" d="M176.03,-215.88C163.45,-206.89 147.87,-195.76 134.25,-186.03"/>
<polygon fill="black" stroke="black" points="136.24,-183.15 126.07,-180.19 132.17,-188.85 136.24,-183.15"/>
</g>

<g id="node3" class="node">
<title>v</title>
<polygon fill="none" stroke="black" points="227,-180 173,-180 173,-144 227,-144 227,-180"/>
<text text-anchor="middle" x="200" y="-158.3" font-family="Times,serif" font-size="14.00">verb</text>
</g>

<g id="edge2" class="edge">
<title>s->v</title>
<path fill="none" stroke="black" d="M200,-215.7C200,-207.98 200,-198.71 200,-190.11"/>
<polygon fill="black" stroke="black" points="203.5,-190.1 200,-180.1 196.5,-190.1 203.5,-190.1"/>
</g>

<g id="node4" class="node">
<title>nf2</title>
<polygon fill="none" stroke="black" points="350.5,-180 245.5,-180 245.5,-144 350.5,-144 350.5,-180"/>
<text text-anchor="middle" x="298" y="-158.3" font-family="Times,serif" font-size="14.00">noun-phrase</text>
</g>

<g id="edge3" class="edge">
<title>s->nf2</title>
<path fill="none" stroke="black" d="M223.97,-215.88C236.55,-206.89 252.13,-195.76 265.75,-186.03"/>
<polygon fill="black" stroke="black" points="267.83,-188.85 273.93,-180.19 263.76,-183.15 267.83,-188.85"/>
</g>

<g id="node5" class="node">
<title>a1</title>
<polygon fill="none" stroke="black" points="62,-108 0,-108 0,-72 62,-72 62,-108"/>
<text text-anchor="middle" x="31" y="-86.3" font-family="Times,serif" font-size="14.00">article</text>
</g>

<g id="edge4" class="edge">
<title>nf1->a1</title>
<path fill="none" stroke="black" d="M84.45,-143.7C75.77,-135.14 65.14,-124.66 55.65,-115.3"/>
<polygon fill="black" stroke="black" points="57.93,-112.63 48.35,-108.1 53.01,-117.62 57.93,-112.63"/>
</g>

<g id="node6" class="node">
<title>n1</title>
<polygon fill="none" stroke="black" points="134,-108 80,-108 80,-72 134,-72 134,-108"/>
<text text-anchor="middle" x="107" y="-86.3" font-family="Times,serif" font-size="14.00">noun</text>
</g>

<g id="edge5" class="edge">
<title>nf1->n1</title>
<path fill="none" stroke="black" d="M103.24,-143.7C103.79,-135.98 104.45,-126.71 105.06,-118.11"/>
<polygon fill="black" stroke="black" points="108.56,-118.33 105.78,-108.1 101.57,-117.83 108.56,-118.33"/>
</g>

<g id="node7" class="node">
<title>chase</title>
<ellipse fill="none" stroke="black" cx="200" cy="-90" rx="42.79" ry="18"/>
<text text-anchor="middle" x="200" y="-86.3" font-family="Times,serif" font-size="14.00">chased</text>
</g>

<g id="edge6" class="edge">
<title>v->chase</title>
<path fill="none" stroke="black" d="M200,-143.7C200,-135.98 200,-126.71 200,-118.11"/>
<polygon fill="black" stroke="black" points="203.5,-118.1 200,-108.1 196.5,-118.1 203.5,-118.1"/>
</g>

<g id="node8" class="node">
<title>a2</title>
<polygon fill="none" stroke="black" points="326,-108 264,-108 264,-72 326,-72 326,-108"/>
<text text-anchor="middle" x="295" y="-86.3" font-family="Times,serif" font-size="14.00">article</text>
</g>

<g id="edge7" class="edge">
<title>nf2->a2</title>
<path fill="none" stroke="black" d="M297.26,-143.7C296.93,-135.98 296.53,-126.71 296.16,-118.11"/>
<polygon fill="black" stroke="black" points="299.66,-117.95 295.73,-108.1 292.66,-118.25 299.66,-117.95"/>
</g>

<g id="node9" class="node">
<title>n2</title>
<polygon fill="none" stroke="black" points="398,-108 344,-108 344,-72 398,-72 398,-108"/>
<text text-anchor="middle" x="371" y="-86.3" font-family="Times,serif" font-size="14.00">noun</text>
</g>

<g id="edge8" class="edge">
<title>nf2->n2</title>
<path fill="none" stroke="black" d="M316.04,-143.7C325.06,-135.05 336.12,-124.45 345.94,-115.03"/>
<polygon fill="black" stroke="black" points="348.37,-117.55 353.16,-108.1 343.52,-112.5 348.37,-117.55"/>
</g>

<g id="node10" class="node">
<title>The</title>
<ellipse fill="none" stroke="black" cx="31" cy="-18" rx="27.9" ry="18"/>
<text text-anchor="middle" x="31" y="-14.3" font-family="Times,serif" font-size="14.00">The</text>
</g>

<g id="edge9" class="edge">
<title>a1->The</title>
<path fill="none" stroke="black" d="M31,-71.7C31,-63.98 31,-54.71 31,-46.11"/>
<polygon fill="black" stroke="black" points="34.5,-46.1 31,-36.1 27.5,-46.1 34.5,-46.1"/>
</g>

<g id="node11" class="node">
<title>dog</title>
<ellipse fill="none" stroke="black" cx="107" cy="-18" rx="27.9" ry="18"/>
<text text-anchor="middle" x="107" y="-14.3" font-family="Times,serif" font-size="14.00">dog</text>
</g>

<g id="edge10" class="edge">
<title>n1->dog</title>
<path fill="none" stroke="black" d="M107,-71.7C107,-63.98 107,-54.71 107,-46.11"/>
<polygon fill="black" stroke="black" points="110.5,-46.1 107,-36.1 103.5,-46.1 110.5,-46.1"/>
</g>

<g id="node12" class="node">
<title>the</title>
<ellipse fill="none" stroke="black" cx="295" cy="-18" rx="27" ry="18"/>
<text text-anchor="middle" x="295" y="-14.3" font-family="Times,serif" font-size="14.00">the</text>
</g>

<g id="edge11" class="edge">
<title>a2->the</title>
<path fill="none" stroke="black" d="M295,-71.7C295,-63.98 295,-54.71 295,-46.11"/>
<polygon fill="black" stroke="black" points="298.5,-46.1 295,-36.1 291.5,-46.1 298.5,-46.1"/>
</g>

<g id="node13" class="node">
<title>cat</title>
<ellipse fill="none" stroke="black" cx="371" cy="-18" rx="27" ry="18"/>
<text text-anchor="middle" x="371" y="-14.3" font-family="Times,serif" font-size="14.00">cat</text>
</g>

<g id="edge12" class="edge">
<title>n2->cat</title>
<path fill="none" stroke="black" d="M371,-71.7C371,-63.98 371,-54.71 371,-46.11"/>
<polygon fill="black" stroke="black" points="374.5,-46.1 371,-36.1 367.5,-46.1 374.5,-46.1"/>
</g>
</g>
</svg>

</div>

## Grammar for Expressions
- Syntactic structure for arithmetic expressions can be formalized with a **grammar**:

```bnf
<exp>   ::= <exp> + <exp> | <exp> * <exp> | ( <exp> ) | <nat>
<nat>   ::= <digit>{<digit>}*
<digit> ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
```

- Consider the parse tree for the expression:
    * `(4 + 2) * 3`
- 
 Most grammars are complicated and their specification is long
- 
 An **abstract syntax tree** is sometimes easier to work with than a parse tree

</div>
<div>

<div class='jekyll-diagrams diagrams graphviz'>
  

<svg width="294pt" height="476pt"
 viewBox="0.00 0.00 293.99 476.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 472)">
<title>tree</title>
<polygon fill="white" stroke="transparent" points="-4,4 -4,-472 289.99,-472 289.99,4 -4,4"/>

<g id="node1" class="node">
<title>e1</title>
<polygon fill="none" stroke="black" points="205.5,-468 151.5,-468 151.5,-432 205.5,-432 205.5,-468"/>
<text text-anchor="middle" x="178.5" y="-446.3" font-family="Times,serif" font-size="14.00">exp</text>
</g>

<g id="node2" class="node">
<title>e2</title>
<polygon fill="none" stroke="black" points="132.5,-396 78.5,-396 78.5,-360 132.5,-360 132.5,-396"/>
<text text-anchor="middle" x="105.5" y="-374.3" font-family="Times,serif" font-size="14.00">exp</text>
</g>

<g id="edge1" class="edge">
<title>e1->e2</title>
<path fill="none" stroke="black" d="M160.45,-431.7C151.44,-423.05 140.38,-412.45 130.55,-403.03"/>
<polygon fill="black" stroke="black" points="132.97,-400.5 123.33,-396.1 128.13,-405.55 132.97,-400.5"/>
</g>

<g id="node3" class="node">
<title>*</title>
<ellipse fill="none" stroke="black" cx="178.5" cy="-378" rx="27" ry="18"/>
<text text-anchor="middle" x="178.5" y="-374.3" font-family="Times,serif" font-size="14.00">*</text>
</g>

<g id="edge2" class="edge">
<title>e1->*</title>
<path fill="none" stroke="black" d="M178.5,-431.7C178.5,-423.98 178.5,-414.71 178.5,-406.11"/>
<polygon fill="black" stroke="black" points="182,-406.1 178.5,-396.1 175,-406.1 182,-406.1"/>
</g>

<g id="node4" class="node">
<title>e3</title>
<polygon fill="none" stroke="black" points="277.5,-396 223.5,-396 223.5,-360 277.5,-360 277.5,-396"/>
<text text-anchor="middle" x="250.5" y="-374.3" font-family="Times,serif" font-size="14.00">exp</text>
</g>

<g id="edge3" class="edge">
<title>e1->e3</title>
<path fill="none" stroke="black" d="M196.29,-431.7C205.1,-423.14 215.88,-412.66 225.5,-403.3"/>
<polygon fill="black" stroke="black" points="228.17,-405.58 232.9,-396.1 223.29,-400.57 228.17,-405.58"/>
</g>

<g id="node5" class="node">
<title>(</title>
<ellipse fill="none" stroke="black" cx="32.5" cy="-306" rx="27" ry="18"/>
<text text-anchor="middle" x="32.5" y="-302.3" font-family="Times,serif" font-size="14.00">(</text>
</g>

<g id="edge4" class="edge">
<title>e2->(</title>
<path fill="none" stroke="black" d="M87.45,-359.7C77.53,-350.18 65.14,-338.3 54.64,-328.23"/>
<polygon fill="black" stroke="black" points="57.03,-325.67 47.39,-321.28 52.18,-330.73 57.03,-325.67"/>
</g>

<g id="node6" class="node">
<title>e4</title>
<polygon fill="none" stroke="black" points="131.5,-324 77.5,-324 77.5,-288 131.5,-288 131.5,-324"/>
<text text-anchor="middle" x="104.5" y="-302.3" font-family="Times,serif" font-size="14.00">exp</text>
</g>

<g id="edge5" class="edge">
<title>e2->e4</title>
<path fill="none" stroke="black" d="M105.25,-359.7C105.14,-351.98 105.01,-342.71 104.88,-334.11"/>
<polygon fill="black" stroke="black" points="108.38,-334.05 104.74,-324.1 101.38,-334.15 108.38,-334.05"/>
</g>

<g id="node7" class="node">
<title>)</title>
<ellipse fill="none" stroke="black" cx="176.5" cy="-306" rx="27" ry="18"/>
<text text-anchor="middle" x="176.5" y="-302.3" font-family="Times,serif" font-size="14.00">)</text>
</g>

<g id="edge6" class="edge">
<title>e2->)</title>
<path fill="none" stroke="black" d="M123.05,-359.7C132.6,-350.28 144.51,-338.53 154.66,-328.53"/>
<polygon fill="black" stroke="black" points="157.35,-330.79 162.01,-321.28 152.44,-325.81 157.35,-330.79"/>
</g>

<g id="node11" class="node">
<title>n1</title>
<ellipse fill="none" stroke="black" cx="252.5" cy="-306" rx="27" ry="18"/>
<text text-anchor="middle" x="252.5" y="-302.3" font-family="Times,serif" font-size="14.00">nat</text>
</g>

<g id="edge7" class="edge">
<title>e3->n1</title>
<path fill="none" stroke="black" d="M250.99,-359.7C251.21,-351.98 251.48,-342.71 251.72,-334.11"/>
<polygon fill="black" stroke="black" points="255.22,-334.2 252.01,-324.1 248.22,-334 255.22,-334.2"/>
</g>

<g id="node8" class="node">
<title>e5</title>
<polygon fill="none" stroke="black" points="59.5,-252 5.5,-252 5.5,-216 59.5,-216 59.5,-252"/>
<text text-anchor="middle" x="32.5" y="-230.3" font-family="Times,serif" font-size="14.00">exp</text>
</g>

<g id="edge8" class="edge">
<title>e4->e5</title>
<path fill="none" stroke="black" d="M86.7,-287.7C77.89,-279.14 67.11,-268.66 57.5,-259.3"/>
<polygon fill="black" stroke="black" points="59.7,-256.57 50.09,-252.1 54.82,-261.58 59.7,-256.57"/>
</g>

<g id="node9" class="node">
<title>+</title>
<ellipse fill="none" stroke="black" cx="104.5" cy="-234" rx="27" ry="18"/>
<text text-anchor="middle" x="104.5" y="-230.3" font-family="Times,serif" font-size="14.00">+</text>
</g>

<g id="edge9" class="edge">
<title>e4->+</title>
<path fill="none" stroke="black" d="M104.5,-287.7C104.5,-279.98 104.5,-270.71 104.5,-262.11"/>
<polygon fill="black" stroke="black" points="108,-262.1 104.5,-252.1 101,-262.1 108,-262.1"/>
</g>

<g id="node10" class="node">
<title>e6</title>
<polygon fill="none" stroke="black" points="203.5,-252 149.5,-252 149.5,-216 203.5,-216 203.5,-252"/>
<text text-anchor="middle" x="176.5" y="-230.3" font-family="Times,serif" font-size="14.00">exp</text>
</g>

<g id="edge10" class="edge">
<title>e4->e6</title>
<path fill="none" stroke="black" d="M122.29,-287.7C131.1,-279.14 141.88,-268.66 151.5,-259.3"/>
<polygon fill="black" stroke="black" points="154.17,-261.58 158.9,-252.1 149.29,-256.57 154.17,-261.58"/>
</g>

<g id="node12" class="node">
<title>n2</title>
<ellipse fill="none" stroke="black" cx="32.5" cy="-162" rx="27" ry="18"/>
<text text-anchor="middle" x="32.5" y="-158.3" font-family="Times,serif" font-size="14.00">nat</text>
</g>

<g id="edge11" class="edge">
<title>e5->n2</title>
<path fill="none" stroke="black" d="M32.5,-215.7C32.5,-207.98 32.5,-198.71 32.5,-190.11"/>
<polygon fill="black" stroke="black" points="36,-190.1 32.5,-180.1 29,-190.1 36,-190.1"/>
</g>

<g id="node13" class="node">
<title>n3</title>
<ellipse fill="none" stroke="black" cx="176.5" cy="-162" rx="27" ry="18"/>
<text text-anchor="middle" x="176.5" y="-158.3" font-family="Times,serif" font-size="14.00">nat</text>
</g>

<g id="edge12" class="edge">
<title>e6->n3</title>
<path fill="none" stroke="black" d="M176.5,-215.7C176.5,-207.98 176.5,-198.71 176.5,-190.11"/>
<polygon fill="black" stroke="black" points="180,-190.1 176.5,-180.1 173,-190.1 180,-190.1"/>
</g>

<g id="node14" class="node">
<title>d1</title>
<ellipse fill="none" stroke="black" cx="253.5" cy="-234" rx="32.49" ry="18"/>
<text text-anchor="middle" x="253.5" y="-230.3" font-family="Times,serif" font-size="14.00">digit</text>
</g>

<g id="edge13" class="edge">
<title>n1->d1</title>
<path fill="none" stroke="black" d="M252.74,-287.7C252.85,-279.98 252.99,-270.71 253.11,-262.11"/>
<polygon fill="black" stroke="black" points="256.61,-262.15 253.25,-252.1 249.61,-262.05 256.61,-262.15"/>
</g>

<g id="node15" class="node">
<title>d2</title>
<ellipse fill="none" stroke="black" cx="32.5" cy="-90" rx="32.49" ry="18"/>
<text text-anchor="middle" x="32.5" y="-86.3" font-family="Times,serif" font-size="14.00">digit</text>
</g>

<g id="edge14" class="edge">
<title>n2->d2</title>
<path fill="none" stroke="black" d="M32.5,-143.7C32.5,-135.98 32.5,-126.71 32.5,-118.11"/>
<polygon fill="black" stroke="black" points="36,-118.1 32.5,-108.1 29,-118.1 36,-118.1"/>
</g>

<g id="node16" class="node">
<title>d3</title>
<ellipse fill="none" stroke="black" cx="176.5" cy="-90" rx="32.49" ry="18"/>
<text text-anchor="middle" x="176.5" y="-86.3" font-family="Times,serif" font-size="14.00">digit</text>
</g>

<g id="edge15" class="edge">
<title>n3->d3</title>
<path fill="none" stroke="black" d="M176.5,-143.7C176.5,-135.98 176.5,-126.71 176.5,-118.11"/>
<polygon fill="black" stroke="black" points="180,-118.1 176.5,-108.1 173,-118.1 180,-118.1"/>
</g>

<g id="node17" class="node">
<title>3</title>
<ellipse fill="none" stroke="black" cx="253.5" cy="-162" rx="27" ry="18"/>
<text text-anchor="middle" x="253.5" y="-158.3" font-family="Times,serif" font-size="14.00">3</text>
</g>

<g id="edge16" class="edge">
<title>d1->3</title>
<path fill="none" stroke="black" d="M253.5,-215.7C253.5,-207.98 253.5,-198.71 253.5,-190.11"/>
<polygon fill="black" stroke="black" points="257,-190.1 253.5,-180.1 250,-190.1 257,-190.1"/>
</g>

<g id="node18" class="node">
<title>4</title>
<ellipse fill="none" stroke="black" cx="32.5" cy="-18" rx="27" ry="18"/>
<text text-anchor="middle" x="32.5" y="-14.3" font-family="Times,serif" font-size="14.00">4</text>
</g>

<g id="edge17" class="edge">
<title>d2->4</title>
<path fill="none" stroke="black" d="M32.5,-71.7C32.5,-63.98 32.5,-54.71 32.5,-46.11"/>
<polygon fill="black" stroke="black" points="36,-46.1 32.5,-36.1 29,-46.1 36,-46.1"/>
</g>

<g id="node19" class="node">
<title>2</title>
<ellipse fill="none" stroke="black" cx="176.5" cy="-18" rx="27" ry="18"/>
<text text-anchor="middle" x="176.5" y="-14.3" font-family="Times,serif" font-size="14.00">2</text>
</g>

<g id="edge18" class="edge">
<title>d3->2</title>
<path fill="none" stroke="black" d="M176.5,-71.7C176.5,-63.98 176.5,-54.71 176.5,-46.11"/>
<polygon fill="black" stroke="black" points="180,-46.1 176.5,-36.1 173,-46.1 180,-46.1"/>
</g>
</g>
</svg>

</div>

</div>
</div>

## Writing a Grammar
- Let's start writing a grammar for a C-like language:

```c
int count = 0;
while (count < 100) {
    if (count % 2 == 0) {
        print("foo");
    } else {
        print("bar");
    }
}
```

## Partial BNF Grammar
```bnf
<statement> ::= <expr> | <block> | <if> | <while>
<expr>      ::= {<expr>}? ;
<block>     ::= { {<statement>}* }
<if>        ::= if (<expr>) <statement>
<if>        ::= if (<expr>) <statement> else <statement>
<while>     ::= while (<expr) <statement>
```

# Parser Generators

## Tools for Parsing

- Many tools exist to turn a grammar into a lexer/parser (ANTLR, Yacc, JavaCC)
- 
 Commonly consists of two parts:
    1. 
 **Lexer:** groups characters into *tokens*
    2. 
 **Parser:** uses grammar to produce parse tree
- 
 We could spend an entire course discussing the nuances of lexing and parsing...
- 
 Instead, we are going to focus entirely on how to actually *interpret* a parse tree
- 
 As a result, we are going to use a Haskell library to help us with parsing

</div>
<div>

![Box diagram for parser](https://upload.wikimedia.org/wikipedia/commons/d/d6/Parser_Flow%D5%B8.gif)

</div>
</div>

## Types of Parsers
- **Top-Down Parsers**
    + Aka "recursive descent parsers"
    + 
 Start at the top-level and repeatedly apply rules until the target string is reached
    + 
 LL parsers are in this category
- 
 **Bottom-Up Parsers**
    + Start with the target string and apply the rules in reverse to build up to the top-level rule
    + 
 LR parsers are in this category

## Lookahead
- For efficiency, some parsers include a **lookahead**
    + The parser knows about future tokens as well as the current token when making decisions
-  
 This can help with efficiency for some grammars
- 
 You may see terms "LR(k) parser" which is an LR parser that uses a lookahead of k

# Monadic Parsing

## Monadic Parsers
- Monadic parsers are commonly used in functional programming languages
    + They are top-down parsers
- 
 These are commonly used for domain-specific languages rather than general-purpose languages
    + 
 They are less efficient than their optimized LL counterparts
    + 
 However, they are extremely easy to use!

## Hutton's Parser Monad
- We will be using Hutton's `Parser` monad covered a few weeks ago

```haskell
newtype Parser a = P { runParser :: String -> [(a,String)] }
```

- 
 Intuitively, a parser is a function from a `String` to some value of type `a`
- 
 Having a list of possible parsings allows us to handle errors and multiple possible parsings

## Hutton's Parser Monad
- Since `Parser a` is a monad, we can easily sequence parsers together
- 
 It also supports the `Alternative` typeclass:
    ```haskell
    class Applicative m => Alternative m where
      empty :: m a
      (<|>) :: m a -> m a -> m a
    ```

## Using the Parser Monad
- Consider the following grammar for arithmetic:
    ```bnf
    <expr>    ::= <term> + <expr> | <term>
    <term>    ::= <factor> * <term> | <factor>
    <factor>  ::= ( <expr> ) | <natural>
    <natural> ::= 0 | 1 | 2 | ...
    ```
- 
 Let's implement this in Haskell