import { CodeSurfer, CodeSurferColumns, } from "code-surfer"; import { vsDark, github, nightOwl, } from "@code-surfer/themes";

import { Image, Footer } from "mdx-deck";

import Author from "./slides/Author"; import TableOfContents from "./slides/TableOfContents"; import WhyCompiler from "./slides/WhyCompiler";

import AST from './assets/images/AST.png'; // with import import JSFromAST from "./assets/images/JSFromAST.png"; import Generated2 from "./assets/images/Generated2.png"; import CompilerPhases from "./assets/images/CompilerPhases.jpg"; import ErrorMessages from "./assets/images/ErrorMessages.png"; import TypeInference from "./assets/images/TypeInference.png"; import { Row } from "./components/ui/Common";

export const theme = nightOwl;

Made with ❤️ by @dylanirlbeck

Functional Programming, React, and Compilers in ReasonML

Chicago ReasonML Meetup, April 2020

Why I'm doing this talk

I ❤️ functional programming
ReasonML has made me a better developer

1. Want to share my experience from the world of academia 2. I want to help other people just get started building cool things with it

What I hope you take away

An understanding of the core concepts behind functional programming
Ideas of what you might want to build in Reason

1. The concepts exist beyond the language (and how they're implemented in ReasonML) 2. We'll dive into some examples...

ReasonML

Reason

New syntax for OCaml that looks like (and compiles to) JS
"Mostly" functional (OCaml)
100% type coverage, powerful type inference

It's also really, really fast.

Talk about what OCaml (why is it used in compilers)

Type coverage - Every line of code Soundness - Once it compiles, the types are guaranteed to be accurate

The Functional Approach to Programming

Functions are first-class citizens
Pure functions
Immutability

// Reason array (the annotation is not needed!)
let arr: array(int) = [|1, 2, 3|];

// This is a function
let double = elem => elem*2;

let doubledArr = Array.map(double, arr); // [|2, 4, 6|]

let add = (x, y) => x + y;

let addFive = add(5);

let eleven = addFive(6); // 11

let twelve = addFive(7); // 12

let add = (x, y) => x + y;

let addFive = add(5);

let eleven = addFive(6); // 11

let twelve = addFive(7); // 12

let add = (x, y) => x + y;

let addFive = add(5);

let eleven = addFive(6); // 11

let twelve = addFive(7); // 12

Currying and partial application are also conflated

Partial Application: Fixing a number of arguments to a function, producing another function of a smaller arity

Currying - Technique of translating the evaluation of a function that takes multiple arguments into evaluating a sequence of functions, each with a single argument

add above is really syntax sugar for let add = (x) => (y) => x + y;

// Invalid Reason
let incrementValue = value => {
  value++
};

// Valid Reason
let incrementValue = value => {
  value + 1
};

Pure Functions: No side effects occur throughout the function's execution. A side effect is basically a state change in something other than the function that's currently executing it. Also, same result for same input.

Here we're trying to change the state of a variable..

let x = 5;

// Rest of your program

Js.log(x); // 5

Variables will always be of same value + type throughout the program. This is very mathematical in nature (what does it mean to be a variable and function in mathematics).

Flow - Static type checker for JavaScript Hack compiler - Hack extends PHP with static types Infer - Static analyzer for Java, C++, C, and Objective-C (for mobile apps)

What's in a compiler (condensed)

Lexer/Tokenizer - Create a list of tokens from input code
Parser - Turn the tokens into an AST
Code Generation - Generate output code from the AST

Define your frontend and backend (and sometimes your type system)

Frontend: Mini-C
Type System: N/A
Backend: JavaScript

My implementation is based on one by Gary Benhardt (Destroy All Software)...

Lexer/Tokenizer

Let's define the tokens of our language

def f() 1 end

What are our "tokens"?

def

f

(

)

1

end

// This is a variant
type token =
  | DEF     // def
  | IDENT   // f
  | OPAREN  // (
  | CPAREN  // )
  | END     // end
  | INT     // 2,3,etc
  | COMMA;  // ,

Tokens are the smallest pieces that are meaningful

Options of a varaint: Constructors or "tags"

How are we going to consume our input program?

Regular expressions.

//Tokenizer.re

// This is a record
type tokenType = {
  token,
  regExp: Js.Re.t,
};

// ^ means match beginning of input
let defToken = {token: DEF, regExp: [%re "/^(\\bdef\\b)/"]};
let endToken = {token: END, regExp: [%re "/^(\\bend\\b)/"]};
let identToken = {token: IDENT, regExp: [%re "/^(\\b[a-zA-Z]+\\b)/"]};
let intToken = {token: INT, regExp: [%re "/^(\\b[0-9]+\\b)/"]};
let oparenToken = {token: OPAREN, regExp: [%re "/^(\\()/"]};
let cparenToken = {token: CPAREN, regExp: [%re "/^(\\))/"]};
let commaToken = {token: COMMA, regExp: [%re "/^(\\,)/"]};

// These are in a necessary order, i.e. don't want def to be an identifier
let tokenTypes: array(tokenType) = [|
  defToken,
  endToken,
  identToken,
  intToken,
  oparenToken,
  cparenToken,
  commaToken,
|];

// This is a record
type tokenType = {
  token,
  regExp: Js.Re.t,
};

// ^ means match beginning of input
let defToken = {token: DEF, regExp: [%re "/^(\\bdef\\b)/"]};
let endToken = {token: END, regExp: [%re "/^(\\bend\\b)/"]};
let identToken = {token: IDENT, regExp: [%re "/^(\\b[a-zA-Z]+\\b)/"]};
let intToken = {token: INT, regExp: [%re "/^(\\b[0-9]+\\b)/"]};
let oparenToken = {token: OPAREN, regExp: [%re "/^(\\()/"]};
let cparenToken = {token: CPAREN, regExp: [%re "/^(\\))/"]};
let commaToken = {token: COMMA, regExp: [%re "/^(\\,)/"]};

// These are in a necessary order, i.e. don't want def to be an identifier
let tokenTypes: array(tokenType) = [|
  defToken,
  endToken,
  identToken,
  intToken,
  oparenToken,
  cparenToken,
  commaToken,
|];

// This is a record
type tokenType = {
  token,
  regExp: Js.Re.t,
};

// ^ means match beginning of input
let defToken = {token: DEF, regExp: [%re "/^(\\bdef\\b)/"]};
let endToken = {token: END, regExp: [%re "/^(\\bend\\b)/"]};
let identToken = {token: IDENT, regExp: [%re "/^(\\b[a-zA-Z]+\\b)/"]};
let intToken = {token: INT, regExp: [%re "/^(\\b[0-9]+\\b)/"]};
let oparenToken = {token: OPAREN, regExp: [%re "/^(\\()/"]};
let cparenToken = {token: CPAREN, regExp: [%re "/^(\\))/"]};
let commaToken = {token: COMMA, regExp: [%re "/^(\\,)/"]};

// These are in a necessary order, i.e. don't want def to be an identifier
let tokenTypes: array(tokenType) = [|
  defToken,
  endToken,
  identToken,
  intToken,
  oparenToken,
  cparenToken,
  commaToken,
|];

Now we throw it through a tokenizer function and...

// Compiler.re

let program = "def f(x,y,z) 1 end";
let tokens = Tokenizer.make(program);
Js.log(tokens);

[
  { token: 3397029, value: 'def' },
  { token: 895974096, value: 'f' },
  { token: -780845765, value: '(' },
  { token: 895974096, value: 'x' },
  { token: -934581835, value: ',' },
  { token: 895974096, value: 'y' },
  { token: -934581835, value: ',' },
  { token: 895974096, value: 'z' },
  { token: 86829255, value: ')' },
  { token: 3647695, value: '1' },
  { token: 3448763, value: 'end' }
]

Any questions?

On to the parser

Tokens -> Abstract Syntax Tree.

Abstract Syntax Tree - Integer Node

How do we represent the integer literal 1?

VALUE: 1

What is the minimum info needed to represent an integer?

Literals (or constants) are the values we write in a conventional form whose values is obvious; they also do not change in value.

Abstract Syntax Tree - Definition Node

How about the definition def f(...) ... end

DEF
  NAME: "f"
  ARGS: [...]
  BODY:
   ...

type integerNode = {value: int}
and callNode = {
  name: string,
  argExprs: array(exprNode),
}
and varRefNode = {value: string}
and exprNode =
  | CallNode(callNode)
  | IntegerNode(integerNode)
  | VarRefNode(varRefNode);

type defNode = {
  name: string,
  argNames: array(string),
  body: exprNode,
};

type node =
  | DefNode(defNode)
  | ExprNode(exprNode);

type integerNode = {value: int}
and callNode = {
  name: string,
  argExprs: array(exprNode),
}
and varRefNode = {value: string}
and exprNode =
  | CallNode(callNode)
  | IntegerNode(integerNode)
  | VarRefNode(varRefNode);

type defNode = {
  name: string,
  argNames: array(string),
  body: exprNode,
};

type node =
  | DefNode(defNode)
  | ExprNode(exprNode);

type integerNode = {value: int}
and callNode = {
  name: string,
  argExprs: array(exprNode),
}
and varRefNode = {value: string}
and exprNode =
  | CallNode(callNode)
  | IntegerNode(integerNode)
  | VarRefNode(varRefNode);

and defNode = {
  name: string,
  argNames: array(string),
  body: exprNode,
};

type node =
  | DefNode(defNode)
  | ExprNode(exprNode);

Remember, we want to turn this;

def f(x) 1 end

into something like this:

DEF
  NAME: "f"
  ARGS: ["x"]
  BODY:
    INTEGER_LITERAL
      VALUE: 1

let parseInteger = tokens => {
  let (integerToken, newTokens) = consume(tokens, INT);
  let integerNode = {value: int_of_string(integerToken.value)};
  (ExprNode(integerNode), newTokens);
};

let parseInteger = tokens => {
  let (integerToken, newTokens) = consume(tokens, INT);
  let integerNode = {value: int_of_string(integerToken.value)};
  (ExprNode(integerNode), newTokens);
};

let parseInteger = tokens => {
  let (integerToken, newTokens) = consume(tokens, INT);
  let integerNode = {value: int_of_string(integerToken.value)};
  (ExprNode(integerNode), newTokens);
};

let parseInteger = tokens => {
  let (integerToken, newTokens) = consume(tokens, INT);
  let integerNode = {value: int_of_string(integerToken.value)};
  (ExprNode(integerNode), newTokens);
};

Consume gives us back new tokens in a safe way, validates the type is what we say it is.

let parseExpr = tokens =>
  if (peek(tokens, INT, ())) {
    parseInteger(tokens);
  } else {
    parseCall(tokens);
  };

Two options for an expression, integer or function call.

Let's put it all together and create our AST

// Will return a tree
let make = tokens => {
  let rec parse = tokens => {
    switch (Array.length(tokens)) {
    | 0 => [||]
    | _ =>
      let (parsedDef, remainingTokens) = parseDef(tokens);
      Array.concat([[|parsedDef|], parse(remainingTokens)]);
    };
  };
  parse(tokens);
};

// Will return a tree
let make = tokens => {
  let rec parse = tokens => {
    switch (Array.length(tokens)) {
    | 0 => [||]
    | _ =>
      let (parsedDef, remainingTokens) = parseDef(tokens);
      Array.concat([[|parsedDef|], parse(remainingTokens)]);
    };
  };
  parse(tokens);
};

// Will return a tree
let make = tokens => {
  let rec parse = tokens => {
    switch (Array.length(tokens)) {
    | 0 => [||]
    | _ =>
      let (parsedDef, remainingTokens) = parseDef(tokens);
      Array.concat([[|parsedDef|], parse(remainingTokens)]);
    };
  };
  parse(tokens);
};

For an input of

def f(x) ) end

we get this Abstract Syntax Tree:

DefNode(
  {
    name: "f"
    argNames: "x"
    body: ExprNode(
        IntegerNode(
        {
          value: 1        
        }
      )
    )
  } 
)

Questions so far?

Now we're going to generate JavaScript from our input AST

Our generate function will be highly recursive, just like the parser

 let rec generateExpr = exprNode => {
   switch (exprNode) {
    | CallNode(callNode) =>
      let name = callNode.name;
      let argExprs = joinArgExprs(callNode.argExprs);
      {j|$name ($argExprs)|j};
    | IntegerNode(integerNode) => string_of_int(integerNode.value)
    | VarRefNode(varRefNode) => varRefNode.value
    };
  };

 let rec generateExpr = exprNode => {
   switch (exprNode) {
    | CallNode(callNode) =>
      let name = callNode.name;
      let argExprs = joinArgExprs(callNode.argExprs);
      {j|$name ($argExprs)|j};
    | IntegerNode(integerNode) => string_of_int(integerNode.value)
    | VarRefNode(varRefNode) => varRefNode.value
    };
  };

 let rec generateExpr = exprNode => {
   switch (exprNode) {
    | CallNode(callNode) =>
      let name = callNode.name;
      let argExprs = joinArgExprs(callNode.argExprs);
      {j|$name ($argExprs)|j};
    | IntegerNode(integerNode) => string_of_int(integerNode.value)
    | VarRefNode(varRefNode) => varRefNode.value
    };
  };

  let rec generate_ = node => {
    switch (node) {
    | DefNode(defNode) =>
      let {name, argNames, body} = defNode;
      let argNamesAsString = joinArgNames(argNames);
      // recursively generate body
      let bodyAsString = generate_(ExprNode(defNode.body));
      {j|function $name($argNamesAsString) { return $bodyAsString }|j};
    | ExprNode(exprNode) => generateExpr(exprNode)
    };
  };

  let rec generate_ = node => {
    switch (node) {
    | DefNode(defNode) =>
      let {name, argNames, body} = defNode;
      let argNamesAsString = joinArgNames(argNames);
      // recursively generate body
      let bodyAsString = generate_(ExprNode(defNode.body));
      {j|function $name($argNamesAsString) { return $bodyAsString }|j};
    | ExprNode(exprNode) => generateExpr(exprNode)
    };
  };

  let rec generate_ = node => {
    switch (node) {
    | DefNode(defNode) =>
      let {name, argNames, body} = defNode;
      let argNamesAsString = joinArgNames(argNames);
      // recursively generate body
      let bodyAsString = generate_(ExprNode(defNode.body));
      {j|function $name($argNamesAsString) { return $bodyAsString }|j};
    | ExprNode(exprNode) => generateExpr(exprNode)
    };
  };

Let's put it all together with the compiler

// Compiler.re

let compile = program => {
  program
  ->Tokenizer.make
  ->Parser.make
  ->Generator.make;
};

compile("def f(x) 1 end");

let compile = program => {
  program
  ->Tokenizer.make
  ->Parser.make
  ->Generator.make;
};

compile("def f(x, y) add(x, y) end");

let compile = program => {
  program
  ->Tokenizer.make
  ->Parser.make
  ->Generator.make;
};

compile("def f(x, y) add(x, y) end");

let compile = program => {
  program
  ->Tokenizer.make
  ->Parser.make
  ->Generator.make;
};

compile("def f(x, y) add(x, y) end");

let compile = program => {
  program
  ->Tokenizer.make
  ->Parser.make
  ->Generator.make;
};

compile("def f(x, y) add(x, y) end");

And that's our compiler!

compile("def f(x) 1 end");

Our mini-compiler also supports arbitrary nested expressions

compile("def f(x, y) add(x,add(x,y)) end")

Recap

1. Structure of a compiler (Tokenizer -> Parser -> Generator)

1. Talk about where a type system would fit into the flow of a compiler

2. Features of Reason as a language

Variants/Pattern matching: type something = This | That;
Records: type somethingElse = {value: int, name: string}
Type Inferencing

3. Reason/OCaml Type system

Type Inferencing

Contrast this with JavaScript

3. Reason/OCaml Type system cont'd

Error Messages

We've talked about Reason as a language, so what can we do with it?

ReasonReact!

Core concepts of React (from Jordan Walke himself!)

Minimize application state
Minimize side effects

1. Reason, and FP languages in general, minimize state because you can't change state globally. 2. Again, by definition, FP languages are mostly side-effect free.

Jordan Walke did a talk, invented React and Reason etc.

[@react.component]
let make = (~name) => {
  let (count, setCount) = React.useState(() => 0);

  <div>
    <p> {React.string(name ++ " clicked " ++ string_of_int(count) ++ " times")} </p>
    <button onClick={_ => setCount(_ => count + 1)}>
      {React.string("Click me")}
    </button>
  </div>
};

[@react.component]
let make = (~name) => {
  let (count, setCount) = React.useState(() => 0);

  <div>
    <p> {React.string(name ++ " clicked " ++ string_of_int(count) ++ " times")} </p>
    <button onClick={_ => setCount(_ => count + 1)}>
      {React.string("Click me")}
    </button>
  </div>
};

[@react.component]
let make = (~name) => {
  let (count, setCount) = React.useState(() => 0);

  <div>
    <p> {React.string(name ++ " clicked " ++ string_of_int(count) ++ " times")} </p>
    <button onClick={_ => setCount(_ => count + 1)}>
      {React.string("Click me")}
    </button>
  </div>
};

[@react.component]
let make = (~name) => {
  let (count, setCount) = React.useState(() => 0);

  <div>
    <p> {React.string(name ++ " clicked " ++ string_of_int(count) ++ " times")} </p>
    <button onClick={_ => setCount(_ => count + 1)}>
      {React.string("Click me")}
    </button>
  </div>
};

[@react.component]
let make = (~name) => {
  let (count, setCount) = React.useState(() => 0);

  <div>
    <p> {React.string(name ++ " clicked " ++ string_of_int(count) ++ " times")} </p>
    <button onClick={_ => setCount(_ => count + 1)}>
      {React.string("Click me")}
    </button>
  </div>
};

Main differences between this and JS implementation: * Named arguments for props * Decorator attribute that tells ReasonReact you want to compile this into a function that takes a JS object as props which is how React works * React.string wrapper, conversion to an int

Main practical benefits of Reason + React

Types - ease of refactoring
Explicit handling of elements React.string("hello")
First Class Language Tools (immutability, switch statements)

1. Refactoring - this comes from experience 2. This is a nice sanity check 3. Again, minimize application state, switches help for rendering different HTML

// www.hello.com/book/10/edit?name=Jane#author

let url = ReasonReactRouter.useUrl();
switch (url.path) {
| ["book", id, "edit"] => handleBookEdit(id)
| ["book", id] => getBook(id)
| ["book", id, _] => noSuchBookOperation()
| [] => showMainPage()
| ["shop"] | ["shop", "index"] => showShoppingPage()
| ["shop", ...rest] =>
    /* e.g. "shop/cart/10", but let "cart/10" be handled by another function */
  nestedMatch(rest)
| _ => showNotFoundPage()
};
}

// www.hello.com/book/10/edit?name=Jane#author

let url = ReasonReactRouter.useUrl();
switch (url.path) {
| ["book", id, "edit"] => handleBookEdit(id)
| ["book", id] => getBook(id)
| ["book", id, _] => noSuchBookOperation()
| [] => showMainPage()
| ["shop"] | ["shop", "index"] => showShoppingPage()
| ["shop", ...rest] =>
    /* e.g. "shop/cart/10", but let "cart/10" be handled by another function */
  nestedMatch(rest)
| _ => showNotFoundPage()
};
}

The entire implementation is around 20 lines!

open ApolloHooks

module UserQuery = [%Graphql {|
  query UserQuery {
    currentUser {
      name
    }
  }
|}];

[@react.component]
let make = () => {
  /* Both variant and records available */
  let (simple, _full) = useQuery(UserQuery.definition);

  <div>
  {
    switch(simple) {
      | Loading => <p>{React.string("Loading...")}</p>
      | Data(data) =>
        <p>{React.string(data##currentUser##name)}</p>
      | NoData
      | Error(_) => <p>{React.string("Get off my lawn!")}</p>
    }
  }
  </div>
}

open ApolloHooks

module UserQuery = [%Graphql {|
  query UserQuery {
    currentUser {
      name
    }
  }
|}];

[@react.component]
let make = () => {
  /* Both variant and records available */
  let (simple, _full) = useQuery(UserQuery.definition);

  <div>
  {
    switch(simple) {
      | Loading => <p>{React.string("Loading...")}</p>
      | Data(data) =>
        <p>{React.string(data##currentUser##name)}</p>
      | NoData
      | Error(_) => <p>{React.string("Get off my lawn!")}</p>
    }
  }
  </div>
}

open ApolloHooks

module UserQuery = [%Graphql {|
  query UserQuery {
    currentUser {
      name
    }
  }
|}];

[@react.component]
let make = () => {
  /* Both variant and records available */
  let (simple, _full) = useQuery(UserQuery.definition);

  <div>
  {
    switch(simple) {
      | Loading => <p>{React.string("Loading...")}</p>
      | Data(data) =>
        <p>{React.string(data##currentUser##name)}</p>
      | NoData
      | Error(_) => <p>{React.string("Get off my lawn!")}</p>
    }
  }
  </div>
}

open ApolloHooks

module UserQuery = [%Graphql {|
  query UserQuery {
    currentUser {
      name
    }
  }
|}];

[@react.component]
let make = () => {
  /* Both variant and records available */
  let (simple, _full) = useQuery(UserQuery.definition);

  <div>
  {
    switch(simple) {
      | Loading => <p>{React.string("Loading...")}</p>
      | Data(data) =>
        <p>{React.string(data##currentUser##name)}</p>
      | NoData
      | Error(_) => <p>{React.string("Get off my lawn!")}</p>
    }
  }
  </div>
}

So what did we (hopefully) learn?

Functional programming concepts
Deep dive into the language by writing a mini-compiler
Examples of how to use Reason + React to build powerful and extensible web apps

1. Functions as first-class citizens, pure functions, immutability 2. Lexer->Parser->CodeGenerator 3. Mention ReasonReactNative

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Functional Programming, React, and Compilers in ReasonML

Why I'm doing this talk

What I hope you take away

ReasonML

Reason

The Functional Approach to Programming

What's in a compiler (condensed)

Define your frontend and backend (and sometimes your type system)

Lexer/Tokenizer

Any questions?

On to the parser

Abstract Syntax Tree - Integer Node

Abstract Syntax Tree - Definition Node

Let's put it all together and create our AST

Questions so far?

Now we're going to generate JavaScript from our input AST

Our generate function will be highly recursive, just like the parser

Let's put it all together with the compiler

And that's our compiler!

Our mini-compiler also supports arbitrary nested expressions

Recap

1. Structure of a compiler (Tokenizer -> Parser -> Generator)

2. Features of Reason as a language

3. Reason/OCaml Type system

3. Reason/OCaml Type system cont'd

We've talked about Reason as a language, so what can we do with it?

ReasonReact!

Core concepts of React (from Jordan Walke himself!)

Main practical benefits of Reason + React

So what did we (hopefully) learn?

Thank you + Q&A

Sources

FilesExpand file tree

deck.mdx

Latest commit

History

deck.mdx

File metadata and controls

Functional Programming, React, and Compilers in ReasonML

Why I'm doing this talk

What I hope you take away

ReasonML

Reason

The Functional Approach to Programming

What's in a compiler (condensed)

Define your frontend and backend (and sometimes your type system)

Lexer/Tokenizer

Any questions?

On to the parser

Abstract Syntax Tree - Integer Node

Abstract Syntax Tree - Definition Node

Let's put it all together and create our AST

Questions so far?

Now we're going to generate JavaScript from our input AST

Our generate function will be highly recursive, just like the parser

Let's put it all together with the compiler

And that's our compiler!

Our mini-compiler also supports arbitrary nested expressions

Recap

1. Structure of a compiler (Tokenizer -> Parser -> Generator)

2. Features of Reason as a language

3. Reason/OCaml Type system

3. Reason/OCaml Type system cont'd

We've talked about Reason as a language, so what can we do with it?

ReasonReact!

Core concepts of React (from Jordan Walke himself!)

Main practical benefits of Reason + React

So what did we (hopefully) learn?

Thank you + Q&A

Sources