Building JavaScript Tools

Greg Franko

2/12/2014

Use arrow keys to navigate

About The Speaker

Greg Franko

JavaScript Engineer at AddThis

Open Source Developer

Author

Speaker

What Types of Tools Are We Talking About?

Static Code Analysis

What Do These Tools Typically Do?

Analyze Code

Performance, Maintainability, Validation

Generate Code For

Transpilers, Minifiers, Visualizers

Could You Provide Some Examples?

UglifyJS

Compressor/Minifier That Reduces The Size Of Your JS Code

JSHint

Helps Detect Problems In Your JS Code Before It Is Too Late

CSS Lint

Helps Detect Problems In Your CSS Code Before It Is Too Late

CoffeeScript

Language That Compiles To JavaScript

Plato

Code Complexity Visualizations

RequireJS Optimizer

Concatenates AMD Projects Into A Single File

AMDClean

Converts AMD code To Standard JavaScript

Browserify

Concatenates CommonJS Projects Into A Single File For The Browser

Have You Used A Static Code Analysis Tool?

You Probably Have

Have You Written A Static Code Analysis Tool?

You Probably Haven't

Why Not?

• I have a theory...

You Don't Know Where To Start

Let's Change That

Static Code Analysis Basics

Let's Learn

It Starts With A String Of Code

var code = 'function jqcon() {}';

That Is Converted To An Object

{
  "type": "FunctionDeclaration",
  "id": {
    "type": "Identifier",
    "name": "jqcon"
  },
  "params": [],
  "body": {
    "type": "BlockStatement",
    "body": []
  },
  "expression": false
}

This Object Is Called an Abstract Syntax Tree (AST)

Abstract Syntax Tree

• An Object (Tree) That Represents The Structure Of Your Code

• Does Not Store Everything - It is abstract

  e.g. Does not store that a JS function starts/ends with open/close curly brackets

• Can Be Traversed and Updated

Common AST Questions

• When Would I Ever Use An AST?

  Whenever you want to analyze your code

• Couldn't I Just Use Regular Expressions?

  Pattern matching will only get you so far. It is also notoriously ugly to read.

• How Do I Generate An AST?

  By Using A Parser Library (or writing your own parser)

Let's Generate an AST!

Esprima

• JavaScript Parser That Generates ASTs

• Can Be Used In A Node.js Or Web Environment

• Adheres To The Mozilla SpiderMonkey Parser API

• Created by Ariya Hidayat

Generating An AST With Esprima

// Node.js Environment
var code = 'function jqcon() {}',
  esprima = require('esprima'),
  ast = esprima.parse(code);
  

That's All There Is To Creating An AST

Awesome, It Works!

ASTs Are So Cool

• But I Still Have A Few Questions About Esprima...

Common Esprima Questions

• Do I Have To Use Esprima?

  No, there are alternatives out there. A popular alternative is Acorn

• Why Does Esprima Adhere To The Mozilla SpiderMonkey Parser API?

  Because the SpiderMonkey Parse API is recognized by the community as a standard for structured JS representation (and many other tools adhere to it as well)

What Now?

• Now That We Have Generated Our AST...

• Let's Traverse and Update It!

Estraverse

• JavaScript Library That Provides AST Traversal and Update Methods

• Can Be Used In A Node.js Or Web Environment

• Adheres To The Mozilla SpiderMonkey Parser API

• Created by Yusuke Suzuki

Traversing An AST With Estraverse

// Node.js Environment
var code = 'function jqcon() {}',
  esprima = require('esprima'),
  ast = esprima.parse(code),
  estraverse = require('estraverse');
  
estraverse.traverse(ast, {
  enter: function (node, parent) {},
  leave: function(node, parent) {}
});

Traversing And Updating An AST With Estraverse

estraverse.traverse(ast, {
  enter: function (node, parent) {
    if(node.type === 'Identifier' && node.name === 'jqcon') {
      // Changes the 'jqcon' function name to 'jqcon_is_awesome'
      node.name = node.name + '_is_awesome';  
    }
  },
});

You Could Also Use the Estraverse.replace() Method...

Traversing And Updating An AST With Estraverse

estraverse.replace(ast, {
  enter: function (node, parent) {
    if(node.type === 'Identifier' && node.name === 'jqcon') {
      // Changes the 'jqcon' function name to 'jqcon_is_awesome'
      return { 'type': 'Identifier', 'name': 'jqcon_is_awesome' };
    }
  }
});

Awesome, It Works!

Traversing and Updating ASTs Is Fun

• But I Still Have A Few Questions About Estraverse...

Common Estraverse Questions

• Do I Have To Use Estraverse?

  No, there are alternatives out there. A popular alternative is esquery

• But Couldn't I Traverse And Update The AST Myself?

  You could, but traversing ASTs is tricky since not all AST child nodes have a unified interface.

  e.g. VariableDeclaration has nested children nodes within a declarations property, while an IfStatement has nested children within a consequent property.

Let's Review

• Use Esprima To Generate An AST From Code

• Use Estraverse To Traverse And Update That AST

What Now?

• Now That We Have Generated, Traversed, and Updated Our AST...

• Let's Generate Code!

Escodegen

• JavaScript Library That Generates Code From An AST

• Can Be Used In A Node.js Or Web Environment

• Adheres To The Mozilla SpiderMonkey Parser API

• Created by Yusuke Suzuki

Generating Code From An AST With Escodegen

// Node.js Environment
var code = 'function jqcon() {}',
  esprima = require('esprima'),
  ast = esprima.parse(code),
  escodegen = require('escodegen'),
  // Returns a string of code represented by the AST
  regenerated_code = escodegen.parse(ast);
  

Amazing!

Generating Code From An AST Is Hot

• But I Still Have A Few Questions About Escodegen...

Common Escodegen Questions

• Do I Have To Use Escodegen?

  Good question. I'm not sure of a good alternative. Anyone in the audience know?

• But Couldn't I Write an AST Code Generator Myself?

  You could. I'll see you next year, when you're finished.

Let's Review

• Use Esprima To Generate An AST From Code

• Use Estraverse To Traverse And Update That AST

• Use Escodegen To Generate Code From That AST

Wrap Up Questions

• Your Examples Are Kind Of Simple. Got Anything More Complex?

  Yep, check out the AMDClean website and source

• You Didn't Cover CSS Static Analysis. What Gives?

  Not enough time my friend. But I created an example that you can look at later.

• Are There Any More Resources You Would Recommend?

  Yes, the Esprima Parser Demo and the Mozilla SpiderMonkey Parser API will be your best friends. You current best friends will understand.

Thanks!

Twitter: GregFranko

Github: gfranko