PLY: A Legacy Python Lex-Yacc Parser Generator (Project Abandoned)

Introduction

PLY (Python Lex-Yacc) is a venerable, zero-dependency Python implementation of the classic lex and yacc parsing tools. Authored by David Beazley, PLY was originally developed in 2001 to support an Introduction to Compilers course at the University of Chicago. It utilizes the same LALR(1) parsing algorithm as yacc and provides core features for building Abstract Syntax Trees (ASTs), simple one-pass compilers, and protocol decoders.

Important Notice: As of December 21, 2025, the PLY project has been officially abandoned by its author after 25 years. No further maintenance is expected. While modern, high-quality parsing libraries are now available, users can continue to use PLY by copying it directly into their projects, or explore writing a hand-rolled recursive descent parser as a challenge.

Installation

PLY is not distributed or installed via a package manager. To use PLY, you need to copy its two core files, lex.py and yacc.py, which are contained within a ply package directory. Simply copy this ply directory into your project and import lex and yacc from it. Alternatively, you can install these files into your Python environment using make install.

from .ply import lex
from .ply import yacc

PLY has no third-party dependencies, is very small, and rarely changes, making it flexible for integration into various projects. It requires Python 3.6 or greater. For older Python versions, historical releases are available.

Examples

PLY's primary application is in writing parsers for programming languages. The following example demonstrates how to use PLY to parse simple arithmetic expressions and construct an Abstract Syntax Tree (AST):

# -----------------------------------------------------------------------------
# example.py
#
# Example of using PLY To parse the following simple grammar.
#
#   expression : term PLUS term
#              | term MINUS term
#              | term
#
#   term       : factor TIMES factor
#              | factor DIVIDE factor
#              | factor
#
#   factor     : NUMBER
#              | NAME
#              | PLUS factor
#              | MINUS factor
#              | LPAREN expression RPAREN
#
# -----------------------------------------------------------------------------

from ply.lex import lex
from ply.yacc import yacc

# --- Tokenizer

# All tokens must be named in advance.
tokens = ( 'PLUS', 'MINUS', 'TIMES', 'DIVIDE', 'LPAREN', 'RPAREN',
           'NAME', 'NUMBER' )

# Ignored characters
t_ignore = ' \t'

# Token matching rules are written as regexs
t_PLUS = r'\+'
t_MINUS = r'-'
t_TIMES = r'\*'
t_DIVIDE = r'/'
t_LPAREN = r'\('
t_RPAREN = r'\)'
t_NAME = r'[a-zA-Z_][a-zA-Z0-9_]*'

# A function can be used if there is an associated action.
# Write the matching regex in the docstring.
def t_NUMBER(t):
    r'\d+'
    t.value = int(t.value)
    return t

# Ignored token with an action associated with it
def t_ignore_newline(t):
    r'\n+'
    t.lexer.lineno += t.value.count('\n')

# Error handler for illegal characters
def t_error(t):
    print(f'Illegal character {t.value[0]!r}')
    t.lexer.skip(1)

# Build the lexer object
lexer = lex()
    
# --- Parser

# Write functions for each grammar rule which is
# specified in the docstring.
def p_expression(p):
    '''
    expression : term PLUS term
               | term MINUS term
    '''
    # p is a sequence that represents rule contents.
    #
    # expression : term PLUS term
    #   p[0]     : p[1] p[2] p[3]
    # 
    p[0] = ('binop', p[2], p[1], p[3])

def p_expression_term(p):
    '''
    expression : term
    '''
    p[0] = p[1]

def p_term(p):
    '''
    term : factor TIMES factor
         | factor DIVIDE factor
    '''
    p[0] = ('binop', p[2], p[1], p[3])

def p_term_factor(p):
    '''
    term : factor
    '''
    p[0] = p[1]

def p_factor_number(p):
    '''
    factor : NUMBER
    '''
    p[0] = ('number', p[1])

def p_factor_name(p):
    '''
    factor : NAME
    '''
    p[0] = ('name', p[1])

def p_factor_unary(p):
    '''
    factor : PLUS factor
           | MINUS factor
    '''
    p[0] = ('unary', p[1], p[2])

def p_factor_grouped(p):
    '''
    factor : LPAREN expression RPAREN
    '''
    p[0] = ('grouped', p[2])

def p_error(p):
    print(f'Syntax error at {p.value!r}')

# Build the parser
parser = yacc()

# Parse an expression
ast = parser.parse('2 * 3 + 4 * (5 - x)')
print(ast)

Why Use PLY?

Despite its abandoned status, PLY holds value for several reasons. Its zero-dependency nature and small codebase make it highly portable and easy to integrate without introducing complex dependency chains. For those learning about compiler design and parsing, PLY offers a straightforward, direct implementation of traditional lex/yacc principles, making it an excellent educational tool.

The author explicitly suggests that users copy PLY into their projects, emphasizing its stability and minimal change over time. This approach gives developers full control over their parsing library. Furthermore, PLY is free software, allowing users to copy and modify it to create more powerful or specialized parsing tools, provided the original copyright notice is retained. For a more modern API and active development, the related SLY project is recommended.