FlexModule and BisonModule

These two tools allow a programmer to create Python modules from Flex-based lexical scanners and Bison-based parsers. Each consists of a C header file that is included in the Flex or Bison source and which defines macros (and other code) to handle the interface with Python.


  • 14 Jun 2007: For a completely different project at work, I reimplemented these two packages for C++. Using std::tr1::shared_ptr for reference counted memory management and the STL containers makes it a fairly direct translation. The C/C++ interface does require some finesse. Unfortunately, the package is now owned by Cadence, so I cannot put it online here; perhaps I will recreate something similar someday.
  • 22 Mar 2002: FlexBisonModules-2.1: Cleans up push_position in FlexModule.h based on experience with APC and tries to handle some weird error rule behavior by Bison.
  • 28 Jan 2002: FlexBisonModules-2.0: New release! A complete rewrite of FlexModule to use C, have better position handling, and allow sub-files to be inserted into the token stream. BisonModule has much better error handling. This uses a slightly incompatible interface, though.
  • 11 Jun 2001: FlexBisonModules-1.1: Fixed bug in FlexModule.h. LexerInput was repeatedly copying the first block of the string for input longer than 8k. Thanks to Philip J. Mayers for pointing that out.
  • 18 Aug 2000: FlexBisonModules-1.0: FlexBisonModules-1.0 released.

Source code





hoc is the calculator developed in The UNIX Programming Environment by Brian W. Kernighan and Rob Pike. This example uses an intermediate form of the hoc grammar to demonstrate FlexModule and BisonModule. The lexer is hoclexer.l, and the grammar is hocgrammar.y. The complete example is included in the distribution above.

For a larger example, see the Austin Protocol Compiler next door.


I strongly recommend Lex and Yacc (2nd ed.) by J. Levine, T. Mason, and D. Brown from O'Reilly and Associates, 1992. Make sure you get the second edition, which is much better than the first.

The set of notes I wrote while designing these two tools is available as FBModule.notes.tar.gz. The archive contains a short, incoherent PostScript paper that describes several other options that I explored, as well as my own introduction to mixing C with Python.

Other Python Parsing Tools

There's always:

(This list cheerfully stolen from the PLY page.)

And, of course, byacc. To get the source code, I believe you need to use CVS.

gloria i ad inferni
faciamus opus

Return to Top | About this site...
Last edited Sat Aug 8 03:29:10 2009.
Copyright © 2005-2016 Tommy M. McGuire