Language.Python.Version2.Lexer

language-python-0.5.4: Parsing and pretty printing of Python code.

Copyright | (c) 2009 Bernie Pope
License | BSD-style
Maintainer | bjpop@csse.unimelb.edu.au
Stability | experimental
Portability | ghc
Safe Haskell | None
Language | Haskell98

Language.Python.Version2.Lexer

Contents

- Lexical analysis

Description

Lexical analysis for Python version 2.x programs. See: http://www.python.org/doc/2.6/reference/lexical_analysis.html.

Synopsis

- lex :: String -> String -> Either ParseError [Token]
- lexOneToken :: String -> String -> Either ParseError (Token, String)

# Lexical analysis

lex #

Arguments

:: String | The input stream (python source code).
-> String | The name of the python source (filename or input device).
-> Either ParseError [Token] | An error or a list of tokens.

Parse a string into a list of Python Tokens, or return an error.

lexOneToken #

Arguments

:: String | The input stream (python source code).
-> String | The name of the python source (filename or input device).
-> Either ParseError (Token, String) | An error or the next token and the rest of the input after the token.

Try to lex the first token in an input string. Return either a parse error or a pair containing the next token and the rest of the input after the token.