pyre.parsing

Package Contents

class pyre.parsing.token(pattern=None, head='', tail='', **kwds)

Place holders for the token specifications

Descriptors are harvested by {Lexer}, the metaclass of {Scanner} subclasses, and converted into subclasses of {Token}

head =
tail =
pattern =
__str__(self)

Build a representation of the descriptor, mostly for debugging purposes

class pyre.parsing.scanner

The input stream tokenizer

start
finish
whitespace
pyre_tokens
pyre_tokenizer
pyre_stream
pyre_client
pyre_cache
pyre_tokenize(self, uri, stream, client)

Extract lines from {stream}, convert them into token streams and send them to {client}

pyre_start(self)

Indicate the beginning of scanning

pyre_finish(self)

Indicate that scanning is complete

pyre_pushback(self, token)

Push a token back into the token stream

pyre_newline(self, stream)

Hook invoked when a new line of text is pulled from the input stream

pyre_ignoreWhitespace(self, client)

Remove {whitespace} tokens from the input stream

class pyre.parsing.sws

Bases: pyre.parsing.Scanner.Scanner

A scanner for languages that use leading whitespace to indicate the hierarchical structure of the content

pop
push
cdata
comment
pyre_dent = 0
pyre_blocks
margin
pyre_start(self)

Scanning has begun

pyre_finish(self)

Scanning has ended

pyre_newline(self, stream)

A fresh line has been retrieved from the input {stream}

pyre_indent(self, locator, column)

Indent to the given {column}

pyre_dedent(self, locator, column)

Dedent by as many open blocks as it takes to come back to {column}

pyre_cdata(self)

Scan forward from the current location and convert any text in all nested blocks into a {cdata} token. The lexeme of a {cdata} token is a list of strings with the leading indentation, all trivial lines, and all embedded comments removed.

pyre_trim(self, text)

Determine whether {text} contains any structurally relevant information

class pyre.parsing.parser(**kwds)

The base class for parsers

lexer
scanner