pyre.parsing.Scanner

Module Contents

class pyre.parsing.Scanner.Scanner

The input stream tokenizer

start

finish

whitespace

pyre_tokens

pyre_tokenizer

pyre_stream

pyre_client

pyre_cache

pyre_tokenize(self, uri, stream, client)

Extract lines from {stream}, convert them into token streams and send them to {client}

pyre_start(self)

Indicate the beginning of scanning

pyre_finish(self)

Indicate that scanning is complete

pyre_pushback(self, token)

Push a token back into the token stream

pyre_newline(self, stream)

Hook invoked when a new line of text is pulled from the input stream

pyre_ignoreWhitespace(self, client)

Remove {whitespace} tokens from the input stream