GitLab

Split off the specific regular expressions, and the specialized token-scanning functions, into their own files.

Added stream input to the regular expressions, greatly simplifying the usage (in particular, we no longer have to specify the number of characters to be checked).

Wrote a simplified regular expression parser to make life easier (it only does single matches; i.e., no one-or-more matches, etc.).
Fixed some of the whitespace/line break matching.

Also messed around with multiple character peeking on the input, and got something working.

We're now using exceptions for errors, and scanning/pushing tokens is exception-safe (using a set of "limbo tokens").

Will be completely wiped, I think, in favor of a Scanner (to tokens), then Parser mechanism.

No commit message

No commit message

No commit message