GitLab

Instead of deriving different tokens from a base Token class, we now use an enumerated TOKEN_TYPE to distinguish types. This is so we don't have to cast all the time when parsing the resulting token stream.
Also, removed start/end stream tokens.

Moved the three scalar token scanning functions back to scantoken.cpp, so scanscalar.cpp now only has the main scalar scanning function.
Renamed ScanScalarInfo to ScanScalarParams.

Renamed the valid/possible tokens to a single variable status with enums valid, invalid, and unverified.

Moved the input stream, together with line/column info, into its own class, which allowed some other stuff just to pass the stream, and not have to be a member of Scanner.

No commit message

No commit message

There's a bug (and question): should we test simple keys' validity BEFORE stuff or AFTER stuff?

Refactored some common whitespace-parsing code in scanning both scalars.
Implemented the flow collection tokens.

Split off the specific regular expressions, and the specialized token-scanning functions, into their own files.

Added stream input to the regular expressions, greatly simplifying the usage (in particular, we no longer have to specify the number of characters to be checked).

Wrote a simplified regular expression parser to make life easier (it only does single matches; i.e., no one-or-more matches, etc.).
Fixed some of the whitespace/line break matching.

Also messed around with multiple character peeking on the input, and got something working.

We're now using exceptions for errors, and scanning/pushing tokens is exception-safe (using a set of "limbo tokens").

Will be completely wiped, I think, in favor of a Scanner (to tokens), then Parser mechanism.

No commit message

No commit message

No commit message