graphql / graphql-js / 1133

98%

master: 98%

push

travis-ci

Refactors tokenizer to emit a Token double-linked-list

This refactors the tokenizer, simplifying both tokenizer and parser code slightly (fewer utility functions) and returns tokens as part of the AST's `loc` information. Tokens also have gained a few new properties: `line`/`column` for the 1-indexed position the token begins and `prev`/`next` to navigate the dll of Tokens. Finally, this introduces two new Token types: `<SOF>` for the empty token at beginning of the file stream, and `Comment` for ignored comment tokens. These tokens are *included* in the Token list allowing for easier navigation of the parsed document in GraphQL tools and as a basis for parsing comment descriptions in the schema language.

Providing tokens as part of each AST node's `loc` provides a linkage between the abstract AST and the concrete token list which provides more useful information to GraphQL language tools.

Thanks to the addition of the `<SOF>` token, The AST `Document` node returned from `parse()` now definitionally points to `<SOF>` as its `startToken` and `<EOF>` as its `endToken`, effectively the head and tail of the token double linked list.

This also removes the `noSource` parser option in favor of `source` being non-enumerable (the new `startToken` and `endToken` properties are also non-enumerable) so that they do not appear in JSON.stringify/util.inspect.

372 of 372 new or added lines in 4 files covered. (100.0%)

7321 of 7448 relevant lines covered (98.29%)

1384.28 hits per line