github.com/JuliaLang/Tokenize.jl issues | Ecosyste.ms: OpenCollective

lexer code's use of string

spaette opened this issue 3 months ago

add public kw

pfitzseb opened this pull request 8 months ago

feat: add some more arrow ops

pfitzseb opened this pull request 12 months ago

add ⥷ to startchars

pfitzseb opened this pull request 12 months ago

tweak `is_operator_start_char` to not use so heavily nested syntax

KristofferC opened this pull request about 1 year ago

Fix tokenization of numbers followed by `..`

BenPH opened this pull request almost 2 years ago

Fix for tokenization of consecutive nested multiline comments

c42f opened this pull request about 2 years ago

Fails to parse nested open/close comment sequences

quinnj opened this issue about 2 years ago

Can you tokenize a string with a regular expression

Nick-Mul opened this issue about 2 years ago

tokenize fails on double quotes

MichelJuillard opened this issue over 2 years ago

Update TagBot.yml

pfitzseb opened this pull request over 2 years ago

Add unicode minus

pfitzseb opened this pull request over 2 years ago

Disable interpolation lexing for prefixed strings

pfitzseb opened this pull request over 2 years ago

More unicode dots as operators

pfitzseb opened this pull request over 2 years ago

Add top/bot

pfitzseb opened this pull request over 2 years ago

Treat UTF-8 BOM as WS

pfitzseb opened this pull request over 2 years ago

Regression

t-bltg opened this issue almost 3 years ago

add two missing Kind -> Symbol

KristofferC opened this pull request almost 3 years ago

remove unused functions

KristofferC opened this pull request almost 3 years ago

JET self test

goerch opened this issue about 3 years ago

Use Base.hash

pfitzseb opened this pull request about 3 years ago

#188, but fixed

pfitzseb opened this pull request over 3 years ago

Revert "Simplify lex_identifier"

KristofferC opened this pull request over 3 years ago

Tokenize 0.5.19 broke JuMPs formatting

odow opened this issue over 3 years ago

Simplify lex_identifier

ZacLN opened this pull request over 3 years ago

RFC: change optimization level to 1

KristofferC opened this pull request over 3 years ago

lex all of Base in tests

pfitzseb opened this pull request over 3 years ago

add additional operator suffixes

pfitzseb opened this pull request over 3 years ago

Add LEFT_ARROW and DOUBLE_ARROW

pfitzseb opened this pull request over 3 years ago

0.5.17

goretkin opened this pull request over 3 years ago

clean up code

pfitzseb opened this pull request over 3 years ago

fix isopsuffix

pfitzseb opened this pull request over 3 years ago

`isopsuffix` incomplete

goretkin opened this issue over 3 years ago

fix unicode string interpolation

pfitzseb opened this pull request over 3 years ago

String interpolation with unicode error ref JuliaFormatter

domluna opened this issue over 3 years ago

bug with Tokenize.jl v0.5.14 for JuliaFormatter

ericphanson opened this issue over 3 years ago

fix string interpolation

pfitzseb opened this pull request over 3 years ago

Improved string interpolation lexing removes start of string interpolations with parenthesis

bauglir opened this issue over 3 years ago

incorrect attribute values for tokens of the KEYWORD category

alexfdo opened this issue almost 4 years ago

improve string interpolation lexing

pfitzseb opened this pull request almost 4 years ago

string interpolation with single-quote errors

pfitzseb opened this issue almost 4 years ago

Minor documentation

goretkin opened this pull request almost 4 years ago

Use Documenter

goretkin opened this pull request almost 4 years ago

update list of operators

ZacLN opened this pull request almost 4 years ago

allow PRIME after END kw

ZacLN opened this pull request almost 4 years ago

fix invalid float parse

ZacLN opened this pull request almost 4 years ago

emit error on invalid hexadecimal

ZacLN opened this pull request almost 4 years ago

hexadecimal error

ZacLN opened this issue almost 4 years ago

add missing operator

ZacLN opened this pull request almost 4 years ago

TagBot trigger issue

JuliaTagBot opened this issue almost 4 years ago

fix string interpolation of Chars

ZacLN opened this pull request almost 4 years ago

incorrectly errors

ZacLN opened this issue almost 4 years ago

use github actions instead of travis

KristofferC opened this pull request almost 4 years ago

fix comment lexing

ZacLN opened this pull request almost 4 years ago

also lex stdlibs in benchmark

KristofferC opened this pull request over 4 years ago

update Julia test versions

KristofferC opened this pull request over 4 years ago

use Base.is_id_char, etc.

ZacLN opened this pull request over 4 years ago

Don't error on invalid chars

ZacLN opened this pull request almost 5 years ago

Install TagBot as a GitHub Action

JuliaTagBot opened this pull request almost 5 years ago

Evaluated token

aminya opened this pull request about 5 years ago

disallow float w/ trailing . juxt w/ op

ZacLN opened this pull request about 5 years ago

mark suffixed ops (needed for CSTParser)

ZacLN opened this pull request over 5 years ago

Fix outdated links in README

janEbert opened this pull request over 5 years ago

fix float parsing with trailing dotted operator

ZacLN opened this pull request over 5 years ago

number parsing error

ZacLN opened this issue over 5 years ago

add colon ops

ZacLN opened this pull request over 5 years ago

add hat suffix

ZacLN opened this pull request over 5 years ago

fix dotop starting position

ZacLN opened this pull request over 5 years ago

update travis

KristofferC opened this pull request over 5 years ago

add Project file

KristofferC opened this pull request over 5 years ago

add errortoken for invalid numeric constants

ZacLN opened this pull request over 5 years ago

add `new` kw kind

ZacLN opened this pull request over 5 years ago

fix startpos for dotted ops

KristofferC opened this pull request almost 6 years ago

Dot operators have inconsistent location information

maleadt opened this issue almost 6 years ago

update cmd lexing

ZacLN opened this pull request almost 6 years ago

don't peek Chars

ZacLN opened this pull request almost 6 years ago

compound keywords incorrectly tokenized

pfitzseb opened this issue about 6 years ago

escaping within cmds

ZacLN opened this issue about 6 years ago

Fails to precompile on master

MikeInnes opened this issue about 6 years ago

remove old keywords

ZacLN opened this pull request about 6 years ago

Precompilation fails

baggepinnen opened this issue over 6 years ago

Transition to new iteration protocol

pfitzseb opened this pull request over 6 years ago

add outer kw

ZacLN opened this pull request over 6 years ago

v0.7 compatability

ZacLN opened this pull request over 6 years ago

merge dotted ops to one op

ZacLN opened this pull request over 6 years ago

misc v0.7 fixes

ZacLN opened this pull request almost 7 years ago

replace Base.utf8_trailing

ZacLN opened this issue almost 7 years ago

add perp operator

ZacLN opened this pull request almost 7 years ago

add specific BIN/HEX/OCT_INT

ZacLN opened this pull request almost 7 years ago

v0.7 compatability

ZacLN opened this issue almost 7 years ago

add missing methods

KristofferC opened this pull request about 7 years ago

startbyte(t::Token) and endbyte(t::Token) are documented but do not exist

yurivish opened this issue about 7 years ago

`:keyword` incorrectly tokenized

pfitzseb opened this issue about 7 years ago

Store operator metainfo here

ZacLN opened this issue about 7 years ago

return error on invalid binary/octal/hex intege

ZacLN opened this pull request about 7 years ago

wip allow for op suffixes

KristofferC opened this pull request over 7 years ago

inline some functions

KristofferC opened this pull request over 7 years ago

improve benchmarking code

KristofferC opened this pull request over 7 years ago

actually test inferability and make RawToken inferrable

KristofferC opened this pull request over 7 years ago

Add special cstparser token kinds

ZacLN opened this pull request over 7 years ago