tokens_<modeName>.json files keep information about correct tokens and tokenizer states for all modes supported by ace. They are generated from text_<modeName>.txt or demo/kitchen-sink/doc/* with
tokens_<modeName>.json
text_<modeName>.txt
demo/kitchen-sink/doc/*
node highlight_rules_test.js -gen
command.