Support `onInsertedSemicolon` callback when parsing

[ahabhgk]

Describe the feature

Support pass a callback or a Option<HashSet> for swc_ecma_parser::Parser to collect the BytePos position of automatically inserted semicolons

Babel plugin or link to the feature description

https://github.com/acornjs/acorn/blob/ca53db28c88ce5a8ee8c7db7027be71ed29b6aaf/acorn/src/options.js#L19-L24

Additional context

The background is when I trying to solve web-infra-dev/rspack#11551, I notice that the swc_ecma_parser::Lexer will output different tokens when using it in parse() and .collect_vec()

For example:

import {assert} from './a'

const labelType = `${node}Label`

const node = []
assert(labelType)

In parse() this will have correct tokens, but using with .collect_vec() (or Iterator) it will have a lexing error: Error { error: (60..97, UnterminatedTpl) } (and the lexing error causes Rspack collected the wrong semicolons, which is used for Rspack to add ; at proper position to generate the correct code)

So it will be great for correctness and performance if the ASI position can be collected by SWC when parsing (currently Rspack lexes twice, once for parse and once for collecting ASI positions)

Or provide an API that I can get the correct tokens without parsing, which is a workable alternative solution.

[kdy1]

cc @bvanjoi Thoughts?

[hardfist]

Maybe we should support tokens options like babel https://babeljs.io/docs/babel-parser#:~:text=tokens%3A%20Adds%20all%20parsed%20tokens%20to%20a%20tokens%20property%20on%20the%20File%20node
Using lexer outside of parser seems tricky consider JavaScript is context-sensitive grammar for most cases.

[CPunisher]

Maybe we should support tokens options like babel https://babeljs.io/docs/babel-parser#:~:text=tokens%3A%20Adds%20all%20parsed%20tokens%20to%20a%20tokens%20property%20on%20the%20File%20node Using lexer outside of parser seems tricky consider JavaScript is context-sensitive grammar for most cases.

Sorry. I just realize that tokens can be collected with Capturing. See

swc/crates/swc_ts_fast_strip/src/lib.rs

Line 248 in 05f3524

let lexer = Capturing::new(Lexer::new(

[hardfist]

Capturing seems good enough for our case now for correctness tokens from parser, and we can just implement a custom ASICapturing to collect asi info during parsing phase?