How to use the walkTokens function from marked

marked.walkTokens is a function that iterates over the tokens generated by the marked library during Markdown parsing and calls a callback function for each token.

How does marked.walkTokens work?

Sure! marked.walkTokens is a function that iterates over the tokens generated by the marked library during Markdown parsing and calls a callback function for each token. When you call marked.walkTokens with an array of tokens and a callback function, it iterates over the array of tokens and calls the callback function once for each token. The callback function is called with two arguments: the current token and the index of the current token in the array. The walkTokens function is useful for examining or modifying the tokens generated by marked during the parsing process. For example, you might use it to transform links or images to use a different format or to extract metadata from the parsed Markdown. By default, marked.walkTokens iterates over the tokens in order, from the first token to the last. However, you can also provide a third argument to walkTokens, a boolean value reverse, which will cause the function to iterate over the tokens in reverse order. You can also break out of the iteration loop early by returning false from the callback function. This will cause walkTokens to stop iterating and return immediately.

Ai Example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
const marked = require("marked");

const markdown =
  "This is a [link](https://example.com/) and an ![image](https://example.com/image.png).";
const tokens = marked.lexer(markdown);

marked.walkTokens(tokens, (token, index) => {
  if (token.type === "link") {
    token.href = "https://example.net/" + token.href;
  } else if (token.type === "image") {
    token.src = "https://example.net/" + token.src;
  }
});

const html = marked.parser(tokens);

console.log(html);
// Output: "This is a link and an \n"

In this example, we first import the marked library. We then define a Markdown string markdown containing a link and an image. We use the marked.lexer() function to parse the Markdown into an array of tokens, which we store in the tokens variable. We then call marked.walkTokens() with the tokens array and a callback function that transforms the URLs in the link and image tokens to use a different domain. In this case, we prepend "https://example.net/" to the href or src properties of the tokens depending on their type. We then use the marked.parser() function to convert the modified tokens back into HTML. Finally, we log the resulting HTML to the console using console.log(). The output shows that the URLs in the link and image have been modified to use the new domain.