[Cache] Add cachedPrefixes for caching repeated system prompts

[YiyanZhai]

This PR adds the cachedPrefixes field in MLCEngineConfig, allowing users to cache system prompts when creating MLCEngine. It reduces redundant processing of repeated instructions.

Example usage in CreateMLCEngine:

await webllm.CreateMLCEngine(
  selectedModel,
  {
    initProgressCallback: initProgressCallback,
    logLevel: "INFO",
    cachedPrefixes: [
      [ { role: "system", content: "You are a helpful assistant running in the user's browser. You need to answer questions ... " }, ]
    ],
  },
  {
    context_window_size: 2048,
  }
);

[CharlieFRuan]

Thank you for the hard work! Added some comments. Please add an E2E example under examples/. I will take another pass afterwards. Thanks again!

[CharlieFRuan]

Let's add docs to MLCEngineConfig, specifying the behavior of cachedPrefixes (e.g. will prefill when loading the engine to create the prefixes' KV, will only dispose these KV when reloading the engine). Perhaps we can also mark this as experimental to signify potential future API/behavior change

[CharlieFRuan]

Could you also add an examples/cached_prefixes? Where we can demonstrate the prefill time difference between using cachedPrefixes and not using it. We should also test whether the behavior is expected in multi-turn conversation.

[CharlieFRuan]

Now that we have multiple sequence IDs, let's make a constant, say CHAT_SEQUENCE_ID=0 (or maybe a better naming), instead of using a magic number 0 that may be hard to keep track of

[CharlieFRuan]

Use log.info() instead of console.log()

[CharlieFRuan]

Use log.info() instead of console.log()