LlamaTokenizer
Create(Stream, Boolean, Boolean, IReadOnlyDictionary<String, Int32>)
net462
namespace Microsoft.ML.Tokenizers
{
public sealed class LlamaTokenizer : SentencePieceBpeTokenizer
{
public static LlamaTokenizer! Create(Stream! modelStream, bool addBeginOfSentence = true, bool addEndOfSentence = false, IReadOnlyDictionary<string!, int>? specialTokens = null);
}
}
| .NET | 5.06.07.08.09.0 |
|---|---|
| .NET Core | 2.02.12.23.03.1 |
| .NET Framework | 4.6.14.6.24.74.7.14.7.24.84.8.1 |
| .NET Standard | 2.02.1 |
Information specific to net462 | |
| Assembly | Microsoft.ML.Tokenizers , Version=1.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51 |
| Referencing | Your project needs a package reference to |
| Package | Microsoft.ML.Tokenizers netstandard2.0 |
| Preview | This API is contained in a prerelease package. |
| Platform Restrictions | This framework does not have platform annotations. |
- Built-in API
- Package-provided API