Microsoft.ML.Tokenizers
BertOptions- BertTokenizer
- BpeOptions
- BpeTokenizer
- CodeGenTokenizer
- CompositePreTokenizer
EncodedToken- EncodeResults<T>
- EncodeSettings
- EnglishRobertaTokenizer
- LlamaTokenizer
- LowerCaseNormalizer
- Normalizer
- Phi2Tokenizer
- PreTokenizer
- RegexPreTokenizer
- RobertaPreTokenizer
- SentencePieceNormalizer
- SentencePieceTokenizer
- TiktokenTokenizer
- Tokenizer
- UpperCaseNormalizer
- WordPieceOptions
- WordPieceTokenizer
net11.0-windows7.0
namespace Microsoft.ML.Tokenizers
| nuget.org | 0.0 % Reference this API |
|---|---|
| .NET | 5.06.07.08.09.010.011.0 |
| .NET Core | 2.02.12.23.03.1 |
| .NET Framework | 4.6.14.6.24.74.7.14.7.24.84.8.1 |
| .NET Standard | 2.02.1 |
Information specific to net11.0 | |
| Platforms | This API is only available when you target a specific platform: |
| Windows | 7.0 |
Information specific to net11.0-windows7.0 | |
| Assemblies | Microsoft.ML.Tokenizers , Version=1.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51 Microsoft.ML.Tokenizers , Version=1.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51 |
| Referencing | Your project needs a package reference to |
| Package | Microsoft.ML.Tokenizers (3.0.0-preview.26457.2) net8.0 |
- Built-in API
- Package-provided API