netcoreapp2.1 
namespace Microsoft.ML.Tokenizers
{
    public class CodeGenTokenizer : Tokenizer
    {
        public override Normalizer? Normalizer { get; }
    }
}
.NET 5.06.07.08.09.010.0
.NET Core 2.02.12.23.03.1
.NET Framework 4.6.14.6.24.74.7.14.7.24.84.8.1
.NET Standard 2.02.1
Information specific to netcoreapp2.1
Assemblies

Microsoft.ML.Tokenizers , Version=1.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51

Microsoft.ML.Tokenizers , Version=1.0.0.0, PublicKeyToken=cc7b13ffcd2ddd51

Referencing

Your project needs a package reference to Microsoft.ML.Tokenizers.

Package Microsoft.ML.Tokenizers (1.0.2) netstandard2.0
Platform Restrictions

This framework does not have platform annotations.

  • Built-in API
  • Package-provided API