kernel.rst

Transformer Kernels
===================

The transformer kernel API in DeepSpeed can be used to create BERT transformer layer for
more efficient pre-training and fine-tuning, it includes the transformer layer configurations and
transformer layer module initialization.

Here we present the transformer kernel API.
Please see the `BERT pre-training tutorial <https://www.deepspeed.ai/tutorials/bert-pretraining/>`_ for usage details.

DeepSpeed Transformer Config
----------------------------
.. autoclass:: deepspeed.DeepSpeedTransformerConfig

DeepSpeed Transformer Layer
----------------------------
.. autoclass:: deepspeed.DeepSpeedTransformerLayer