Fix wavlm bundles (#3660)

The `encoder_layer_norm_first` should be set to True for the Large model of WavLM. Address https://github.com/pytorch/audio/issues/3347

Fix wavlm bundles (#3660)
The `encoder_layer_norm_first` should be set to True for the Large model of WavLM. Address https://github.com/pytorch/audio/issues/3347
671261c3 · nateanl · GitHub · 0b6ca28d · 671261c3
Unverified Commit 671261c3 authored Oct 17, 2023 by nateanl Committed by GitHub Oct 17, 2023
Hide whitespace changes
Inline Side-by-side

Showing with 1 addition and 1 deletion

src/torchaudio/pipelines/_wav2vec2/impl.py src/torchaudio/pipelines/_wav2vec2/impl.py +1 -1

No files found.
--- a/src/torchaudio/pipelines/_wav2vec2/impl.py
+++ b/src/torchaudio/pipelines/_wav2vec2/impl.py
@@ -1350,7 +1350,7 @@ WAVLM_LARGE = Wav2Vec2Bundle(
        "encoder_ff_interm_features": 4096,
        "encoder_ff_interm_dropout": 0.0,
        "encoder_dropout": 0.1,
-        "encoder_layer_norm_first": False,
+        "encoder_layer_norm_first": True,
        "encoder_layer_drop": 0.05,
        "aux_num_out": None,
    },