GitLab

Inference context optimization

See merge request ADLR/megatron-lm!321

Add temperature to the server API

See merge request ADLR/megatron-lm!325

A temperature value between 0.0 and 100.0 can now be specified via the API
while running the text generation server. The value passed to
--temperature while running the text generation server is kept as the
default value for all API calls that don't include temperature, even if
different values were manually specified previously.
Signed-Off-By: Robert Clark <roclark@nvidia.com>

Fixes a bug in broadcasting that was causing hanging

See merge request ADLR/megatron-lm!327

Changing API to fix LM Harness Evaluation and make API less confusing/terrible

See merge request ADLR/megatron-lm!326

Changing the interface to the lm eval harness and fixing bugs caused by misunderstanding out_seq_length

Reducing memory usage during inference

See merge request ADLR/megatron-lm!320

This file doesn't work withthe new inference code, so it should be remove

See merge request ADLR/megatron-lm!322

use low-priority stream for nccl overlapping all-reduce and gemm

See merge request ADLR/megatron-lm!319

allreduce overlap with wgrad gemm

See merge request ADLR/megatron-lm!316

allreduce overlap with wgrad gemm

change custom delay to dummy add

Letting server return the log-probabilities of the context and generated text

See merge request ADLR/megatron-lm!317

Use nvfuser at pytorch >= 1.10

See merge request ADLR/megatron-lm!314