GitLab

Add Beginning of Sentence token option and adding semaphore while multi-threading to prevent crashes and hangs due to connection keep-alives

See merge request ADLR/megatron-lm!328

Fixing memory bug caused by DDP during task

See merge request ADLR/megatron-lm!329

Inference context optimization

See merge request ADLR/megatron-lm!321

Add temperature to the server API

See merge request ADLR/megatron-lm!325

A temperature value between 0.0 and 100.0 can now be specified via the API
while running the text generation server. The value passed to
--temperature while running the text generation server is kept as the
default value for all API calls that don't include temperature, even if
different values were manually specified previously.
Signed-Off-By: Robert Clark <roclark@nvidia.com>

Fixes a bug in broadcasting that was causing hanging

See merge request ADLR/megatron-lm!327

Changing API to fix LM Harness Evaluation and make API less confusing/terrible

See merge request ADLR/megatron-lm!326

Changing the interface to the lm eval harness and fixing bugs caused by misunderstanding out_seq_length

Reducing memory usage during inference

See merge request ADLR/megatron-lm!320

This file doesn't work withthe new inference code, so it should be remove

See merge request ADLR/megatron-lm!322

use low-priority stream for nccl overlapping all-reduce and gemm

See merge request ADLR/megatron-lm!319