GitLab

Signed-off-by: shane.xb.qian <shane.qian@foxmail.com>

Co-authored-by: Richard Lyons <frob@cloudstaff.com>

engine: add gemma support

This reverts commit c7eae586b899083acebcd9b3847b89ea78c2850c.

This is useful for a few things:
 - Work around bugs, such as having 2 images in one batch
 - Keep the image in a single batch for fully connected attention
 - Improve performance by not evaluating embeddings multiple times

Currently we are using positions, which are relative to a
sequence and may not be unique.

fix: pad tensor item if ge zero