• Devon Rifkin's avatar
    add qwen3-coder tool support · 47991940
    Devon Rifkin authored
    The format qwen3-coder uses is relatively unique, both in rendering and
    in parsing. To implement parsing, I wrote a custom parser in similar
    style to harmony. For the rendering, I found that the logic would be
    much more difficult to follow in a template, so I introduced the concept
    of a built-in renderer that uses go code, rather than a template to
    generate prompts.
    
    I set us up for future built-in parsers and renderers by making it so
    they can be specified in a Modelfile like so:
    
    ```
    RENDERER "qwen3-coder"
    PARSER "qwen3-coder"
    ```
    
    These need to be provided explicitly because the architecture alone is
    not enough to understand what format the model expects to receive, and
    what format we expect it to output (e.g., qwen3-coder is `qwen3moe`,
    which includes other qwen3-family models as well)
    
    I haven't converted harmony to be one of these "built-ins" yet, since
    some of it is in flux with the changes @ParthSareen has been making to
    move harmony to the runner. It is likely that many other built-ins will
    need to move to the runner as well, but I'm able to slightly defer that
    decision since qwen3-coder doesn't have thinking (and therefore doesn't
    need to be in the runner to make structured outputs work). I expect to
    unify harmony with this approach very soon.
    
    Whether a particular model supports tools or thinking was previously
    inferred from templates, but without a template we now also use the
    parser itself to declare what it supports. If we have future models that
    re-use the same parsing format, but have different capabilities, we'll
    want to parameterize them and give them different names to be specified
    as a `PARSER`.
    
    Misc changes:
    
    - I worked on the renderer by diffing outputs from the reference
      implementation and ours. To make it easier to do this, I extended
      <https://github.com/ollama/ollama/pull/11875> to also support
      returning the prompt via the openai compat layer
    47991940
parser_test.go 18.1 KB