1. 11 Dec, 2024 1 commit
    • Blake Mizerany's avatar
      llama: preserve field order in user-defined JSON schemas (#8002) · 9039c821
      Blake Mizerany authored
      Previously we decoded and re-encoded JSON schemas during validation,
      which served no purpose since json.RawMessage already validates JSON
      syntax. Worse, the re-encoding lost field ordering from the original
      schema, which affects inference quality during step-by-step reasoning.
      
      While fixing this ordering issue by using json.RawMessage directly,
      testing revealed that schema_to_grammar (from llama.cpp) also fails to
      preserve field order during grammar generation. This appears to be the
      root cause of inference degradation.
      
      This change prevents us from mangling the user's original schema order,
      but we still need to address the ordering issue in schema_to_grammar.
      That will be a separate change.
      
      Updates #7978
      9039c821