Code

Shortfin llm beam search#1011

Code

Draft

stbaione wants to merge 13 commits intonod-ai:mainnod-ai/shark-ai:mainfrom stbaione:shortfin-llm-beam-searchstbaione/SHARK-Platform:shortfin-llm-beam-searchCopy head branch name to clipboard

+525-23

Commits

Commits on Feb 27, 2025

• Still wanna make a couple changes & cleanup,

  Show description for a5f9c51
  stbaione

  committed
  a5f9c51

  Copy full SHA for a5f9c51

• Move final beam_selection to BeamGroup

  stbaione

  committed
  a4194a3

  Copy full SHA for a4194a3

• Cleanup beam decode loop,

  Show description for a92623c
  stbaione

  committed
  a92623c

  Copy full SHA for a92623c

• Fix typo in greedy decode loop

  stbaione

  committed
  0b8f388

  Copy full SHA for 0b8f388

• Merge branch 'main' of https://github.com/nod-ai/shark-ai into shortfin-llm-beam-search

  stbaione

  committed
  abce7f3

  Copy full SHA for abce7f3

• Small cleanup

  stbaione

  committed
  730664b

  Copy full SHA for 730664b

• Remove unused function,

  Show description for 46371ef
  stbaione

  committed
  46371ef

  Copy full SHA for 46371ef

Commits on Mar 3, 2025

• Shift cumulative log prob to the right by min_log_prob to prevent large negative numbers,

  Show description for 7cede71
  stbaione

  committed
  7cede71

  Copy full SHA for 7cede71

• Abstract beam_search and greedy decode strategies,

  Show description for 3db70d4
  stbaione

  committed
  3db70d4

  Copy full SHA for 3db70d4

• Remove unnecessary sort,

  Show description for 698594c
  stbaione

  committed
  698594c

  Copy full SHA for 698594c

• Add instance_id to messages_test

  stbaione

  committed
  8ff8e11

  Copy full SHA for 8ff8e11

• Add temperature

  stbaione

  committed
  08cbfe1

  Copy full SHA for 08cbfe1

Commits on Mar 4, 2025

• Remove references to n_beams from sharktank,

  Show description for d496e44
  stbaione

  committed
  d496e44

  Copy full SHA for d496e44