Skip to content

Commit 21777be

Browse files
authored
Updates LLM performance matrix (#1245)
9.0 component of #1206 Updates the LLM matrix to reflect latest changes. [Preview](https://docs-v3-preview.elastic.dev/elastic/docs-content/pull/1245/solutions/security/ai/large-language-model-performance-matrix)
1 parent 468e03d commit 21777be

File tree

1 file changed

+17
-15
lines changed

1 file changed

+17
-15
lines changed

solutions/security/ai/large-language-model-performance-matrix.md

+17-15
Original file line numberDiff line numberDiff line change
@@ -24,17 +24,18 @@ Models from third-party LLM providers.
2424

2525
| **Feature** | - | **Assistant - General** | **Assistant - {{esql}} generation** | **Assistant - Alert questions** | **Assistant - Knowledge retrieval** | **Attack Discovery** | **Automatic Migration** |
2626
| --- | --- | --- | --- | --- | --- | --- | --- |
27-
| **Model** | **Claude 3: Opus** | Excellent | Excellent | Excellent | Good | Great | Good
28-
| | **Claude 3.7: Sonnet** | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent
29-
| | **Claude 3.5: Sonnet v2** | Excellent | Excellent | Excellent | Excellent | Great | Excellent
30-
| | **Claude 3.5: Sonnet** | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent
31-
| | **Claude 3.5: Haiku** | Excellent | Excellent | Excellent | Excellent | Poor | Poor
32-
| | **Claude 3: Haiku** | Excellent | Excellent | Excellent | Excellent | Poor | Poor
33-
| | **GPT-4o** | Excellent | Excellent | Excellent | Excellent | Great | Great
34-
| | **GPT-4o-mini** | Excellent | Great | Great | Great | Poor | Good
35-
| | **Gemini 1.5 Pro 002** | Excellent | Excellent | Excellent | Excellent | Excellent | Great
36-
| | **Gemini 1.5 Flash 002** | Excellent | Poor | Good | Excellent | Poor | Excellent
37-
| | **Gemini 2.0 Flash 001** | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent
27+
| **Model** | **Claude 3.7: Sonnet** | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent
28+
| | **Claude 3.5: Sonnet v2** | Excellent | Excellent | Excellent | Excellent | Great | Excellent
29+
| | **Claude 3.5: Sonnet** | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent
30+
| | **Claude 3.5: Haiku** | Excellent | Excellent | Excellent | Excellent | Poor | Poor
31+
| | **Claude 3: Haiku** | Excellent | Excellent | Excellent | Excellent | Poor | Poor
32+
| | **GPT-4o** | Excellent | Excellent | Excellent | Excellent | Great | Great
33+
| | **GPT-4o-mini** | Excellent | Great | Great | Great | Poor | Good
34+
| | **GPT-4.1** | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent
35+
| | **Gemini 1.5 Pro 002** | Excellent | Excellent | Excellent | Excellent | Excellent | Great
36+
| | **Gemini 1.5 Flash 002** | Excellent | Poor | Good | Excellent | Poor | Excellent
37+
| | **Gemini 2.0 Flash 001** | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent
38+
| | **Gemini 2.5 Pro** | Excellent | Excellent | Excellent | Excellent | Excellent | Excellent
3839

3940

4041
## Open-source models [_open_source_models]
@@ -43,7 +44,8 @@ Models you can [deploy yourself](/solutions/security/ai/connect-to-own-local-llm
4344

4445
| **Feature** | - | **Assistant - General** | **Assistant - {{esql}} generation** | **Assistant - Alert questions** | **Assistant - Knowledge retrieval** | **Attack Discovery** | **Automatic Migration**
4546
| --- | --- | --- | --- | --- | --- | --- |
46-
| **Model** | **Mistral Nemo** | Good | Good | Great | Good | Poor | Poor |
47-
| | **LLama 3.2** | Good | Poor | Good | Poor | Poor | Good |
48-
| | **LLama 3.1 405b** | Good | Great | Good | Good | Poor | Poor |
49-
| | **LLama 3.1 70b** | Good | Good | Poor | Poor | Poor | Good |
47+
| **Model** | **Mistral Nemo** | Good | Good | Great | Good | Poor | Poor |
48+
| | **Mistral-Small-3.1-24B-Instruct-2503** | Excellent | Poor | Excellent | Excellent | Good | N/A
49+
| | **LLama 3.2** | Good | Poor | Good | Poor | Poor | Good |
50+
| | **LLama 3.1 405b** | Good | Great | Good | Good | Poor | Poor |
51+
| | **LLama 3.1 70b** | Good | Good | Poor | Poor | Poor | Good |

0 commit comments

Comments
 (0)