MacBook Pro 16" 2023 M3 Max 128Gb GPT4All settings

[art-lucas]

I own a MacBook Pro 16" 2023 M3 Max 128Gb. I set the following in GTP4All's settings:

• Device: Metal
• CPU Threads: 14

I'm running Bartowski's LlaMa Q6_K_L model with the following parameters:

• Context Length = 2028 => changed to 2200
• Prompt Batch Size = 128 => changed to 512
• Top-P = 0.4 (default)
• Min-P = 0 (default)
• GPU Layers = 80 (default)
• Max Length = 4096 (default)
• Temperature = 0.7 (default)
• Top-K = 40 (default)
• Repeat Penalty Tokens = 64 (default)
• Repeat Penalty = 1.18 (default)

I think the model parameters are fine for my use—though I'd appreciate comments—but I'm quite interested in the CPU Threads parameter: knowing that the M3 Max has 16-core CPU and 40-core GPU, and that in the settings themselves I set Device to Metal—which is GPU—does the CPU Threads parameter have any impact on the result/settings, i.e., does it become GPU Threads and could be set to a value of 1 to, say, max:40?

PS: Bartowski's LlaMa Q6_K_L is very slow on this MacBook while quite usable with Bartowski's LlaMa Q4_K. I'm tempted to try Q5_K_L and see if it's more usable.

[art-lucas]

I learned that if Metal is selected, CPU Threads has no influence on the behavior.