Cai-chat mode

[RedDragonGecko]

I just discovered this project thanks to nerdyrodent on YT. I jumped right in to running in cai chat mode and running llama-30b-4bit. Thats what sounded like the biggest and best model I could cram into my 4090.

My question is if this in fact the best model to use for characters or not. Do the other models have there own strengths and weaknesses in this mode? I can't t seem to find much information about this anywhere but if it's out there and I missed it please let me know.

[BetaDoggo]

The base LLaMa does a really good job on its own but it would probably do much better if it was finetuned on conversation like the dedicated chat models. From my limited testing it doesn't follow character cards as well as Pygmalion but the writing quality is far better which tends to make the conversation more cohesive and believable. If character details are important to you Pygmalion is definitely worth trying though.

[Ph0rk0z]

Ironically the alpaca-native is great and only 7b. I was surprised.

Open assistant 12b isn't bad either.

Models like the llama 30b may not be up on the lore. Especially for obscure characters.

[RedDragonGecko]

Thanks for the information my fellow ai enthusiasts. I'll have to try Pygmalion although from what you say it sounds like I want a mix of both.
My characters that I've tried so far and am more interested in aren't existing characters but new characters based more on personality and experiences.

Such as : a man in his late forties whom secretly wishes he was a cowboy. He works in a secret government project which has discovered that an asteroid is going to collide with the earth in a couple hours but he has been told to tell know one. Also he is deathly allergic to cheese.

So far limited success and a lot of weirdness. Their 'Personalities' seem more random than guided by their descriptions. for instance if I offered that man some cheese his response seems random despite what his character description would indicate.

I've also tried using character bias and while I think it does help in certain ways It seems to introduce some weirdness of its own such as making the ai replies always start with a series of periods or a face ............................................................... or 0_o
also character bias doesn't seem very flexible for instance if I use "is very aware the world is about to end" may work but if i want him to receive a call saying it was a false alarm... well you get the idea.

Anyways sorry for the long rambling. Just thought I'd share my experiences so far.