Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Find API suggestions for hallucinated attributes #88

Open
psaegert opened this issue Feb 14, 2024 · 0 comments
Open

Find API suggestions for hallucinated attributes #88

psaegert opened this issue Feb 14, 2024 · 0 comments
Assignees

Comments

@psaegert
Copy link
Collaborator

When LLMCoder hallucinates an attribute, get a list of existing attributes with Jedi and sort it by similarity to the hallucinated attribute.

Since transformer similarity is expensive at (on the order of 600 possible attributes as discovered by @KushalGaywala), we first try to use the Levenshtein distance at runtime to test the concept. Then, we try to set up an index of all attributes and their embeddings and only embed the hallucinated attribute at runtime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants