Support more training data structures in LoRA #191
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
davidkoski
reviewed
Feb 3, 2025
Libraries/MLXLLM/Lora+Data.swift
Outdated
| } | ||
|
|
||
| if let _ = try? decoder.decode(ToolStructure.self, from: data) { | ||
| return { _ in line } |
There was a problem hiding this comment.
this is not supported? Should it produce an error rather than silent failure?
davidkoski
reviewed
Feb 3, 2025
Libraries/MLXLLM/Lora+Data.swift
Outdated
| .components(separatedBy: .newlines) | ||
| .filter { | ||
| $0.first == "{" | ||
| private func determineDecoder(line: String) throws -> (Data) throws -> String { |
There was a problem hiding this comment.
Would it be valuable to return a protocol like LoRAData? Then the structs below could implement that along with the conversion to String. This would be useful for callers who wanted the structured data for some reason (if that seems reasonable).
Not strictly required I think because the training is going to compare string (token) outputs, right?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a draft pull request to add support for
chat,tool, andcompletiondata structures as documented in MLX-examples and discussed in #189. As of now it's still incomplete and the scope has expanded quite significantly since there needs to be a rewrite oftrain()to accommodate the different data structures.