Wrong keyboard shown first time after focusing on a field shown conditionally

[fraune]

Hello, I hope you are doing well!

I'm not sure that this is an issue with this library, but I was curious to hear your thoughts.

Description

Background

SwiftUI supports conditionally showing views, by letting you place if-else or switch blocks directly in the view code. This allows SwiftUI to manage when an element should be displayed or removed from the view (other approaches involve modifying an element's opacity conditionally, but that element will still take up space when invisible).

Expectation

When using a conditional to switch to a TextField that has a custom keyboard set on it, I want to also focus the TextField to automatically present the keyboard. However, when doing so, the "standard" keyboard is presented instead. After toggling the conditional again, the TextField can present the custom keyboard.

I don't know how the introspect library works, but I'm guessing someone will tell me that conditional blocks in SwiftUI prevents a view from entering memory or something...

Demonstration

Code to replicate:

import SwiftUI
import CustomKeyboardKit

struct ContentView: View {
    @State private var selectedEntryType = EntryType.date
    @State private var textFieldContent = ""
    @FocusState private var textFieldFocused
    
    var body: some View {
        Form {
            Picker("Entry Type", selection: $selectedEntryType) {
                Text(EntryType.date.rawValue)
                    .tag(EntryType.date)
                Text(EntryType.binary.rawValue)
                    .tag(EntryType.binary)
            }
            .pickerStyle(.segmented)
            .onChange(of: selectedEntryType) { oldState, newState in
                if newState == .binary {
                    textFieldFocused = true
                }
            }
            
            switch selectedEntryType {
            case .date:
                DatePicker("some date", selection: Binding(get: { Date.now }, set: {_ in}))
            case .binary:
                TextField("title", text: $textFieldContent, axis: .vertical)
                    .customKeyboard(.binary)
                    .focused($textFieldFocused)
            }
        }
    }
}

private enum EntryType: String, CaseIterable {
    case date = "Date"
    case binary = "Binary"
}

extension CustomKeyboard {
    static var binary: CustomKeyboard {
        let keyWidth = 20.0
        let keyHeight = 34.0
        return CustomKeyboardBuilder { textDocumentProxy, submit, playSystemFeedback in
            HStack {
                Button {
                    textDocumentProxy.insertText("0")
                    playSystemFeedback?()
                } label: {
                    Text("0")
                        .frame(minWidth: keyWidth, minHeight: keyHeight)
                }
                Button {
                    textDocumentProxy.insertText("1")
                    playSystemFeedback?()
                } label: {
                    Text("1")
                        .frame(minWidth: keyWidth, minHeight: keyHeight)
                }
                .padding(.trailing, 10)
                
                Button {
                    textDocumentProxy.deleteBackward()
                    playSystemFeedback?()
                } label: {
                    Image(systemName: "delete.backward")
                        .frame(minHeight: keyHeight)
                }
                Button {
                    playSystemFeedback?()
                    submit()
                } label: {
                    Text("Done")
                        .frame(minHeight: keyHeight)
                }
            }
            .font(.title)
            .buttonStyle(.bordered)
            .padding([.top, .bottom], 20)
        }
    }
}

My workaround

This little hack is the only way I've found to resolve the issue so far.

// Updating the focus assignment in the demo code:
DispatchQueue.main.asyncAfter(deadline: .now() + 0.1) {
    textFieldFocused = true
}

[paescebu]

@fraune Thank you for that amazingly detailed and well formed bug report!
Damn, you got me there. This is a very tricky case.
I will investigate if i can find a change in my library to accomodate for that. Cause it seems SwiftUI introspect works flawlessly in your conditional View. So it must be something with the nature of CustomKeyboardKit that doesn't work properly.

An alternative approach (but also kinda hacky) would also be to (but that works only if in this page the TextField is the only TextField), to statically assign the custom Keyboard to all TextViews (due to your axis parameter it becomes a UITexView) on appearance, and assign again the default one on disappearance again.
Like so:

TextField("title", text: $textFieldContent, axis: .vertical)
    .onAppear {
        UITextView.appearance().inputView = CustomKeyboard.binary.keyboardInputView
    }
     .onDisappear {
        UITextView.appearance().inputView = nil
    }
    .focused($textFieldFocused)

In any case, this is not what and how we want to use CustomKeyboardKit.
I will try to find a better solution in the near future.

I will keep the issue open to let you know if i found something or if i eventually gave up :)

[paescebu]

Correction, there seems to be a limitation with the SwiftUI introspect library (currently).

import SwiftUI
import SwiftUIIntrospect

struct ContentView: View {
    @State private var isShown: Bool = false
    @FocusState private var textFieldFocused
    @State private var textFieldContent = ""

    var body: some View {
        Form {
            Toggle("Show Keyboard", isOn: $isShown)
            if isShown {
                TextField("title", text: $textFieldContent, axis: .vertical)
                    .introspect(.textEditor, on: .iOS(.v18), customize: { textView in
                        let view = UIView()
                        view.backgroundColor = .red
                        view.frame = .init(origin: .zero, size: .init(width: 200, height: 100))
                        textView.backgroundColor = .red
                        textView.inputView = view
                    })
                    .onAppear {
                        textFieldFocused = true
                    }
                    .focused($textFieldFocused)
            }
        }
    }
}

This seems to set the background of the TextField directly, but the inputView is not affected properly.
I will raise an Issue on SwiftUI Introspect, let's see if there's something they can do.

[paescebu]

Issue created:
siteline/swiftui-introspect#436

[paescebu]

A less hacky workaround that could be acceptable temporarily for you (and also less overhead than you have currently):

import SwiftUI
import CustomKeyboardKit

struct ContentView: View {
    @State private var selectedEntryType = EntryType.date
    @State private var textFieldContent = ""
    @FocusState private var textFieldFocused
    
    var body: some View {
        Form {
            Picker("Entry Type", selection: $selectedEntryType) {
                Text(EntryType.date.rawValue)
                    .tag(EntryType.date)
                Text(EntryType.binary.rawValue)
                    .tag(EntryType.binary)
            }
            .pickerStyle(.segmented)

            switch selectedEntryType {
            case .date:
                DatePicker("some date", selection: Binding(get: { Date.now }, set: {_ in}))
            case .binary:
                TextField("title", text: $textFieldContent, axis: .vertical)
                    .customKeyboard(.binary)
                    .focused($textFieldFocused)
                    .task {
                        textFieldFocused = true
                    }
            }
        }
    }
}

[paescebu]

Hi @fraune.
I investigated a bit and checked what can be done, for that i simplified introspection to only a simple TextField to soo where the problem might be lying. And I don't know, but maybe you have an idea.

Conceptually and to the bones it works like this:

import SwiftUI

struct ContentView: View {
    @State private var isShown: Bool = false
    @FocusState private var textFieldFocused
    @State private var textFieldContent = ""

    var body: some View {
        Form {
            Toggle("Show Keyboard", isOn: $isShown)
            if isShown {
                TextField("title", text: $textFieldContent)
                    .inputView { uiTextField in
                        let view = UIView()
                        view.backgroundColor = .red
                        view.frame = .init(origin: .zero, size: .init(width: 200, height: 100))
                        uiTextField.backgroundColor = .red
                        uiTextField.inputView = view
                        return
                    }
                    .onAppear {
                        textFieldFocused = true
                    }
                    .focused($textFieldFocused)
            }
        }
    }
}

/// Custom TextField Keyboard TextField Modifier
extension TextField {
    @ViewBuilder
    func inputView(@ViewBuilder content: @escaping (UITextField) -> ()) -> some View {
        self
            .background {
                TextFieldIntrospectionView(textFieldAction: content)
            }
    }
}

fileprivate struct TextFieldIntrospectionView: UIViewRepresentable {
    var textFieldAction: (UITextField) -> ()
    
    func makeUIView(context: Context) -> UIView {  return UIView() }
    
    func makeCoordinator() -> Coordinator { return .init() }
    
    func updateUIView(_ uiView: UIView, context: Context) {
        DispatchQueue.main.async {
            if let textFieldContainerView = uiView.superview?.superview {
                if let textField = textFieldContainerView.findViews(subclassOf: UITextField.self).first {
                    textFieldAction(textField)
                }
            }
        }
    }
    
    class Coordinator { }
}

extension UIView {
    func findViews<T: UIView>(subclassOf: T.Type) -> [T] {
        return recursiveSubviews.compactMap { $0 as? T }
    }

    var recursiveSubviews: [UIView] {
        return subviews + subviews.flatMap { $0.recursiveSubviews }
    }
}

The goal would be to adjust the solution SOMEHOW to have the red input view appear directly after the toggle was hit first.
Currently it only works the second time, but thats only because even if you hide it again, the textField is already in memory and it shows the same object again, after showing it again.
Feel free to chime in if you have a thoughts/ideas.

I might have an idea what we can do to improve CustomKeyboard kit to workaround this limitation

[paescebu]

Can you give branch: bugfix/issue_25_conditionally_revealed_textfields_wont_pickup_custom_keyboard
a try, this is roughly the workaround that should cover your case.

[paescebu]

I think it should be resolved now,
version 1.1.0 should be your new tag.

Feel free to close the issue if your problem is resolved.
Thank you again for the very detailed report!

[fraune]

Hi! Thanks for taking the time to respond and do so much work on this. I have been away from my computer and was surprised how much you got done on this in the meantime.

For the record, I wasn't aware of the task method in SwiftUI, but I found that was a pretty nice workaround.

Unfortunately, I think the 1.1.0 has a regression. It appears that although the keyboard is presented, when I edit the TextField, the data binding may not get updated.

Demo

In the demo, the Submit button sets the message with the text found in the input field.

Code to replicate (with CustomKeyboardKit 1.1.0 tag)

import SwiftUI
import CustomKeyboardKit

struct ContentView: View {
    @State private var selectedEntryType = EntryType.date
    @State private var textFieldContent = ""
    @FocusState private var textFieldFocused
    @State private var message = ""
    
    var body: some View {
        Form {
            Picker("Entry Type", selection: $selectedEntryType) {
                Text(EntryType.date.rawValue)
                    .tag(EntryType.date)
                Text(EntryType.binary.rawValue)
                    .tag(EntryType.binary)
            }
            .pickerStyle(.segmented)
            .onChange(of: selectedEntryType) { oldState, newState in
                if newState == .binary {
                    textFieldFocused = true
                }
            }
            
            switch selectedEntryType {
            case .date:
                DatePicker("some date", selection: Binding(get: { Date.now }, set: {_ in}))
            case .binary:
                TextField("title", text: $textFieldContent, axis: .vertical)
                    .customKeyboard(.binary)
                    .focused($textFieldFocused)
            }
            
            Button("Submit") {
                message = "text you entered: \"\(textFieldContent)\""
            }
            
            Section("Message") {
                Text(message)
            }
        }
    }
}

private enum EntryType: String, CaseIterable {
    case date = "Date"
    case binary = "Binary"
}

extension CustomKeyboard {
    static var binary: CustomKeyboard {
        let keyWidth = 20.0
        let keyHeight = 34.0
        return CustomKeyboardBuilder { textDocumentProxy, submit, playSystemFeedback in
            HStack {
                Button {
                    textDocumentProxy.insertText("0")
                    playSystemFeedback?()
                } label: {
                    Text("0")
                        .frame(minWidth: keyWidth, minHeight: keyHeight)
                }
                Button {
                    textDocumentProxy.insertText("1")
                    playSystemFeedback?()
                } label: {
                    Text("1")
                        .frame(minWidth: keyWidth, minHeight: keyHeight)
                }
                .padding(.trailing, 10)
                
                Button {
                    textDocumentProxy.deleteBackward()
                    playSystemFeedback?()
                } label: {
                    Image(systemName: "delete.backward")
                        .frame(minHeight: keyHeight)
                }
            }
            .font(.title)
            .buttonStyle(.bordered)
            .padding([.top, .bottom], 20)
        }
    }
}

[fraune]

As a side note: I know it may not be free to run a GitHub task for building/testing an iOS/Mac app, but I am open to contributing to this repo by adding a small example app and writing a few simple UI tests against it.

Those UI tests could be run locally as a verification step after adding/changing library code. And having a small example app might help with replicating new issues or discussing feature requests in the future.

But I'd only bother with something like that if you felt it would add value and not make things too difficult. If you want to chat about it more, we could discuss here or in another issue.

[paescebu]

Puhh,
you saved the day.
I reverted the release of 1.1.0.
I always thought this tiny library is so small and uncomplicated that it would survive without tests.
Once again i got humbled 😁.

I am confident though that i'll find a solution to your problem, without regressions, already have 1-2 prototypes, each a but quirky still. Still looking for a silver bullet here.

I love your idea of a UI Test. In fact for some private repositories i have a mac mini agent running github actions. After that disaster I am definitely open to the idea of a few UI Tests that smoke tests the basic functionalities. Also for PR contributions to run automatically. As long as my data security can somehow be assured.

Thanks again for your help!

[fraune]

No worries, that's how i feel about every project too. I'll see if I can put something together to help with the testing portion (if you don't beat me to it).

And thank YOU for your help! Honestly, no big rush on this issue either. For now, I'm happy with the .task workaround. I just wanted to reach out with this issue to see what your thoughts are before I curse my app code with this DispatchQueue.main.asyncAfter(deadline: .now() + 0.1) haha.

[paescebu]

Sometimes, those delayed dispatches are inevitable when working with the limited customizability of UI frameworks. Or at least, I’d go so far as to say that. Haha.

But I’m determined to avoid making it necessary, at least for the API users.

Absolutely, feel free to add some UI tests in a separate PR. No hurry on that, either.

Thanks for your efforts!
Let's keep this issue open though. :)

[fraune]

Awesome, thanks!

[paescebu]

Hey @fraune how is the latest commit on branch issue_25_conditionally_revealed_textfields_wont_pickup_custom_keyboard working for you? :)

[fraune]

I haven't had a chance to test yet, i hope get around to it by the weekend. I'll keep you posted

[paescebu]

No hurry, I pushed some new changes, I think I found a reasonable solution im happy with. Refactoring it a bit at the moment :)
Just hoping it passes your smoke test :P

[fraune]

Hi just wanted to say that I didn't forget about this - just haven't had much time to look at it. Will try to get back to this this weekend.

[paescebu]

Hey @fraune,
No issues!
We're not in a hurry.
This issue is going nowhere.

[paescebu]

Hey @fraune
Did you have any chance to give my latest changes a shot? 😁

Regards!

[fraune]

Hi – thanks for your patience.

I got a chance to test it out, and it worked exactly as expected! Amazing work, and I appreciate your effort as always.

[paescebu]

1.1.0 got released (again) :)
Let's see if it holds up this time

Thank you so much for your time and support!
Let me know if there are any further issues.

Kind regards
Pascal

[paescebu]

Thx for your donation! ❤️
Much appreciated!
Couldn't do it without such great reports and support :)

[fraune]

Of course! I'm honestly thrilled to support. You've done way more for my project than any other dev or library :) take care!

[paescebu]

Let me know about your app!
Would love to see how it's used :)