Unintuitive behavior when testing @Shared state

[nesium]

Hello!

While upgrading from 0.56.0 to 1.17.1 (🥵) I've replaced a couple of generic state wrappers with shared state. Now, trying to fix my tests, I somewhat struggle with - what I would call - pretty unintuitive behavior with @Shared state.

Let's assume this simple Reducer and State…

struct MySharedStruct: Equatable {
  var count = 0
  var items = [String]()
}

@Reducer
struct MyReducer {
  struct State: Equatable {
    @Shared var shared: MySharedStruct
  }

  enum Action {
    case addItem(String)
    case addTwoItems(String, String)
    
    case increaseCountByTwo
    case increaseCountThreeThenTwo
  }

  var body: some Reducer<State, Action> {
    Reduce { state, action in
      switch action {
      case let .addItem(item):
        state.$shared.withLock { $0.items.append(item) }
        return .none

      case let .addTwoItems(item1, item2):
        state.$shared.withLock { $0.items.append(item1) }
        return .send(.addItem(item2))
      
      case .increaseCountByTwo:
        state.$shared.withLock { $0.count += 2 }
        return .none
      
      case .increaseCountThreeThenTwo:
        state.$shared.withLock { $0.count += 3 }
        return .send(.increaseCountByTwo)
      }
    }
  }
}

Returning the send effects is quite contrived, but these could either be async responses from a HTTP API or actions triggered from long lived effects like a WebSocket.

Let's test the simple case first…

@Suite("Singles")
struct Singles {
  @Test func addItem() async throws {
    let store = await TestStore(initialState: .init(shared: Shared(value: .init()))) {
      MyReducer()
    }

    await store.send(.addItem("Item 1")) {
      $0.$shared.withLock { $0.items.append("Item 1") }
    }

    await store.assert {
      #expect($0.shared.items == ["Item 1"])
    }
  }
  
  @Test func increaseCount() async throws {
    let store = await TestStore(initialState: .init(shared: Shared(value: .init()))) {
      MyReducer()
    }
    
    await store.send(.increaseCountByTwo) {
      $0.$shared.withLock { $0.count += 2 }
    }
    
    await store.assert {
      #expect($0.shared.count == 2)
    }
  }
}

This looks perfectly reasonable. I send an action and assert the change. It works just like regular state with value semantics.

Now let's look at the second case where an effect is returned so that a second action is sent back into the reducer…

@Suite("Doubles")
struct Doubles {
  @Test func addTwoItems() async throws {
    let store = await TestStore(initialState: .init(shared: Shared(value: .init()))) {
      MyReducer()
    }

    await store.send(.addTwoItems("Item 1", "Item 2")) {
      // Somehow this would fail the test
      // $0.$shared.withLock { $0.items.append("Item 1") }
      $0.$shared.withLock { $0.items.append("Item 2") }
    }

    await store.receive(\.addItem)

    await store.assert {
      #expect($0.shared.items == ["Item 1", "Item 2"])
    }
  }
  
  @Test func increaseCountTwice() async throws {
    let store = await TestStore(initialState: .init(shared: Shared(value: .init()))) {
      MyReducer()
    }
    
    await store.send(.increaseCountThreeThenTwo) {
      // Just once?
      $0.$shared.withLock { $0.count += 2 }
    }

    await store.receive(\.increaseCountByTwo)
    
    await store.assert {
      #expect($0.shared.count == 5)
    }
  }
}

This is where it gets weird. While the store.assert works as expected, the assert closures in store.send and store.receive do not. Somehow I only have to assert the result of the second action sent to the reducer.

I'm not sure what to make of this. It feels like fudging the tests until they work and I'm afraid that it makes the tests difficult to understand. Am I missing something?

This appears to be partially related to this discussion: #3308

[stephencelis]

Hi @nesium! Thanks for the example and context.

This is a known issue and a thorny one. The test store was designed long before shared state, and it relies on eagerly performing all effects and snapshotting the changes to the value types along the way. Because @Shared is wrapped in a reference, snapshotting is a bit more complex, and eager evaluation basically puts the assertion in the position of always asserting against the most recent eagerly-evaluated effect and not the current received action.

We do have a branch showing how lazy evaluation would work, and if you point to it you'll see the tests behave how you expect:

https://github.com/pointfreeco/swift-composable-architecture/tree/lazy-test-store

Unfortunately this breaks other expectations when it comes to effect ordering, and so it would be a breaking change to tests.

We do think there is a path forward to fixing this, but it will take some time and might be a breaking change. In the meantime, at the very least the tests are deterministic, if unintuitive as you point out.

[nesium]

Thanks @stephencelis for confirming the issue and taking the time to explain the underlying mechanism. It clears things up a bit. What could take off the edge a bit is a more granular exhaustivity flag that would allow only non-shared changes to be asserted in the send and receive methods. That way people (like me) wouldn't create a second path that breaks with a future update. Would that make sense?