wundergraph_cli's print_schema::tests::round_trip test fails

[p-alik]

I've faced the issue by working on #48, but got it even by testing master branch based on 6c8cb7b

$ DATABASE_URL=wundergraph_cli.db cargo test --features sqlite --no-default-features  print_schema::tests::round_trip
   Compiling wundergraph_cli v0.1.1 (/opt/devel/rust/wundergraph/wundergraph_cli)
warning: use of deprecated item 'std::error::Error::description': use the Display impl or to_string()
   --> wundergraph_cli/src/infer_schema_internals/sqlite.rs:280:29
    |
280 |         Err(e) => assert!(e.description().starts_with(
    |                             ^^^^^^^^^^^
    |
    = note: `#[warn(deprecated)]` on by default

    Finished test [unoptimized + debuginfo] target(s) in 17.16s
     Running /opt/devel/rust/wundergraph/target/debug/deps/wundergraph_cli-95ce337ef33f77ec

running 1 test
     Created binary (application) `wundergraph_roundtrip_test` package
test print_schema::tests::round_trip ... test print_schema::tests::round_trip has been running for over 60 seconds
Started http server: http://127.0.0.1:8001
test print_schema::tests::round_trip ... FAILED

failures:

---- print_schema::tests::round_trip stdout ----
"[package]\nname = \"wundergraph_roundtrip_test\"\nversion = \"0.1.0\"\nauthors = [\"***\"]\nedition = \"2018\"\n\n# See more keys and their definitions at https://doc.r"
Updating crates.io index
Compiling libc v0.2.70
Compiling cfg-if v0.1.10
Compiling autocfg v1.0.0
Compiling proc-macro2 v1.0.13
Compiling unicode-xid v0.2.0
Compiling syn v1.0.22
Compiling byteorder v1.3.4
Compiling log v0.4.8
Compiling futures v0.1.29
Compiling lazy_static v1.4.0
Compiling semver-parser v0.7.0
Compiling scopeguard v1.1.0
Compiling fnv v1.0.7
Compiling maybe-uninit v2.0.0
Compiling smallvec v1.4.0
Compiling slab v0.4.2
Compiling autocfg v0.1.7
Compiling proc-macro2 v0.4.30
Compiling unicode-xid v0.1.0
Compiling rand_core v0.4.2
Compiling matches v0.1.8
Compiling syn v0.15.44
Compiling memchr v2.3.3
Compiling serde v1.0.110
Compiling getrandom v0.1.14
Compiling cc v1.0.53
Compiling regex-syntax v0.6.17
Compiling proc-macro-hack v0.5.15
Compiling failure_derive v0.1.8
Compiling itoa v0.4.5
Compiling gimli v0.21.0
Compiling object v0.19.0
Compiling rustc-demangle v0.1.16
Compiling percent-encoding v2.1.0
Compiling copyless v0.1.4
Compiling percent-encoding v1.0.1
Compiling match_cfg v0.1.0
Compiling linked-hash-map v0.5.3
Compiling crc32fast v1.2.0
Compiling ryu v1.0.4
Compiling quick-error v1.2.3
Compiling either v1.5.3
Compiling ppv-lite86 v0.2.8
Compiling pkg-config v0.3.17
Compiling httparse v1.3.4
Compiling arc-swap v0.4.6
Compiling encoding_rs v0.8.23
Compiling bitflags v1.2.1
Compiling dtoa v0.4.5
Compiling sha1 v0.6.0
Compiling language-tags v0.2.2
Compiling mime v0.3.16
Compiling uuid v0.7.4
Compiling crossbeam-utils v0.7.2
Compiling num-traits v0.2.11
Compiling num-integer v0.1.42
Compiling indexmap v1.3.2
Compiling thread_local v1.0.1
Compiling lock_api v0.3.4
Compiling semver v0.9.0
Compiling unicode-normalization v0.1.12
Compiling rand_core v0.3.1
Compiling rand_jitter v0.1.4
Compiling unicode-bidi v0.3.4
Compiling rand_chacha v0.1.1
Compiling rand_pcg v0.1.2
Compiling rand v0.6.5
Compiling hashbrown v0.6.3
Compiling tokio-sync v0.1.8
Compiling actix-service v0.4.2
Compiling miniz-sys v0.1.12
Compiling brotli-sys v0.3.2
Compiling addr2line v0.12.1
Compiling lru-cache v0.1.2
Compiling libsqlite3-sys v0.17.3
Compiling rustc_version v0.2.3
Compiling rand_xorshift v0.1.1
Compiling rand_hc v0.1.0
Compiling rand_isaac v0.1.1
Compiling idna v0.1.5
Compiling idna v0.2.0
Compiling quote v1.0.6
Compiling smallvec v0.6.13
Compiling iovec v0.1.4
Compiling net2 v0.2.34
Compiling num_cpus v1.13.0
Compiling rand_os v0.1.3
Compiling time v0.1.43
Compiling backtrace v0.3.48
Compiling hostname v0.3.1
Compiling socket2 v0.3.12
Compiling parking_lot_core v0.7.2
Compiling signal-hook-registry v1.2.0
Compiling base64 v0.10.1
Compiling parking_lot_core v0.6.2
Compiling parking_lot v0.9.0
Compiling quote v0.6.13
Compiling aho-corasick v0.7.10
Compiling url v1.7.2
Compiling url v2.1.1
Compiling tokio-executor v0.1.10
Compiling bytes v0.4.12
Compiling mio v0.6.22
Compiling threadpool v1.8.1
Compiling rand_core v0.5.1
Compiling const-random-macro v0.1.8
Compiling resolv-conf v0.6.3
Compiling parking_lot v0.10.2
Compiling flate2 v1.0.14
Compiling regex v1.3.7
Compiling tokio-timer v0.2.13
Compiling tokio-current-thread v0.1.7
Compiling chrono v0.4.11
Compiling tokio-io v0.1.13
Compiling http v0.1.21
Compiling string v0.2.1
Compiling rand_chacha v0.2.2
Compiling mio-uds v0.6.8
Compiling brotli2 v0.3.2
Compiling const-random v0.1.8
Compiling scheduled-thread-pool v0.2.4
Compiling synstructure v0.12.3
Compiling tokio-codec v0.1.2
Compiling enum-as-inner v0.2.1
Compiling derive_more v0.15.0
Compiling rand v0.7.3
Compiling ahash v0.2.18
Compiling r2d2 v0.8.8
Compiling actix-codec v0.1.2
Compiling serde_derive v1.0.110
Compiling diesel_derives v1.4.1
Compiling thiserror-impl v1.0.18
Compiling paste-impl v0.1.12
Compiling juniper_codegen v0.14.2
Compiling actix-web-codegen v0.1.3
Compiling wundergraph_derive v0.1.0 (/opt/devel/rust/wundergraph/wundergraph_derive)
Compiling tokio-reactor v0.1.12
Compiling actix-utils v0.4.7
Compiling thiserror v1.0.18
Compiling actix-threadpool v0.1.2
Compiling paste v0.1.12
Compiling tokio-tcp v0.1.4
Compiling tokio-udp v0.1.6
Compiling tokio-signal v0.2.9
Compiling actix-rt v0.2.6
Compiling diesel v1.4.4
Compiling actix-server-config v0.1.2
Compiling failure v0.1.8
Compiling actix-server v0.6.1
Compiling trust-dns-proto v0.7.4
Compiling actix-testing v0.1.0
Compiling serde_urlencoded v0.6.1
Compiling serde_json v1.0.53
Compiling actix-router v0.1.5
Compiling trust-dns-resolver v0.11.1
Compiling h2 v0.1.26
Compiling juniper v0.14.2
Compiling actix-connect v0.2.5
Compiling actix-http v0.2.11
Compiling wundergraph v0.1.2 (/opt/devel/rust/wundergraph/wundergraph)
Compiling awc v0.2.8
Compiling actix-web v1.0.9
Compiling wundergraph_roundtrip_test v0.1.0 (/tmp/roundtrip_test.knICxKiRUKHR/wundergraph_roundtrip_test)
Finished dev [unoptimized + debuginfo] target(s) in 2m 18s
Running `target/debug/wundergraph_roundtrip_test`
Started server
thread 'print_schema::tests::round_trip' panicked at 'called `Result::unwrap()` on an `Err` value: Error(Io(Custom { kind: TimedOut, error: "timed out" }), "http://127.0.0.1:8001/graphql")', wundergraph_cli/src1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    print_schema::tests::round_trip

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 9 filtered out

error: test failed, to rerun pass '--bin wundergraph_cli'

[weiznich]

Blind guess: It's caused by a to small sleep here (As this test passes with the CI)

[p-alik]

It's much more weird.
cargo test starts wundergraph_roundtrip_test process, which stays alive after test ends.

$ ss -lntp | grep 8001
LISTEN   0         128               127.0.0.1:8001             0.0.0.0:*        users:(("wundergraph_rou",pid=16591,fd=7))                                     
$ kill 16591
$ DATABASE_URL=wundergraph_cli.db cargo test --features sqlite --no-default-features  print_schema::tests::round_trip &>/dev/null 
$ ss -lntp | grep 8001
LISTEN   0         128               127.0.0.1:8001             0.0.0.0:*        users:(("wundergraph_rou",pid=21702,fd=7))

[weiznich]

So what this test does is basically creating and compiling a complete wundergraph example project. Then it "waits" for the project to be started and issues a few test requests to this server. All of that is done by creating a new cargo project in a temporary directory and executing the code there via cargo run. Your process output indicates that all of that is working fine, but the test cannot connect to that server for whatever reason.

(I think the process is not killed because I detached the process here and it is then explicitly killed here. Maybe that should be done via some drop guard 🤔 )

[p-alik]

I guess all assertions and unwraps have potential risk. Wouldn't it be good to run a part of rount_trip implementation in some try/catch block to ensure killing is executed at the end?

wundergraph/wundergraph_cli/src/print_schema/mod.rs

Line 348 in 6c8cb7b

child.kill().unwrap();

[weiznich]

Yes that should probably be done, but that will not fix your issue (only terminate the zombie processes)

[p-alik]

Either besides client termination there is an other issue with round_triptest or something is wrong with my local environment.
The round_triptest fails in my environment even for code based on 6c8cb7b:

$ DATABASE_URL=wundergraph_cli.db cargo test --features sqlite --no-default-features  print_schema::tests::round_trip
...
Compiling wundergraph_roundtrip_test v0.1.0 (/tmp/roundtrip_test.6CpPmVIXSacj/wundergraph_roundtrip_test)
Finished dev [unoptimized + debuginfo] target(s) in 2m 09s
Running `target/debug/wundergraph_roundtrip_test`
Started server
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Differences ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot file: wundergraph_cli/src/print_schema/snapshots/tests__round_trip-2.snap
Snapshot: round_trip-2
Source: wundergraph_cli/src/print_schema/mod.rs:335
-old snapshot
+new results
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
r.json::<serde_json::Value>().unwrap()
────────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
    1     1 │ {
    2       │-  "data": {
    3       │-    "CreateUser": {
    4       │-      "id": 1,
    5       │-      "name": "Max"
          2 │+  "errors": [
          3 │+    {
          4 │+      "locations": [
          5 │+        {
          6 │+          "column": 23,
          7 │+          "line": 2
          8 │+        }
          9 │+      ],
         10 │+      "message": "Invalid value for argument \"NewUser\", expected type \"NewUser!\""
    6    11 │     }
    7       │-  }
         12 │+  ]
    8    13 │ }
────────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

[p-alik]

The round_triptest fails in my environment even for code based on 6c8cb7b

test runs fine with b01985a

[weiznich]

Closed by #53