You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Probably also in the test/ subdirectory of the repo
Should test a full end-to-end workflow: deploy to a separate server, clone/fetch/etc. with Git
Tests are designed around scenarios, written from the user's perspective - treat services as a black box. End-to-end tests should look like a PM wrote them, vs. integration tests designed by an engineer.
The text was updated successfully, but these errors were encountered:
I think the biggest hurdle here is deciding on a language & framework. Collecting a bunch of rough thoughts on the topic:
What kind of functionality do we want?
Since we're testing a bunch of CLIs (git-bundle-server, git-bundle-web-server, git), the language we use here should have relatively simple "call program" syntax; bonus points if it can be configured to run via shell.
For the sake of test reporting, whatever framework we use should be able to emit results in some kind of standardized format (e.g. junit).
Built-in global "setup" and "teardown" functions would be nice for cleaning up test repositories.
Should be configurable to run tests in parallel.
Which language?
Go
I'm actually leaning against Go, since using another language would really reinforce the "black-box" nature of end-to-end tests.
Shell
Shell tests are clearly the best choice for "ease of executing shell commands", but they'd involve a lot of boilerplate and, given our experience with Git's test suite, aren't particularly easy to work with. Leaning no on this one as well.
Typescript (Node.js)
This isn't a bad option, but personally it's not my first choice (npm is not my friend). That said, it's flexible and supports a number of fantastic testing frameworks.
Ruby
My personal top choice, mainly due to its extremely simple shell execution syntax. Like JS, there are a bunch of good test frameworks to choose from.
Test framework?
I really like Cucumber - tests are written in human-readable documents, those documents are portable across languages (should we want to change language), and most language integrations will output results in a standard format. I'm open to other ideas though!
Update: after doing some rough prototyping, I ended up on Typescript after all (mostly because its seems overall better supported in VSCode). It took some finagling to get Cucumber set up the way I wanted it, but at this point I'm able to run tests successfully. Hoping to have one or two scenarios implemented by the end of the week!
test/
subdirectory of the repoThe text was updated successfully, but these errors were encountered: