Skip to content

Commit bd40499

Browse files
bitwisemanbitwiseman
committed
Improve behavior and documentation for WireMock testing
1 parent 0bb9f0c commit bd40499

File tree

70 files changed

+1959
-30
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

70 files changed

+1959
-30
lines changed

‎.github/workflows/maven.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,4 @@ jobs:
1616
- name: Maven Download all dependencies
1717
run: mvn -B org.apache.maven.plugins:maven-dependency-plugin:3.1.1:go-offline
1818
- name: Maven Build
19-
run: mvn -B package --file pom.xml -Dtest=GistTest,PullRequestTest,UserTest
19+
run: mvn -B package --file pom.xml -Dtest=CommitTest,GistTest,PullRequestTest,UserTest,WireMockStatusReporterTest

‎CONTRIBUTING.md

+62
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Contributing
2+
3+
## Using WireMock and Snapshots
4+
5+
This project has started converting to using WireMock to stub out http responses instead of use live data.
6+
This change will allow tests to run in a CI environment without needing to touch github.com.
7+
The tests will instead serve previously recorded responses from local data files.
8+
9+
### Running WireMock tests
10+
11+
Example:
12+
13+
`mvn install -Dtest=WireMockStatusReporterTest`
14+
15+
This the default behavior.
16+
17+
18+
### Setting up credential
19+
20+
1. Create an OAuth token on github.com
21+
2. Set the GITHUB_OAUTH environment variable to the value of that token
22+
3. Set the system property `test.github.useProxy` (usually like "-Dtest.github.useProxy" as a Java VM option)
23+
24+
`mvn install -Dtest.github.useProxy -Dtest=WireMockStatusReporterTest`
25+
26+
4. The above should report no test failures and include the following console output:
27+
28+
`WireMockStatusReporterTest: GitHub proxying and user auth correctly configured for user login: <your login>`
29+
30+
Whenever you run tests with `-Dtest.github.useProxy`, they will try to get data from local files but will fallback to proxying to github if not found.
31+
32+
33+
### Writing a new test
34+
35+
Once you have credentials setup, you add new test classes and test methods as you would normally.
36+
Keep `useProxy` enabled and iterate on your tests as needed. Remember, while proxying your tests are interacting with GitHub - you will need to clean up your state between runs.
37+
38+
When you are ready to create a snapshot of your test data,
39+
run your test with `test.github.takeSnapshot` ("-Dtest.github.takeSnapshot" as a Java VM option). For example:
40+
41+
`mvn install -Dtest.github.takeSnapshot -Dtest=YourTestClassName`
42+
43+
The above command would create snapshot WireMock data files under the path `src/test/resources/org/kohsuhke/github/YourTestClassName/wiremock`.
44+
Each method would get a separate director that would hold the data files for that test method.
45+
46+
Add all files including the generated data to your commit and submit a PR.
47+
48+
### Modifying existing tests
49+
50+
When modifying existing tests, you can change the stubbed WireMock data files by hand or you can try generating a new snapshot.
51+
52+
#### Manual editing of data (minor changes only)
53+
54+
If you know what data will change, it is sometimes simplest to make any required changes to the data files manually.
55+
This can be easier if the changes are minor or when you development environment is not setup to to take updated snapshots.
56+
57+
#### Generating a new snapshot
58+
59+
For more most changes, it is recommended to take a new snapshot when updating tests.
60+
Delete the wiremock data files for the test method you will be modifying.
61+
For more significant changes, you can even delete the WireMock files for an entire test class.
62+
Then follow the same as when writing a new test: run with proxy enabled to debug, take a new snapshot when done, commit everything, and submit the PR.

‎src/main/java/org/kohsuke/github/GitHubBuilder.java

+1-5
Original file line numberDiff line numberDiff line change
@@ -118,17 +118,13 @@ public static GitHubBuilder fromEnvironment(String loginVariableName, String pas
118118
* login, password, oauth
119119
*/
120120
public static GitHubBuilder fromEnvironment() throws IOException {
121-
return fromProperties(getPropertiesFromEnvironment());
122-
}
123-
124-
static Properties getPropertiesFromEnvironment() {
125121
Properties props = new Properties();
126122
for (Entry<String, String> e : System.getenv().entrySet()) {
127123
String name = e.getKey().toLowerCase(Locale.ENGLISH);
128124
if (name.startsWith("github_")) name=name.substring(7);
129125
props.put(name,e.getValue());
130126
}
131-
return props;
127+
return fromProperties(props);
132128
}
133129

134130
public static GitHubBuilder fromPropertyFile() throws IOException {

‎src/test/java/org/kohsuke/github/AbstractGitHubApiWireMockTest.java

+27-22
Original file line numberDiff line numberDiff line change
@@ -21,10 +21,21 @@
2121
*/
2222
public abstract class AbstractGitHubApiWireMockTest extends Assert {
2323

24+
// By default the wiremock tests will, run without proxy or taking a snapshot.
25+
// The tests will use only the stubbed data and will fail if requests are made for missing data.
26+
// You can use the proxy without taking a snapshot while writing and debugging tests.
27+
// You cannot take a snapshot without proxying.
28+
protected final static boolean takeSnapshot = System.getProperty("test.github.takeSnapshot", "false") != "false";
29+
protected final static boolean useProxy = takeSnapshot || System.getProperty("test.github.useProxy", "false") != "false";
30+
31+
public final static String STUBBED_USER_LOGIN = "placeholder-user";
32+
public final static String STUBBED_USER_PASSWORD = "placeholder-password";
33+
2434
protected GitHub gitHub;
2535
private final String baseFilesClassPath = this.getClass().getName().replace('.', '/');
2636
protected final String baseRecordPath = "src/test/resources/" + baseFilesClassPath + "/wiremock";
27-
private boolean takeSnapshot = false;
37+
38+
2839

2940
@Rule
3041
public WireMockRule githubApi = new WireMockRule(WireMockConfiguration.options()
@@ -51,44 +62,38 @@ public Response transform(Request request, Response response, FileSource files,
5162

5263
@Override
5364
public String getName() {
54-
return "url-rewrite";
65+
return "github-api-url-rewrite";
5566
}
5667
})
5768
);
5869

5970
@Before
6071
public void wireMockSetup() throws Exception {
6172

62-
Properties props = GitHubBuilder.getPropertiesFromEnvironment();
73+
GitHubBuilder builder = GitHubBuilder.fromEnvironment()
74+
.withEndpoint("http://localhost:" + githubApi.port())
75+
.withRateLimitHandler(RateLimitHandler.FAIL);
6376

64-
// By default the wiremock tests will proxy to github transparently without taking snapshots
65-
// This lets you debug any
66-
if(Boolean.parseBoolean(props.getProperty("snapshot", "false"))) {
67-
takeSnapshot = true;
68-
}
6977

70-
if(!props.containsKey("oauth") && !takeSnapshot) {
71-
// This sets the oauth token to a placeholder wiremock tests
78+
if(useProxy) {
79+
githubApi.stubFor(
80+
proxyAllTo("https://api.github.com/")
81+
.atPriority(100)
82+
);
83+
} else {
84+
// This sets the user and password to a placeholder for wiremock testing
7285
// This makes the tests believe they are running with permissions
7386
// The recorded stubs will behave like they running with permissions
74-
props.setProperty("oauth", "placeholder-will-fail-when-not-mocking");
87+
builder.withPassword(STUBBED_USER_LOGIN, STUBBED_USER_PASSWORD);
7588

89+
// Just to be super clear
7690
githubApi.stubFor(
7791
any(urlPathMatching(".*"))
78-
.willReturn(status(500))
92+
.willReturn(status(500).withBody("Stubbed data not found. Set test.github.use-proxy to have WireMock proxy to github"))
7993
.atPriority(100));
8094
}
8195

82-
githubApi.stubFor(
83-
proxyAllTo("https://api.github.com/")
84-
.atPriority(100)
85-
);
86-
87-
gitHub = GitHubBuilder.fromProperties(props)
88-
.withEndpoint("http://localhost:" + githubApi.port())
89-
.withRateLimitHandler(RateLimitHandler.FAIL)
90-
.build();
91-
96+
gitHub = builder.build();
9297
}
9398

9499
@After

‎src/test/java/org/kohsuke/github/CommitTest.java

+2-2
Original file line numberDiff line numberDiff line change
@@ -8,11 +8,11 @@
88
/**
99
* @author Kohsuke Kawaguchi
1010
*/
11-
public class CommitTest extends AbstractGitHubApiTestBase {
11+
public class CommitTest extends AbstractGitHubApiWireMockTest {
1212
@Test // issue 152
1313
public void lastStatus() throws IOException {
1414
GHTag t = gitHub.getRepository("stapler/stapler").listTags().iterator().next();
15-
t.getCommit().getLastStatus();
15+
assertNotNull(t.getCommit().getLastStatus());
1616
}
1717

1818
@Test // issue 230

‎src/test/java/org/kohsuke/github/WireMockStatusReporterTest.java

+137
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
package org.kohsuke.github;
2+
3+
import org.kohsuke.github.junit.WireMockRule;
4+
import org.hamcrest.Matchers;
5+
import org.junit.Ignore;
6+
import org.junit.Test;
7+
import static org.hamcrest.Matchers.*;
8+
import static org.junit.Assume.assumeFalse;
9+
import static org.junit.Assume.assumeTrue;
10+
11+
12+
/**
13+
* Tests in this class are meant to show the behavior of {@link AbstractGitHubApiWireMockTest} with proxying on or off.
14+
*
15+
* The wiremock data for these tests should only be modified by hand - thus most are skipped when snapshotting.
16+
*
17+
* @author Liam Newman
18+
*/
19+
public class WireMockStatusReporterTest extends AbstractGitHubApiWireMockTest {
20+
21+
@Test
22+
public void user_whenProxying_AuthCorrectlyConfigured() throws Exception {
23+
assumeFalse("Test only valid when not taking a snapshot", takeSnapshot);
24+
assumeTrue("Test only valid when proxying (-Dtest.github.useProxy to enable)", useProxy);
25+
26+
assertThat(
27+
"GitHub connection believes it is anonymous. Make sure you set GITHUB_OAUTH or both GITHUB_USER and GITHUB_PASSWORD environment variables",
28+
gitHub.isAnonymous(), is(false));
29+
30+
assertThat(gitHub.login, not(equalTo(STUBBED_USER_LOGIN)));
31+
32+
// If this user query fails, either the proxying config has broken (unlikely)
33+
// or your auth settings are not being retrieved from the environemnt.
34+
// Check your settings.
35+
GHUser user = gitHub.getMyself();
36+
assertThat(user.getLogin(), notNullValue());
37+
38+
System.out.println();
39+
System.out.println("WireMockStatusReporterTest: GitHub proxying and user auth correctly configured for user login: " + user.getLogin());
40+
System.out.println();
41+
}
42+
43+
@Test
44+
public void user_whenNotProxying_Stubbed() throws Exception {
45+
assumeFalse("Test only valid when not taking a snapshot", takeSnapshot);
46+
assumeFalse("Test only valid when not proxying", useProxy);
47+
48+
assertThat(gitHub.isAnonymous(), is(false));
49+
assertThat(gitHub.login, equalTo(STUBBED_USER_LOGIN));
50+
51+
GHUser user = gitHub.getMyself();
52+
// NOTE: the stubbed user does not have to match the login provided from the github object
53+
// github.login is literally just a placeholder when mocking
54+
assertThat(user.getLogin(), not(equalTo(STUBBED_USER_LOGIN)));
55+
assertThat(user.getLogin(), equalTo("stubbed-user-login"));
56+
57+
System.out.println("GitHub proxying and user auth correctly configured for user login: " + user.getLogin());
58+
}
59+
60+
@Test
61+
public void BasicBehaviors_whenNotProxying() throws Exception {
62+
assumeFalse("Test only valid when not taking a snapshot", takeSnapshot);
63+
assumeFalse("Test only valid when not proxying", useProxy);
64+
65+
Exception e = null;
66+
GHRepository repo = null;
67+
68+
// Valid repository, stubbed
69+
repo = gitHub.getRepository("github-api/github-api");
70+
assertThat(repo.getDescription(), equalTo("this is a stubbed description"));
71+
72+
// Valid repository, without stub - fails 500 when not proxying
73+
try {
74+
gitHub.getRepository("jenkinsci/jenkins");
75+
fail();
76+
} catch (Exception ex) {
77+
e = ex;
78+
}
79+
assertThat(e, Matchers.<Exception>instanceOf(HttpException.class));
80+
assertThat("Status should be 500 for missing stubs", ((HttpException)e).getResponseCode(), equalTo(500));
81+
assertThat(e.getMessage(), equalTo("Stubbed data not found. Set test.github.use-proxy to have WireMock proxy to github"));
82+
83+
// Invalid repository, without stub - fails 500 when not proxying
84+
e = null;
85+
try {
86+
gitHub.getRepository("github-api/non-existant-repository");
87+
fail();
88+
} catch (Exception ex) {
89+
e = ex;
90+
}
91+
92+
assertThat(e, Matchers.<Exception>instanceOf(HttpException.class));
93+
assertThat("Status should be 500 for missing stubs", ((HttpException)e).getResponseCode(), equalTo(500));
94+
assertThat(e.getMessage(), equalTo("Stubbed data not found. Set test.github.use-proxy to have WireMock proxy to github"));
95+
}
96+
97+
@Test
98+
public void BasicBehaviors_whenProxying() throws Exception {
99+
assumeFalse("Test only valid when not taking a snapshot", takeSnapshot);
100+
assumeTrue("Test only valid when proxying (-Dtest.github.useProxy to enable)", useProxy);
101+
Exception e = null;
102+
GHRepository repo = null;
103+
104+
// Valid repository, stubbed
105+
repo = gitHub.getRepository("github-api/github-api");
106+
assertThat(repo.getDescription(), equalTo("this is a stubbed description"));
107+
108+
// Valid repository, without stub - succeeds when proxying
109+
repo = gitHub.getRepository("jenkinsci/jenkins");
110+
assertThat(repo.getDescription(), notNullValue());
111+
112+
// Invalid repository, without stub - fails 404 when proxying
113+
e = null;
114+
try {
115+
gitHub.getRepository("github-api/non-existant-repository");
116+
} catch (Exception ex) {
117+
e = ex;
118+
}
119+
120+
assertThat(e, Matchers.<Exception>instanceOf(GHFileNotFoundException.class));
121+
assertThat(e.getMessage(), equalTo("{\"message\":\"Not Found\",\"documentation_url\":\"https://developer.github.com/v3/repos/#get\"}"));
122+
}
123+
124+
@Test
125+
public void whenSnapshot_EnsureProxy() throws Exception {
126+
assumeTrue("Test only valid when Snapshotting (-Dtest.github.takeSnapshot to enable)", takeSnapshot);
127+
128+
assertTrue("When taking a snapshot, proxy should automatically be enabled", useProxy);
129+
}
130+
131+
@Ignore("Not implemented yet")
132+
@Test
133+
public void whenSnapshot_EnsureRecordToExpectedLocation() throws Exception {
134+
assumeTrue("Test only valid when Snapshotting (-Dtest.github.takeSnapshot to enable)", takeSnapshot);
135+
136+
}
137+
}

‎src/test/resources/org/kohsuke/github/CommitTest/wiremock/lastStatus/__files/repos_stapler_stapler-dea29c4a-c5b6-460d-8550-09d75d487830.json

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"id":1548514,"node_id":"MDEwOlJlcG9zaXRvcnkxNTQ4NTE0","name":"stapler","full_name":"stapler/stapler","private":false,"owner":{"login":"stapler","id":700341,"node_id":"MDEyOk9yZ2FuaXphdGlvbjcwMDM0MQ==","avatar_url":"https://avatars1.githubusercontent.com/u/700341?v=4","gravatar_id":"","url":"https://api.github.com/users/stapler","html_url":"https://github.com/stapler","followers_url":"https://api.github.com/users/stapler/followers","following_url":"https://api.github.com/users/stapler/following{/other_user}","gists_url":"https://api.github.com/users/stapler/gists{/gist_id}","starred_url":"https://api.github.com/users/stapler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stapler/subscriptions","organizations_url":"https://api.github.com/users/stapler/orgs","repos_url":"https://api.github.com/users/stapler/repos","events_url":"https://api.github.com/users/stapler/events{/privacy}","received_events_url":"https://api.github.com/users/stapler/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/stapler/stapler","description":"Stapler web framework","fork":false,"url":"https://api.github.com/repos/stapler/stapler","forks_url":"https://api.github.com/repos/stapler/stapler/forks","keys_url":"https://api.github.com/repos/stapler/stapler/keys{/key_id}","collaborators_url":"https://api.github.com/repos/stapler/stapler/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/stapler/stapler/teams","hooks_url":"https://api.github.com/repos/stapler/stapler/hooks","issue_events_url":"https://api.github.com/repos/stapler/stapler/issues/events{/number}","events_url":"https://api.github.com/repos/stapler/stapler/events","assignees_url":"https://api.github.com/repos/stapler/stapler/assignees{/user}","branches_url":"https://api.github.com/repos/stapler/stapler/branches{/branch}","tags_url":"https://api.github.com/repos/stapler/stapler/tags","blobs_url":"https://api.github.com/repos/stapler/stapler/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/stapler/stapler/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/stapler/stapler/git/refs{/sha}","trees_url":"https://api.github.com/repos/stapler/stapler/git/trees{/sha}","statuses_url":"https://api.github.com/repos/stapler/stapler/statuses/{sha}","languages_url":"https://api.github.com/repos/stapler/stapler/languages","stargazers_url":"https://api.github.com/repos/stapler/stapler/stargazers","contributors_url":"https://api.github.com/repos/stapler/stapler/contributors","subscribers_url":"https://api.github.com/repos/stapler/stapler/subscribers","subscription_url":"https://api.github.com/repos/stapler/stapler/subscription","commits_url":"https://api.github.com/repos/stapler/stapler/commits{/sha}","git_commits_url":"https://api.github.com/repos/stapler/stapler/git/commits{/sha}","comments_url":"https://api.github.com/repos/stapler/stapler/comments{/number}","issue_comment_url":"https://api.github.com/repos/stapler/stapler/issues/comments{/number}","contents_url":"https://api.github.com/repos/stapler/stapler/contents/{+path}","compare_url":"https://api.github.com/repos/stapler/stapler/compare/{base}...{head}","merges_url":"https://api.github.com/repos/stapler/stapler/merges","archive_url":"https://api.github.com/repos/stapler/stapler/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/stapler/stapler/downloads","issues_url":"https://api.github.com/repos/stapler/stapler/issues{/number}","pulls_url":"https://api.github.com/repos/stapler/stapler/pulls{/number}","milestones_url":"https://api.github.com/repos/stapler/stapler/milestones{/number}","notifications_url":"https://api.github.com/repos/stapler/stapler/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/stapler/stapler/labels{/name}","releases_url":"https://api.github.com/repos/stapler/stapler/releases{/id}","deployments_url":"https://api.github.com/repos/stapler/stapler/deployments","created_at":"2011-03-30T22:39:45Z","updated_at":"2019-08-27T16:42:33Z","pushed_at":"2019-08-19T18:47:57Z","git_url":"git://github.com/stapler/stapler.git","ssh_url":"[email protected]:stapler/stapler.git","clone_url":"https://github.com/stapler/stapler.git","svn_url":"https://github.com/stapler/stapler","homepage":"http://stapler.kohsuke.org/","size":41906,"stargazers_count":112,"watchers_count":112,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":75,"mirror_url":null,"archived":false,"disabled":false,"open_issues_count":28,"license":{"key":"bsd-2-clause","name":"BSD 2-Clause \"Simplified\" License","spdx_id":"BSD-2-Clause","url":"https://api.github.com/licenses/bsd-2-clause","node_id":"MDc6TGljZW5zZTQ="},"forks":75,"open_issues":28,"watchers":112,"default_branch":"master","permissions":{"admin":false,"push":false,"pull":true},"organization":{"login":"stapler","id":700341,"node_id":"MDEyOk9yZ2FuaXphdGlvbjcwMDM0MQ==","avatar_url":"https://avatars1.githubusercontent.com/u/700341?v=4","gravatar_id":"","url":"https://api.github.com/users/stapler","html_url":"https://github.com/stapler","followers_url":"https://api.github.com/users/stapler/followers","following_url":"https://api.github.com/users/stapler/following{/other_user}","gists_url":"https://api.github.com/users/stapler/gists{/gist_id}","starred_url":"https://api.github.com/users/stapler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stapler/subscriptions","organizations_url":"https://api.github.com/users/stapler/orgs","repos_url":"https://api.github.com/users/stapler/repos","events_url":"https://api.github.com/users/stapler/events{/privacy}","received_events_url":"https://api.github.com/users/stapler/received_events","type":"Organization","site_admin":false},"network_count":75,"subscribers_count":12}

0 commit comments

Comments
 (0)